summaryrefslogtreecommitdiff
path: root/bolt/lib/Core/BinaryFunction.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [BOLT][NFC] Add hash computation for basic blocksspupyrev2023-05-021-25/+10
| | | | | | | | | | Extending yaml profile format with block hashes, which are used for stale profile matching. To avoid duplication of the code, created a new class with a collection of utilities for computing hashes. Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D144306
* [BOLT] Fix creation of invalid CFG in presence of dead codeRafael Auler2023-04-111-0/+1
| | | | | | | | | | | | | | When there is a direct jump right after an indirect one, in the absence of code jumpting to this direct jump, this is obviously dead code. However, BOLT was failing to recognize that by mistakenly placing both jmp instructions in the same basic block, and creating wrong successor edges. Fix that, so we can safely run UCE on that. This bug also causes validateCFG to fail and BOLT to crash if it is running ICP on that function. Reviewed By: #bolt, Amir Differential Revision: https://reviews.llvm.org/D148055
* [MC] Always encode instruction into SmallVectorAlexis Engelke2023-04-061-2/+1
| | | | | | | | | | | | | | | All users of MCCodeEmitter::encodeInstruction use a raw_svector_ostream to encode the instruction into a SmallVector. The raw_ostream however incurs some overhead for the actual encoding. This change allows an MCCodeEmitter to directly emit an instruction into a SmallVector without using a raw_ostream and therefore allow for performance improvments in encoding. A default path that uses existing raw_ostream implementations is provided. Reviewed By: MaskRay, Amir Differential Revision: https://reviews.llvm.org/D145791
* [BOLT] computing raw branch count for yaml profilesspupyrev2023-03-281-0/+1
| | | | | | | | | `Function.RawBranchCount` is initialized for fdata profile but not for yaml one. The diff adds the computation of the field for yaml profiles Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D144211
* Use *{Map,Set}::contains (NFC)Kazu Hirata2023-03-151-3/+2
|
* [BOLT][NFC] Move addRelocation{X86,AArch64} into MCPlusBuilderAmir Ayupov2023-03-141-0/+16
| | | | | | | | | The two methods don't belong in BinaryFunction methods. Move the dispatch tables into target-specific MCPlusBuilder methods. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D131813
* [BOLT][NFC] Use llvm::is_containedAmir Ayupov2023-03-141-1/+1
| | | | | | | | Apply the replacement throughout BOLT. Reviewed By: #bolt, rafauler Differential Revision: https://reviews.llvm.org/D145464
* [BOLT] Improve dynamic relocations support for CIVladislav Khmelevsky2023-03-131-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes few problems with supporting dynamic relocations in CI. 1. After dynamic relocations and functions were read search for dynamic relocations located in functions. Currently we expected them only to be relative and only to be in constant island. Mark islands of such functions to have dynamic relocations and create CI access symbol on the relocation offset, so the BD would be created for such place. 2. During function disassemble and handling address reference for constant island check if the referred external CI has dynamic relocation. And if it has one we would continue to refer original CI rather then creating a local copy. 3. After function disassembly stage mark function that has dynamic reloc in CI as non-simple. We don't want such functions to be optimized, since such passes as split function would create 2 copies of CI which we unable to support currently. 4. During updating output values for BF search for BD located in CI and update their output locations. 5. On dynamic relocation patching stage search for binary data located on relocation offset. If it was moved use new relocation offset value rather then an old one. Vladislav Khmelevsky, Advanced Software Technology Lab, Huawei Differential Revision: https://reviews.llvm.org/D143748
* [BOLT][NFC] Simplify BinaryFunction::setTrapOnEntryAmir Ayupov2023-02-271-7/+3
| | | | | | Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D144758
* [BOLT] Prevent unsetting unknown control flow for split jump tableAmir Ayupov2023-02-271-1/+6
| | | | | | | | | | | | | | In case of a function with unknown control flow but with a single jump table and a single jump table site, we attempt to match the jump table and a site and update block successors using jump table targets. Restrict this behavior for split jump tables which have targets in a fragment function. Fixes https://github.com/llvm/llvm-project/issues/60795. Reviewed By: #bolt, rafauler Differential Revision: https://reviews.llvm.org/D144602
* [BOLT][NFC] Replace anonymous namespace functions with staticAmir Ayupov2023-02-061-20/+12
| | | | | | | | | | Follow LLVM Coding Standards guideline on using anonymous namespaces (https://llvm.org/docs/CodingStandards.html#anonymous-namespaces) and use `static` modifier for function definitions. Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D143124
* [BOLT][NFC] Replace ambiguous BinarySection::isReadOnly with isWritableAmir Ayupov2023-01-181-1/+1
| | | | | | | | Address feedback in https://reviews.llvm.org/D102284#2755060 Reviewed By: yota9 Differential Revision: https://reviews.llvm.org/D141733
* [BOLT][NFC] Use llvm::reverseAmir Ayupov2023-01-031-5/+6
| | | | | | | | Use llvm::reverse instead of `for (auto I = rbegin(), E = rend(); I != E; ++I)` Reviewed By: #bolt, rafauler Differential Revision: https://reviews.llvm.org/D140516
* [BOLT][NFC] Remove unused PrintInstructions argumentMaksim Panchenko2022-12-201-5/+4
| | | | | | | | PrintInstructions was unused in BinaryFunction::print() and dump(). Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D140440
* [BOLT][NFC] Use std::optional for readDWARFExpressionTargetRegAmir Ayupov2022-12-111-3/+6
|
* [BOLT][NFC] Use std::optional in has*NameRegexAmir Ayupov2022-12-111-8/+5
|
* [BOLT][NFC] Use std::optional in MCPlusBuilderAmir Ayupov2022-12-061-3/+4
| | | | | | Reviewed By: maksfb, #bolt Differential Revision: https://reviews.llvm.org/D139260
* [BOLT] Fix instruction encoding validationMaksim Panchenko2022-10-181-8/+3
| | | | | | | | | | Always use non-symbolizing disassembler for instruction encoding validation as symbols will be treated as undefined/zeros be the encoder and causing byte sequence mismatches. Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D136118
* [BOLT][NFC] Refactor creation of symbol+addend referencesRafael Auler2022-10-121-10/+5
| | | | | | | | | Put code that creates references to symbol+addend behind MCPlusBuilder. Will use this later in validate memory references pass. Reviewed By: #bolt, maksfb, yota9 Differential Revision: https://reviews.llvm.org/D134097
* [BOLT] Verify externally referenced blocks against jump table targetsAmir Ayupov2022-09-161-0/+45
| | | | | | | | | | For functions with references to internal offsets from data, verify externally referenced blocks against the set of jump table targets. Mark the function as non-simple if there are any unclaimed data to code references. Reviewed By: #bolt, maksfb Differential Revision: https://reviews.llvm.org/D132495
* [BOLT] Emit LSDA call sites for all fragmentsFabian Parzefall2022-09-081-7/+10
| | | | | | | | | | | For exception handling, LSDA call sites have to be emitted for each fragment individually. With this patch, call sites and respective LSDA symbols are generated and associated with each fragment of their function, such that they can be used by the emitter. Reviewed By: maksfb Differential Revision: https://reviews.llvm.org/D132052
* [BOLT] Use range-based for loops (NFC)Kazu Hirata2022-09-031-2/+4
| | | | | LLVM Coding Standards discourage for_each unless callable objects already exist.
* [BOLT][NFC] Use llvm::any_ofAmir Ayupov2022-08-271-7/+3
| | | | | | | | | | | | | | | | | | | | | | | Replace the imperative pattern of the following kind ``` bool IsTrue = false; for (Element : Range) { if (Condition(Element)) { IsTrue = true; break; } } ``` with functional style `llvm::any_of`: ``` bool IsTrue = llvm::any_of(Range, [&](Element) { return Condition(Element); }); ``` Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132276
* [BOLT] Track fragment info for all split fragmentsFabian Parzefall2022-08-241-40/+36
| | | | | | | | | | | To generate all symbols correctly, it is necessary to record the address of each fragment. This patch moves the address info for the main and cold fragments from BinaryFunction to FunctionFragment, where this data is recorded for all fragments. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132051
* [BOLT] Allocate FunctionFragment on heapFabian Parzefall2022-08-241-3/+3
| | | | | | | | | | This changes `FunctionFragment` from being used as a temporary proxy object to access basic block ranges to a heap-allocated object that can store fragment-specific information. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132050
* [BOLT] Towards FunctionLayout const-correctnessFabian Parzefall2022-08-241-4/+13
| | | | | | | | | | | A const-qualified reference to function layout allows accessing non-const qualified basic blocks on a const-qualified function. This patch adds or removes const-qualifiers where necessary to indicate where basic blocks are used in a non-const manner. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132049
* Revert "[BOLT] Towards FunctionLayout const-correctness"Fabian Parzefall2022-08-241-13/+4
| | | | This reverts commit 587d2653420d75ef10f30bd612d86f1e08fe9ea7.
* Revert "[BOLT] Allocate FunctionFragment on heap"Fabian Parzefall2022-08-241-3/+3
| | | | This reverts commit 101344af1af82d1633c773b718788eaa813d7f79.
* Revert "[BOLT] Track fragment info for all split fragments"Fabian Parzefall2022-08-241-36/+40
| | | | This reverts commit 7e254818e49454a53bd00e3737007025b62d0f79.
* [BOLT] Track fragment info for all split fragmentsFabian Parzefall2022-08-241-40/+36
| | | | | | | | | | | To generate all symbols correctly, it is necessary to record the address of each fragment. This patch moves the address info for the main and cold fragments from BinaryFunction to FunctionFragment, where this data is recorded for all fragments. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132051
* [BOLT] Allocate FunctionFragment on heapFabian Parzefall2022-08-241-3/+3
| | | | | | | | | | This changes `FunctionFragment` from being used as a temporary proxy object to access basic block ranges to a heap-allocated object that can store fragment-specific information. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132050
* [BOLT] Towards FunctionLayout const-correctnessFabian Parzefall2022-08-241-4/+13
| | | | | | | | | | | A const-qualified reference to function layout allows accessing non-const qualified basic blocks on a const-qualified function. This patch adds or removes const-qualifiers where necessary to indicate where basic blocks are used in a non-const manner. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132049
* [BOLT][NFC] Move out handleAArch64IndirectCallAmir Ayupov2022-08-231-22/+22
| | | | | | | | | Move the large lambda out of BinaryFunction::disassemble, reducing its size from 255 to 233 LoC. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132104
* [BOLT][NFC] Move out handleIndirectBranchAmir Ayupov2022-08-231-40/+41
| | | | | | | Move the large lambda out of BinaryFunction::disassemble, reducing its size from 295 to 255 LoC. Differential Revision: https://reviews.llvm.org/D132101
* [BOLT][NFC] Move out handleExternalReferenceAmir Ayupov2022-08-231-43/+45
| | | | | | | | | Move the large lambda out of BinaryFunction::disassemble, reducing its size from 338 to 295 LoC. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132100
* [BOLT][NFC] Move out handlePCRelOperandAmir Ayupov2022-08-231-39/+40
| | | | | | | | | Move the large lambda out of BinaryFunction::disassemble, reducing its size from 377 to 338 LoC. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132099
* [BOLT] Generate sections for multiple fragmentsFabian Parzefall2022-08-181-40/+61
| | | | | | | | | | | | | | | | This patch adds support to generate any number of sections that are assigned to fragments of functions that are split more than two-way. With this, a function's *nth* split fragment goes into section `.text.cold.n`. This also changes `FunctionLayout::erase` to make sure, that there are no empty fragments at the end of the function. This sometimes happens when blocks are erased from the function. To avoid creating symbols pointing to these fragments, they need to be removed. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D130521
* [BOLT] Make exception handling fragment awareFabian Parzefall2022-08-181-5/+8
| | | | | | | | | This adds basic fragment awareness in the exception handling passes and generates the necessary symbols for fragments. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D130520
* [BOLT] Support passing fragments to code emissionFabian Parzefall2022-08-181-4/+5
| | | | | | | | | | | | | | | | This changes code emission such that it can emit specific function fragments instead of scanning all basic blocks of a function and just emitting those that are hot or cold. To implement this, `FunctionLayout` explicitly distinguishes the "main" fragment (i.e. the one that contains the entry block and is associated with the original symbol) from "split" fragments. Additionally, `BinaryFunction` receives support for multiple cold symbols - one for each split fragment. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D130052
* [BOLT][AArch64] Ignore functions with islandsInfo during VeneerEliminarion ↵Denis Revunov2022-08-181-1/+1
| | | | | | | | and ICF Differential Revision: https://reviews.llvm.org/D131881 Reviewed By: yota9
* [BOLT][NFC] Simplify scanExternalRefsAmir Ayupov2022-08-171-8/+4
| | | | | | Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D132013
* [BOLT] Add main fragment to function layoutFabian Parzefall2022-08-171-4/+4
| | | | | | | | | | | | | Functions that do not contain any code still have to be emitted. This occurs on AArch64 where functions can consist only of a constant island. To support fragment semantics in code emission, this commits adds a guaranteed main fragment to function layout. This fragment might be empty, but allows us omit checks whether the function is empty in most places. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D130051
* [BOLT] Remove always true if statementRafael Auler2022-08-031-7/+4
| | | | | | | | Got a warning from GCC when building this. Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D131092
* [bolt] Remove redundaunt control-flow statements (NFC)Kazu Hirata2022-07-301-1/+0
| | | | Identified with readability-redundant-control-flow.
* [BOLT] Add function layout classFabian Parzefall2022-07-161-170/+161
| | | | | | | | | | This patch adds a dedicated class to keep track of each function's layout. It also lays the groundwork for splitting functions into multiple fragments (as opposed to a strict hot/cold split). Reviewed By: maksfb Differential Revision: https://reviews.llvm.org/D129518
* [BOLT] Replace uses of layout with basic block listFabian Parzefall2022-07-141-51/+59
| | | | | | | | | | | | | | | | As we are moving towards support for multiple fragments, loops that iterate over all basic blocks of a function, but do not depend on the order of basic blocks in the final layout, should iterate over binary functions directly, rather than the layout. Eventually, all loops using the layout list should either iterate over the function, or be aware of multiple layouts. This patch replaces references to binary function's block layout with the binary function itself where only little code changes are necessary. Reviewed By: maksfb Differential Revision: https://reviews.llvm.org/D129585
* [BOLT] Support multiple parents for split jump tableHuan Nguyen2022-07-131-6/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are two assumptions regarding jump table: (a) It is accessed by only one fragment, say, Parent (b) All entries target instructions in Parent For (a), BOLT stores jump table entries as relative offset to Parent. For (b), BOLT treats jump table entries target somewhere out of Parent as INVALID_OFFSET, including fragment of same split function. In this update, we extend (a) and (b) to include fragment of same split functinon. For (a), we store jump table entries in absolute offset instead. In addition, jump table will store all fragments that access it. A fragment uses this information to only create label for jump table entries that target to that fragment. For (b), using absolute offset allows jump table entries to target fragments of same split function, i.e., extend support for split jump table. This can be done using relocation (fragment start/size) and fragment detection heuristics (e.g., using symbol name pattern for non-stripped binaries). For jump table targets that can only be reached by one fragment, we mark them as local label; otherwise, they would be the secondary function entry to the target fragment. Test Plan ``` ninja check-bolt ``` Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D128474
* [BOLT][AArch64] Handle gold linker veneersVladislav Khmelevsky2022-07-131-32/+22
| | | | | | | | | | The gold linker veneers are written between functions without symbols, so we to handle it specially in BOLT. Vladislav Khmelevsky, Advanced Software Technology Lab, Huawei Differential Revision: https://reviews.llvm.org/D129260
* [BOLT] Increase coverage of shrink wrapping [1/5]Rafael Auler2022-07-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Change how function score is calculated and provide more detailed statistics when reporting back frame optimizer and shrink wrapping results. In this new statistics, we provide dynamic coverage numbers. The main metric for shrink wrapping is the number of executed stores that were saved because of shrink wrapping (push instructions that were either entirely moved away from the hot block or converted to a stack adjustment instruction). There is still a number of reduced load instructions (pop) that we are not counting at the moment. Also update alloc combiner to report dynamic numbers, as well as frame optimizer. For debugging purposes, we also include a list of top 10 functions optimized by shrink wrapping. These changes are aimed at better understanding the impact of shrink wrapping in a given binary. We also remove an assertion in dataflow analysis to do not choke on empty functions (which makes no sense). Reviewed By: Amir Differential Revision: https://reviews.llvm.org/D126111
* [BOLT] Respect shouldPrint in dump-dot-allAmir Ayupov2022-06-291-1/+8
| | | | | | | | Don't dump dot CFG graph for functions that should not be printed. Reviewed By: rafauler, maksfb Differential Revision: https://reviews.llvm.org/D128699
View Lorry Controller Status