summaryrefslogtreecommitdiffstats
path: root/contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp
diff options
context:
space:
mode:
authordim <dim@FreeBSD.org>2017-04-02 17:24:58 +0000
committerdim <dim@FreeBSD.org>2017-04-02 17:24:58 +0000
commit60b571e49a90d38697b3aca23020d9da42fc7d7f (patch)
tree99351324c24d6cb146b6285b6caffa4d26fce188 /contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp
parentbea1b22c7a9bce1dfdd73e6e5b65bc4752215180 (diff)
downloadFreeBSD-src-60b571e49a90d38697b3aca23020d9da42fc7d7f.zip
FreeBSD-src-60b571e49a90d38697b3aca23020d9da42fc7d7f.tar.gz
Update clang, llvm, lld, lldb, compiler-rt and libc++ to 4.0.0 release:
MFC r309142 (by emaste): Add WITH_LLD_AS_LD build knob If set it installs LLD as /usr/bin/ld. LLD (as of version 3.9) is not capable of linking the world and kernel, but can self-host and link many substantial applications. GNU ld continues to be used for the world and kernel build, regardless of how this knob is set. It is on by default for arm64, and off for all other CPU architectures. Sponsored by: The FreeBSD Foundation MFC r310840: Reapply 310775, now it also builds correctly if lldb is disabled: Move llvm-objdump from CLANG_EXTRAS to installed by default We currently install three tools from binutils 2.17.50: as, ld, and objdump. Work is underway to migrate to a permissively-licensed tool-chain, with one goal being the retirement of binutils 2.17.50. LLVM's llvm-objdump is intended to be compatible with GNU objdump although it is currently missing some options and may have formatting differences. Enable it by default for testing and further investigation. It may later be changed to install as /usr/bin/objdump, it becomes a fully viable replacement. Reviewed by: emaste Differential Revision: https://reviews.freebsd.org/D8879 MFC r312855 (by emaste): Rename LLD_AS_LD to LLD_IS_LD, for consistency with CLANG_IS_CC Reported by: Dan McGregor <dan.mcgregor usask.ca> MFC r313559 | glebius | 2017-02-10 18:34:48 +0100 (Fri, 10 Feb 2017) | 5 lines Don't check struct rtentry on FreeBSD, it is an internal kernel structure. On other systems it may be API structure for SIOCADDRT/SIOCDELRT. Reviewed by: emaste, dim MFC r314152 (by jkim): Remove an assembler flag, which is redundant since r309124. The upstream took care of it by introducing a macro NO_EXEC_STACK_DIRECTIVE. http://llvm.org/viewvc/llvm-project?rev=273500&view=rev Reviewed by: dim MFC r314564: Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to 4.0.0 (branches/release_40 296509). The release will follow soon. Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11 support to build; see UPDATING for more information. Also note that as of 4.0.0, lld should be able to link the base system on amd64 and aarch64. See the WITH_LLD_IS_LLD setting in src.conf(5). Though please be aware that this is work in progress. Release notes for llvm, clang and lld will be available here: <http://releases.llvm.org/4.0.0/docs/ReleaseNotes.html> <http://releases.llvm.org/4.0.0/tools/clang/docs/ReleaseNotes.html> <http://releases.llvm.org/4.0.0/tools/lld/docs/ReleaseNotes.html> Thanks to Ed Maste, Jan Beich, Antoine Brodin and Eric Fiselier for their help. Relnotes: yes Exp-run: antoine PR: 215969, 216008 MFC r314708: For now, revert r287232 from upstream llvm trunk (by Daniil Fukalov): [SCEV] limit recursion depth of CompareSCEVComplexity Summary: CompareSCEVComplexity goes too deep (50+ on a quite a big unrolled loop) and runs almost infinite time. Added cache of "equal" SCEV pairs to earlier cutoff of further estimation. Recursion depth limit was also introduced as a parameter. Reviewers: sanjoy Subscribers: mzolotukhin, tstellarAMD, llvm-commits Differential Revision: https://reviews.llvm.org/D26389 This commit is the cause of excessive compile times on skein_block.c (and possibly other files) during kernel builds on amd64. We never saw the problematic behavior described in this upstream commit, so for now it is better to revert it. An upstream bug has been filed here: https://bugs.llvm.org/show_bug.cgi?id=32142 Reported by: mjg MFC r314795: Reapply r287232 from upstream llvm trunk (by Daniil Fukalov): [SCEV] limit recursion depth of CompareSCEVComplexity Summary: CompareSCEVComplexity goes too deep (50+ on a quite a big unrolled loop) and runs almost infinite time. Added cache of "equal" SCEV pairs to earlier cutoff of further estimation. Recursion depth limit was also introduced as a parameter. Reviewers: sanjoy Subscribers: mzolotukhin, tstellarAMD, llvm-commits Differential Revision: https://reviews.llvm.org/D26389 Pull in r296992 from upstream llvm trunk (by Sanjoy Das): [SCEV] Decrease the recursion threshold for CompareValueComplexity Fixes PR32142. r287232 accidentally increased the recursion threshold for CompareValueComplexity from 2 to 32. This change reverses that change by introducing a separate flag for CompareValueComplexity's threshold. The latter revision fixes the excessive compile times for skein_block.c. MFC r314907 | mmel | 2017-03-08 12:40:27 +0100 (Wed, 08 Mar 2017) | 7 lines Unbreak ARMv6 world. The new compiler_rt library imported with clang 4.0.0 have several fatal issues (non-functional __udivsi3 for example) with ARM specific instrict functions. As temporary workaround, until upstream solve these problems, disable all thumb[1][2] related feature. MFC r315016: Update clang, llvm, lld, lldb, compiler-rt and libc++ to 4.0.0 release. We were already very close to the last release candidate, so this is a pretty minor update. Relnotes: yes MFC r316005: Revert r314907, and pull in r298713 from upstream compiler-rt trunk (by Weiming Zhao): builtins: Select correct code fragments when compiling for Thumb1/Thum2/ARM ISA. Summary: Value of __ARM_ARCH_ISA_THUMB isn't based on the actual compilation mode (-mthumb, -marm), it reflect's capability of given CPU. Due to this: - use __tbumb__ and __thumb2__ insteand of __ARM_ARCH_ISA_THUMB - use '.thumb' directive consistently in all affected files - decorate all thumb functions using DEFINE_COMPILERRT_THUMB_FUNCTION() --------- Note: This patch doesn't fix broken Thumb1 variant of __udivsi3 ! Reviewers: weimingz, rengolin, compnerd Subscribers: aemerson, dim Differential Revision: https://reviews.llvm.org/D30938 Discussed with: mmel
Diffstat (limited to 'contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp')
-rw-r--r--contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp559
1 files changed, 307 insertions, 252 deletions
diff --git a/contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp b/contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp
index c9d075e..6b32f6c 100644
--- a/contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp
+++ b/contrib/llvm/lib/Transforms/IPO/FunctionImport.cpp
@@ -21,6 +21,7 @@
#include "llvm/IR/DiagnosticPrinter.h"
#include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/Module.h"
+#include "llvm/IR/Verifier.h"
#include "llvm/IRReader/IRReader.h"
#include "llvm/Linker/Linker.h"
#include "llvm/Object/IRObjectFile.h"
@@ -35,7 +36,10 @@
using namespace llvm;
-STATISTIC(NumImported, "Number of functions imported");
+STATISTIC(NumImportedFunctions, "Number of functions imported");
+STATISTIC(NumImportedModules, "Number of modules imported from");
+STATISTIC(NumDeadSymbols, "Number of dead stripped symbols in index");
+STATISTIC(NumLiveSymbols, "Number of live symbols in index");
/// Limit on instruction count of imported functions.
static cl::opt<unsigned> ImportInstrLimit(
@@ -49,9 +53,28 @@ static cl::opt<float>
"`import-instr-limit` threshold by this factor "
"before processing newly imported functions"));
+static cl::opt<float> ImportHotInstrFactor(
+ "import-hot-evolution-factor", cl::init(1.0), cl::Hidden,
+ cl::value_desc("x"),
+ cl::desc("As we import functions called from hot callsite, multiply the "
+ "`import-instr-limit` threshold by this factor "
+ "before processing newly imported functions"));
+
+static cl::opt<float> ImportHotMultiplier(
+ "import-hot-multiplier", cl::init(3.0), cl::Hidden, cl::value_desc("x"),
+ cl::desc("Multiply the `import-instr-limit` threshold for hot callsites"));
+
+// FIXME: This multiplier was not really tuned up.
+static cl::opt<float> ImportColdMultiplier(
+ "import-cold-multiplier", cl::init(0), cl::Hidden, cl::value_desc("N"),
+ cl::desc("Multiply the `import-instr-limit` threshold for cold callsites"));
+
static cl::opt<bool> PrintImports("print-imports", cl::init(false), cl::Hidden,
cl::desc("Print imported functions"));
+static cl::opt<bool> ComputeDead("compute-dead", cl::init(true), cl::Hidden,
+ cl::desc("Compute dead symbols"));
+
// Temporary allows the function import pass to disable always linking
// referenced discardable symbols.
static cl::opt<bool>
@@ -88,69 +111,6 @@ static std::unique_ptr<Module> loadFile(const std::string &FileName,
namespace {
-// Return true if the Summary describes a GlobalValue that can be externally
-// referenced, i.e. it does not need renaming (linkage is not local) or renaming
-// is possible (does not have a section for instance).
-static bool canBeExternallyReferenced(const GlobalValueSummary &Summary) {
- if (!Summary.needsRenaming())
- return true;
-
- if (Summary.hasSection())
- // Can't rename a global that needs renaming if has a section.
- return false;
-
- return true;
-}
-
-// Return true if \p GUID describes a GlobalValue that can be externally
-// referenced, i.e. it does not need renaming (linkage is not local) or
-// renaming is possible (does not have a section for instance).
-static bool canBeExternallyReferenced(const ModuleSummaryIndex &Index,
- GlobalValue::GUID GUID) {
- auto Summaries = Index.findGlobalValueSummaryList(GUID);
- if (Summaries == Index.end())
- return true;
- if (Summaries->second.size() != 1)
- // If there are multiple globals with this GUID, then we know it is
- // not a local symbol, and it is necessarily externally referenced.
- return true;
-
- // We don't need to check for the module path, because if it can't be
- // externally referenced and we call it, it is necessarilly in the same
- // module
- return canBeExternallyReferenced(**Summaries->second.begin());
-}
-
-// Return true if the global described by \p Summary can be imported in another
-// module.
-static bool eligibleForImport(const ModuleSummaryIndex &Index,
- const GlobalValueSummary &Summary) {
- if (!canBeExternallyReferenced(Summary))
- // Can't import a global that needs renaming if has a section for instance.
- // FIXME: we may be able to import it by copying it without promotion.
- return false;
-
- // Check references (and potential calls) in the same module. If the current
- // value references a global that can't be externally referenced it is not
- // eligible for import.
- bool AllRefsCanBeExternallyReferenced =
- llvm::all_of(Summary.refs(), [&](const ValueInfo &VI) {
- return canBeExternallyReferenced(Index, VI.getGUID());
- });
- if (!AllRefsCanBeExternallyReferenced)
- return false;
-
- if (auto *FuncSummary = dyn_cast<FunctionSummary>(&Summary)) {
- bool AllCallsCanBeExternallyReferenced = llvm::all_of(
- FuncSummary->calls(), [&](const FunctionSummary::EdgeTy &Edge) {
- return canBeExternallyReferenced(Index, Edge.first.getGUID());
- });
- if (!AllCallsCanBeExternallyReferenced)
- return false;
- }
- return true;
-}
-
/// Given a list of possible callee implementation for a call site, select one
/// that fits the \p Threshold.
///
@@ -188,7 +148,7 @@ selectCallee(const ModuleSummaryIndex &Index,
if (Summary->instCount() > Threshold)
return false;
- if (!eligibleForImport(Index, *Summary))
+ if (Summary->notEligibleToImport())
return false;
return true;
@@ -210,63 +170,17 @@ static const GlobalValueSummary *selectCallee(GlobalValue::GUID GUID,
return selectCallee(Index, CalleeSummaryList->second, Threshold);
}
-/// Mark the global \p GUID as export by module \p ExportModulePath if found in
-/// this module. If it is a GlobalVariable, we also mark any referenced global
-/// in the current module as exported.
-static void exportGlobalInModule(const ModuleSummaryIndex &Index,
- StringRef ExportModulePath,
- GlobalValue::GUID GUID,
- FunctionImporter::ExportSetTy &ExportList) {
- auto FindGlobalSummaryInModule =
- [&](GlobalValue::GUID GUID) -> GlobalValueSummary *{
- auto SummaryList = Index.findGlobalValueSummaryList(GUID);
- if (SummaryList == Index.end())
- // This global does not have a summary, it is not part of the ThinLTO
- // process
- return nullptr;
- auto SummaryIter = llvm::find_if(
- SummaryList->second,
- [&](const std::unique_ptr<GlobalValueSummary> &Summary) {
- return Summary->modulePath() == ExportModulePath;
- });
- if (SummaryIter == SummaryList->second.end())
- return nullptr;
- return SummaryIter->get();
- };
-
- auto *Summary = FindGlobalSummaryInModule(GUID);
- if (!Summary)
- return;
- // We found it in the current module, mark as exported
- ExportList.insert(GUID);
-
- auto GVS = dyn_cast<GlobalVarSummary>(Summary);
- if (!GVS)
- return;
- // FunctionImportGlobalProcessing::doPromoteLocalToGlobal() will always
- // trigger importing the initializer for `constant unnamed addr` globals that
- // are referenced. We conservatively export all the referenced symbols for
- // every global to workaround this, so that the ExportList is accurate.
- // FIXME: with a "isConstant" flag in the summary we could be more targetted.
- for (auto &Ref : GVS->refs()) {
- auto GUID = Ref.getGUID();
- auto *RefSummary = FindGlobalSummaryInModule(GUID);
- if (RefSummary)
- // Found a ref in the current module, mark it as exported
- ExportList.insert(GUID);
- }
-}
-
-using EdgeInfo = std::pair<const FunctionSummary *, unsigned /* Threshold */>;
+using EdgeInfo = std::tuple<const FunctionSummary *, unsigned /* Threshold */,
+ GlobalValue::GUID>;
/// Compute the list of functions to import for a given caller. Mark these
/// imported functions and the symbols they reference in their source module as
/// exported from their source module.
static void computeImportForFunction(
const FunctionSummary &Summary, const ModuleSummaryIndex &Index,
- unsigned Threshold, const GVSummaryMapTy &DefinedGVSummaries,
+ const unsigned Threshold, const GVSummaryMapTy &DefinedGVSummaries,
SmallVectorImpl<EdgeInfo> &Worklist,
- FunctionImporter::ImportMapTy &ImportsForModule,
+ FunctionImporter::ImportMapTy &ImportList,
StringMap<FunctionImporter::ExportSetTy> *ExportLists = nullptr) {
for (auto &Edge : Summary.calls()) {
auto GUID = Edge.first.getGUID();
@@ -277,7 +191,18 @@ static void computeImportForFunction(
continue;
}
- auto *CalleeSummary = selectCallee(GUID, Threshold, Index);
+ auto GetBonusMultiplier = [](CalleeInfo::HotnessType Hotness) -> float {
+ if (Hotness == CalleeInfo::HotnessType::Hot)
+ return ImportHotMultiplier;
+ if (Hotness == CalleeInfo::HotnessType::Cold)
+ return ImportColdMultiplier;
+ return 1.0;
+ };
+
+ const auto NewThreshold =
+ Threshold * GetBonusMultiplier(Edge.second.Hotness);
+
+ auto *CalleeSummary = selectCallee(GUID, NewThreshold, Index);
if (!CalleeSummary) {
DEBUG(dbgs() << "ignored! No qualifying callee with summary found.\n");
continue;
@@ -293,40 +218,59 @@ static void computeImportForFunction(
} else
ResolvedCalleeSummary = cast<FunctionSummary>(CalleeSummary);
- assert(ResolvedCalleeSummary->instCount() <= Threshold &&
+ assert(ResolvedCalleeSummary->instCount() <= NewThreshold &&
"selectCallee() didn't honor the threshold");
+ auto GetAdjustedThreshold = [](unsigned Threshold, bool IsHotCallsite) {
+ // Adjust the threshold for next level of imported functions.
+ // The threshold is different for hot callsites because we can then
+ // inline chains of hot calls.
+ if (IsHotCallsite)
+ return Threshold * ImportHotInstrFactor;
+ return Threshold * ImportInstrFactor;
+ };
+
+ bool IsHotCallsite = Edge.second.Hotness == CalleeInfo::HotnessType::Hot;
+ const auto AdjThreshold = GetAdjustedThreshold(Threshold, IsHotCallsite);
+
auto ExportModulePath = ResolvedCalleeSummary->modulePath();
- auto &ProcessedThreshold = ImportsForModule[ExportModulePath][GUID];
+ auto &ProcessedThreshold = ImportList[ExportModulePath][GUID];
/// Since the traversal of the call graph is DFS, we can revisit a function
/// a second time with a higher threshold. In this case, it is added back to
/// the worklist with the new threshold.
- if (ProcessedThreshold && ProcessedThreshold >= Threshold) {
+ if (ProcessedThreshold && ProcessedThreshold >= AdjThreshold) {
DEBUG(dbgs() << "ignored! Target was already seen with Threshold "
<< ProcessedThreshold << "\n");
continue;
}
+ bool PreviouslyImported = ProcessedThreshold != 0;
// Mark this function as imported in this module, with the current Threshold
- ProcessedThreshold = Threshold;
+ ProcessedThreshold = AdjThreshold;
// Make exports in the source module.
if (ExportLists) {
auto &ExportList = (*ExportLists)[ExportModulePath];
ExportList.insert(GUID);
- // Mark all functions and globals referenced by this function as exported
- // to the outside if they are defined in the same source module.
- for (auto &Edge : ResolvedCalleeSummary->calls()) {
- auto CalleeGUID = Edge.first.getGUID();
- exportGlobalInModule(Index, ExportModulePath, CalleeGUID, ExportList);
- }
- for (auto &Ref : ResolvedCalleeSummary->refs()) {
- auto GUID = Ref.getGUID();
- exportGlobalInModule(Index, ExportModulePath, GUID, ExportList);
+ if (!PreviouslyImported) {
+ // This is the first time this function was exported from its source
+ // module, so mark all functions and globals it references as exported
+ // to the outside if they are defined in the same source module.
+ // For efficiency, we unconditionally add all the referenced GUIDs
+ // to the ExportList for this module, and will prune out any not
+ // defined in the module later in a single pass.
+ for (auto &Edge : ResolvedCalleeSummary->calls()) {
+ auto CalleeGUID = Edge.first.getGUID();
+ ExportList.insert(CalleeGUID);
+ }
+ for (auto &Ref : ResolvedCalleeSummary->refs()) {
+ auto GUID = Ref.getGUID();
+ ExportList.insert(GUID);
+ }
}
}
// Insert the newly imported function to the worklist.
- Worklist.push_back(std::make_pair(ResolvedCalleeSummary, Threshold));
+ Worklist.emplace_back(ResolvedCalleeSummary, AdjThreshold, GUID);
}
}
@@ -335,8 +279,9 @@ static void computeImportForFunction(
/// another module (that may require promotion).
static void ComputeImportForModule(
const GVSummaryMapTy &DefinedGVSummaries, const ModuleSummaryIndex &Index,
- FunctionImporter::ImportMapTy &ImportsForModule,
- StringMap<FunctionImporter::ExportSetTy> *ExportLists = nullptr) {
+ FunctionImporter::ImportMapTy &ImportList,
+ StringMap<FunctionImporter::ExportSetTy> *ExportLists = nullptr,
+ const DenseSet<GlobalValue::GUID> *DeadSymbols = nullptr) {
// Worklist contains the list of function imported in this module, for which
// we will analyse the callees and may import further down the callgraph.
SmallVector<EdgeInfo, 128> Worklist;
@@ -344,6 +289,10 @@ static void ComputeImportForModule(
// Populate the worklist with the import for the functions in the current
// module
for (auto &GVSummary : DefinedGVSummaries) {
+ if (DeadSymbols && DeadSymbols->count(GVSummary.first)) {
+ DEBUG(dbgs() << "Ignores Dead GUID: " << GVSummary.first << "\n");
+ continue;
+ }
auto *Summary = GVSummary.second;
if (auto *AS = dyn_cast<AliasSummary>(Summary))
Summary = &AS->getAliasee();
@@ -353,21 +302,26 @@ static void ComputeImportForModule(
continue;
DEBUG(dbgs() << "Initalize import for " << GVSummary.first << "\n");
computeImportForFunction(*FuncSummary, Index, ImportInstrLimit,
- DefinedGVSummaries, Worklist, ImportsForModule,
+ DefinedGVSummaries, Worklist, ImportList,
ExportLists);
}
+ // Process the newly imported functions and add callees to the worklist.
while (!Worklist.empty()) {
auto FuncInfo = Worklist.pop_back_val();
- auto *Summary = FuncInfo.first;
- auto Threshold = FuncInfo.second;
-
- // Process the newly imported functions and add callees to the worklist.
- // Adjust the threshold
- Threshold = Threshold * ImportInstrFactor;
+ auto *Summary = std::get<0>(FuncInfo);
+ auto Threshold = std::get<1>(FuncInfo);
+ auto GUID = std::get<2>(FuncInfo);
+
+ // Check if we later added this summary with a higher threshold.
+ // If so, skip this entry.
+ auto ExportModulePath = Summary->modulePath();
+ auto &LatestProcessedThreshold = ImportList[ExportModulePath][GUID];
+ if (LatestProcessedThreshold > Threshold)
+ continue;
computeImportForFunction(*Summary, Index, Threshold, DefinedGVSummaries,
- Worklist, ImportsForModule, ExportLists);
+ Worklist, ImportList, ExportLists);
}
}
@@ -378,14 +332,31 @@ void llvm::ComputeCrossModuleImport(
const ModuleSummaryIndex &Index,
const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
StringMap<FunctionImporter::ImportMapTy> &ImportLists,
- StringMap<FunctionImporter::ExportSetTy> &ExportLists) {
+ StringMap<FunctionImporter::ExportSetTy> &ExportLists,
+ const DenseSet<GlobalValue::GUID> *DeadSymbols) {
// For each module that has function defined, compute the import/export lists.
for (auto &DefinedGVSummaries : ModuleToDefinedGVSummaries) {
- auto &ImportsForModule = ImportLists[DefinedGVSummaries.first()];
+ auto &ImportList = ImportLists[DefinedGVSummaries.first()];
DEBUG(dbgs() << "Computing import for Module '"
<< DefinedGVSummaries.first() << "'\n");
- ComputeImportForModule(DefinedGVSummaries.second, Index, ImportsForModule,
- &ExportLists);
+ ComputeImportForModule(DefinedGVSummaries.second, Index, ImportList,
+ &ExportLists, DeadSymbols);
+ }
+
+ // When computing imports we added all GUIDs referenced by anything
+ // imported from the module to its ExportList. Now we prune each ExportList
+ // of any not defined in that module. This is more efficient than checking
+ // while computing imports because some of the summary lists may be long
+ // due to linkonce (comdat) copies.
+ for (auto &ELI : ExportLists) {
+ const auto &DefinedGVSummaries =
+ ModuleToDefinedGVSummaries.lookup(ELI.first());
+ for (auto EI = ELI.second.begin(); EI != ELI.second.end();) {
+ if (!DefinedGVSummaries.count(*EI))
+ EI = ELI.second.erase(EI);
+ else
+ ++EI;
+ }
}
#ifndef NDEBUG
@@ -431,45 +402,120 @@ void llvm::ComputeCrossModuleImportForModule(
#endif
}
+DenseSet<GlobalValue::GUID> llvm::computeDeadSymbols(
+ const ModuleSummaryIndex &Index,
+ const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols) {
+ if (!ComputeDead)
+ return DenseSet<GlobalValue::GUID>();
+ if (GUIDPreservedSymbols.empty())
+ // Don't do anything when nothing is live, this is friendly with tests.
+ return DenseSet<GlobalValue::GUID>();
+ DenseSet<GlobalValue::GUID> LiveSymbols = GUIDPreservedSymbols;
+ SmallVector<GlobalValue::GUID, 128> Worklist;
+ Worklist.reserve(LiveSymbols.size() * 2);
+ for (auto GUID : LiveSymbols) {
+ DEBUG(dbgs() << "Live root: " << GUID << "\n");
+ Worklist.push_back(GUID);
+ }
+ // Add values flagged in the index as live roots to the worklist.
+ for (const auto &Entry : Index) {
+ bool IsLiveRoot = llvm::any_of(
+ Entry.second,
+ [&](const std::unique_ptr<llvm::GlobalValueSummary> &Summary) {
+ return Summary->liveRoot();
+ });
+ if (!IsLiveRoot)
+ continue;
+ DEBUG(dbgs() << "Live root (summary): " << Entry.first << "\n");
+ Worklist.push_back(Entry.first);
+ }
+
+ while (!Worklist.empty()) {
+ auto GUID = Worklist.pop_back_val();
+ auto It = Index.findGlobalValueSummaryList(GUID);
+ if (It == Index.end()) {
+ DEBUG(dbgs() << "Not in index: " << GUID << "\n");
+ continue;
+ }
+
+ // FIXME: we should only make the prevailing copy live here
+ for (auto &Summary : It->second) {
+ for (auto Ref : Summary->refs()) {
+ auto RefGUID = Ref.getGUID();
+ if (LiveSymbols.insert(RefGUID).second) {
+ DEBUG(dbgs() << "Marking live (ref): " << RefGUID << "\n");
+ Worklist.push_back(RefGUID);
+ }
+ }
+ if (auto *FS = dyn_cast<FunctionSummary>(Summary.get())) {
+ for (auto Call : FS->calls()) {
+ auto CallGUID = Call.first.getGUID();
+ if (LiveSymbols.insert(CallGUID).second) {
+ DEBUG(dbgs() << "Marking live (call): " << CallGUID << "\n");
+ Worklist.push_back(CallGUID);
+ }
+ }
+ }
+ if (auto *AS = dyn_cast<AliasSummary>(Summary.get())) {
+ auto AliaseeGUID = AS->getAliasee().getOriginalName();
+ if (LiveSymbols.insert(AliaseeGUID).second) {
+ DEBUG(dbgs() << "Marking live (alias): " << AliaseeGUID << "\n");
+ Worklist.push_back(AliaseeGUID);
+ }
+ }
+ }
+ }
+ DenseSet<GlobalValue::GUID> DeadSymbols;
+ DeadSymbols.reserve(
+ std::min(Index.size(), Index.size() - LiveSymbols.size()));
+ for (auto &Entry : Index) {
+ auto GUID = Entry.first;
+ if (!LiveSymbols.count(GUID)) {
+ DEBUG(dbgs() << "Marking dead: " << GUID << "\n");
+ DeadSymbols.insert(GUID);
+ }
+ }
+ DEBUG(dbgs() << LiveSymbols.size() << " symbols Live, and "
+ << DeadSymbols.size() << " symbols Dead \n");
+ NumDeadSymbols += DeadSymbols.size();
+ NumLiveSymbols += LiveSymbols.size();
+ return DeadSymbols;
+}
+
/// Compute the set of summaries needed for a ThinLTO backend compilation of
/// \p ModulePath.
void llvm::gatherImportedSummariesForModule(
StringRef ModulePath,
const StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
- const StringMap<FunctionImporter::ImportMapTy> &ImportLists,
+ const FunctionImporter::ImportMapTy &ImportList,
std::map<std::string, GVSummaryMapTy> &ModuleToSummariesForIndex) {
// Include all summaries from the importing module.
ModuleToSummariesForIndex[ModulePath] =
ModuleToDefinedGVSummaries.lookup(ModulePath);
- auto ModuleImports = ImportLists.find(ModulePath);
- if (ModuleImports != ImportLists.end()) {
- // Include summaries for imports.
- for (auto &ILI : ModuleImports->second) {
- auto &SummariesForIndex = ModuleToSummariesForIndex[ILI.first()];
- const auto &DefinedGVSummaries =
- ModuleToDefinedGVSummaries.lookup(ILI.first());
- for (auto &GI : ILI.second) {
- const auto &DS = DefinedGVSummaries.find(GI.first);
- assert(DS != DefinedGVSummaries.end() &&
- "Expected a defined summary for imported global value");
- SummariesForIndex[GI.first] = DS->second;
- }
+ // Include summaries for imports.
+ for (auto &ILI : ImportList) {
+ auto &SummariesForIndex = ModuleToSummariesForIndex[ILI.first()];
+ const auto &DefinedGVSummaries =
+ ModuleToDefinedGVSummaries.lookup(ILI.first());
+ for (auto &GI : ILI.second) {
+ const auto &DS = DefinedGVSummaries.find(GI.first);
+ assert(DS != DefinedGVSummaries.end() &&
+ "Expected a defined summary for imported global value");
+ SummariesForIndex[GI.first] = DS->second;
}
}
}
/// Emit the files \p ModulePath will import from into \p OutputFilename.
-std::error_code llvm::EmitImportsFiles(
- StringRef ModulePath, StringRef OutputFilename,
- const StringMap<FunctionImporter::ImportMapTy> &ImportLists) {
- auto ModuleImports = ImportLists.find(ModulePath);
+std::error_code
+llvm::EmitImportsFiles(StringRef ModulePath, StringRef OutputFilename,
+ const FunctionImporter::ImportMapTy &ModuleImports) {
std::error_code EC;
raw_fd_ostream ImportsOS(OutputFilename, EC, sys::fs::OpenFlags::F_None);
if (EC)
return EC;
- if (ModuleImports != ImportLists.end())
- for (auto &ILI : ModuleImports->second)
- ImportsOS << ILI.first() << "\n";
+ for (auto &ILI : ModuleImports)
+ ImportsOS << ILI.first() << "\n";
return std::error_code();
}
@@ -489,6 +535,15 @@ void llvm::thinLTOResolveWeakForLinkerModule(
DEBUG(dbgs() << "ODR fixing up linkage for `" << GV.getName() << "` from "
<< GV.getLinkage() << " to " << NewLinkage << "\n");
GV.setLinkage(NewLinkage);
+ // Remove functions converted to available_externally from comdats,
+ // as this is a declaration for the linker, and will be dropped eventually.
+ // It is illegal for comdats to contain declarations.
+ auto *GO = dyn_cast_or_null<GlobalObject>(&GV);
+ if (GO && GO->isDeclarationForLinker() && GO->hasComdat()) {
+ assert(GO->hasAvailableExternallyLinkage() &&
+ "Expected comdat on definition (possibly available external)");
+ GO->setComdat(nullptr);
+ }
};
// Process functions and global now
@@ -506,7 +561,7 @@ void llvm::thinLTOInternalizeModule(Module &TheModule,
// Parse inline ASM and collect the list of symbols that are not defined in
// the current module.
StringSet<> AsmUndefinedRefs;
- object::IRObjectFile::CollectAsmUndefinedRefs(
+ ModuleSymbolTable::CollectAsmSymbols(
Triple(TheModule.getTargetTriple()), TheModule.getModuleInlineAsm(),
[&AsmUndefinedRefs](StringRef Name, object::BasicSymbolRef::Flags Flags) {
if (Flags & object::BasicSymbolRef::SF_Undefined)
@@ -561,7 +616,7 @@ void llvm::thinLTOInternalizeModule(Module &TheModule,
// Automatically import functions in Module \p DestModule based on the summaries
// index.
//
-bool FunctionImporter::importFunctions(
+Expected<bool> FunctionImporter::importFunctions(
Module &DestModule, const FunctionImporter::ImportMapTy &ImportList,
bool ForceImportReferencedDiscardableSymbols) {
DEBUG(dbgs() << "Starting import for Module "
@@ -579,14 +634,17 @@ bool FunctionImporter::importFunctions(
// Get the module for the import
const auto &FunctionsToImportPerModule = ImportList.find(Name);
assert(FunctionsToImportPerModule != ImportList.end());
- std::unique_ptr<Module> SrcModule = ModuleLoader(Name);
+ Expected<std::unique_ptr<Module>> SrcModuleOrErr = ModuleLoader(Name);
+ if (!SrcModuleOrErr)
+ return SrcModuleOrErr.takeError();
+ std::unique_ptr<Module> SrcModule = std::move(*SrcModuleOrErr);
assert(&DestModule.getContext() == &SrcModule->getContext() &&
"Context mismatch");
// If modules were created with lazy metadata loading, materialize it
// now, before linking it (otherwise this will be a noop).
- SrcModule->materializeMetadata();
- UpgradeDebugInfo(*SrcModule);
+ if (Error Err = SrcModule->materializeMetadata())
+ return std::move(Err);
auto &ImportGUIDs = FunctionsToImportPerModule->second;
// Find the globals to import
@@ -600,7 +658,8 @@ bool FunctionImporter::importFunctions(
<< " " << F.getName() << " from "
<< SrcModule->getSourceFileName() << "\n");
if (Import) {
- F.materialize();
+ if (Error Err = F.materialize())
+ return std::move(Err);
if (EnableImportMetadata) {
// Add 'thinlto_src_module' metadata for statistics and debugging.
F.setMetadata(
@@ -622,7 +681,8 @@ bool FunctionImporter::importFunctions(
<< " " << GV.getName() << " from "
<< SrcModule->getSourceFileName() << "\n");
if (Import) {
- GV.materialize();
+ if (Error Err = GV.materialize())
+ return std::move(Err);
GlobalsToImport.insert(&GV);
}
}
@@ -648,13 +708,19 @@ bool FunctionImporter::importFunctions(
<< " " << GO->getName() << " from "
<< SrcModule->getSourceFileName() << "\n");
#endif
- GO->materialize();
+ if (Error Err = GO->materialize())
+ return std::move(Err);
GlobalsToImport.insert(GO);
- GA.materialize();
+ if (Error Err = GA.materialize())
+ return std::move(Err);
GlobalsToImport.insert(&GA);
}
}
+ // Upgrade debug info after we're done materializing all the globals and we
+ // have loaded all the required metadata!
+ UpgradeDebugInfo(*SrcModule);
+
// Link in the specified functions.
if (renameModuleForThinLTO(*SrcModule, Index, &GlobalsToImport))
return true;
@@ -674,9 +740,10 @@ bool FunctionImporter::importFunctions(
report_fatal_error("Function Import: link error");
ImportedCount += GlobalsToImport.size();
+ NumImportedModules++;
}
- NumImported += ImportedCount;
+ NumImportedFunctions += ImportedCount;
DEBUG(dbgs() << "Imported " << ImportedCount << " functions for Module "
<< DestModule.getModuleIdentifier() << "\n");
@@ -689,106 +756,94 @@ static cl::opt<std::string>
SummaryFile("summary-file",
cl::desc("The summary file to use for function importing."));
-static void diagnosticHandler(const DiagnosticInfo &DI) {
- raw_ostream &OS = errs();
- DiagnosticPrinterRawOStream DP(OS);
- DI.print(DP);
- OS << '\n';
-}
+static bool doImportingForModule(Module &M) {
+ if (SummaryFile.empty())
+ report_fatal_error("error: -function-import requires -summary-file\n");
+ Expected<std::unique_ptr<ModuleSummaryIndex>> IndexPtrOrErr =
+ getModuleSummaryIndexForFile(SummaryFile);
+ if (!IndexPtrOrErr) {
+ logAllUnhandledErrors(IndexPtrOrErr.takeError(), errs(),
+ "Error loading file '" + SummaryFile + "': ");
+ return false;
+ }
+ std::unique_ptr<ModuleSummaryIndex> Index = std::move(*IndexPtrOrErr);
+
+ // First step is collecting the import list.
+ FunctionImporter::ImportMapTy ImportList;
+ ComputeCrossModuleImportForModule(M.getModuleIdentifier(), *Index,
+ ImportList);
+
+ // Conservatively mark all internal values as promoted. This interface is
+ // only used when doing importing via the function importing pass. The pass
+ // is only enabled when testing importing via the 'opt' tool, which does
+ // not do the ThinLink that would normally determine what values to promote.
+ for (auto &I : *Index) {
+ for (auto &S : I.second) {
+ if (GlobalValue::isLocalLinkage(S->linkage()))
+ S->setLinkage(GlobalValue::ExternalLinkage);
+ }
+ }
-/// Parse the summary index out of an IR file and return the summary
-/// index object if found, or nullptr if not.
-static std::unique_ptr<ModuleSummaryIndex> getModuleSummaryIndexForFile(
- StringRef Path, std::string &Error,
- const DiagnosticHandlerFunction &DiagnosticHandler) {
- std::unique_ptr<MemoryBuffer> Buffer;
- ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr =
- MemoryBuffer::getFile(Path);
- if (std::error_code EC = BufferOrErr.getError()) {
- Error = EC.message();
- return nullptr;
+ // Next we need to promote to global scope and rename any local values that
+ // are potentially exported to other modules.
+ if (renameModuleForThinLTO(M, *Index, nullptr)) {
+ errs() << "Error renaming module\n";
+ return false;
}
- Buffer = std::move(BufferOrErr.get());
- ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>> ObjOrErr =
- object::ModuleSummaryIndexObjectFile::create(Buffer->getMemBufferRef(),
- DiagnosticHandler);
- if (std::error_code EC = ObjOrErr.getError()) {
- Error = EC.message();
- return nullptr;
+
+ // Perform the import now.
+ auto ModuleLoader = [&M](StringRef Identifier) {
+ return loadFile(Identifier, M.getContext());
+ };
+ FunctionImporter Importer(*Index, ModuleLoader);
+ Expected<bool> Result = Importer.importFunctions(
+ M, ImportList, !DontForceImportReferencedDiscardableSymbols);
+
+ // FIXME: Probably need to propagate Errors through the pass manager.
+ if (!Result) {
+ logAllUnhandledErrors(Result.takeError(), errs(),
+ "Error importing module: ");
+ return false;
}
- return (*ObjOrErr)->takeIndex();
+
+ return *Result;
}
namespace {
/// Pass that performs cross-module function import provided a summary file.
-class FunctionImportPass : public ModulePass {
- /// Optional module summary index to use for importing, otherwise
- /// the summary-file option must be specified.
- const ModuleSummaryIndex *Index;
-
+class FunctionImportLegacyPass : public ModulePass {
public:
/// Pass identification, replacement for typeid
static char ID;
/// Specify pass name for debug output
- const char *getPassName() const override { return "Function Importing"; }
+ StringRef getPassName() const override { return "Function Importing"; }
- explicit FunctionImportPass(const ModuleSummaryIndex *Index = nullptr)
- : ModulePass(ID), Index(Index) {}
+ explicit FunctionImportLegacyPass() : ModulePass(ID) {}
bool runOnModule(Module &M) override {
if (skipModule(M))
return false;
- if (SummaryFile.empty() && !Index)
- report_fatal_error("error: -function-import requires -summary-file or "
- "file from frontend\n");
- std::unique_ptr<ModuleSummaryIndex> IndexPtr;
- if (!SummaryFile.empty()) {
- if (Index)
- report_fatal_error("error: -summary-file and index from frontend\n");
- std::string Error;
- IndexPtr =
- getModuleSummaryIndexForFile(SummaryFile, Error, diagnosticHandler);
- if (!IndexPtr) {
- errs() << "Error loading file '" << SummaryFile << "': " << Error
- << "\n";
- return false;
- }
- Index = IndexPtr.get();
- }
-
- // First step is collecting the import list.
- FunctionImporter::ImportMapTy ImportList;
- ComputeCrossModuleImportForModule(M.getModuleIdentifier(), *Index,
- ImportList);
-
- // Next we need to promote to global scope and rename any local values that
- // are potentially exported to other modules.
- if (renameModuleForThinLTO(M, *Index, nullptr)) {
- errs() << "Error renaming module\n";
- return false;
- }
-
- // Perform the import now.
- auto ModuleLoader = [&M](StringRef Identifier) {
- return loadFile(Identifier, M.getContext());
- };
- FunctionImporter Importer(*Index, ModuleLoader);
- return Importer.importFunctions(
- M, ImportList, !DontForceImportReferencedDiscardableSymbols);
+ return doImportingForModule(M);
}
};
} // anonymous namespace
-char FunctionImportPass::ID = 0;
-INITIALIZE_PASS_BEGIN(FunctionImportPass, "function-import",
- "Summary Based Function Import", false, false)
-INITIALIZE_PASS_END(FunctionImportPass, "function-import",
- "Summary Based Function Import", false, false)
+PreservedAnalyses FunctionImportPass::run(Module &M,
+ ModuleAnalysisManager &AM) {
+ if (!doImportingForModule(M))
+ return PreservedAnalyses::all();
+
+ return PreservedAnalyses::none();
+}
+
+char FunctionImportLegacyPass::ID = 0;
+INITIALIZE_PASS(FunctionImportLegacyPass, "function-import",
+ "Summary Based Function Import", false, false)
namespace llvm {
-Pass *createFunctionImportPass(const ModuleSummaryIndex *Index = nullptr) {
- return new FunctionImportPass(Index);
+Pass *createFunctionImportPass() {
+ return new FunctionImportLegacyPass();
}
}
OpenPOWER on IntegriCloud