diff options
| author | dim <dim@FreeBSD.org> | 2017-09-26 19:56:36 +0000 |
|---|---|---|
| committer | Luiz Souza <luiz@netgate.com> | 2018-02-21 15:12:19 -0300 |
| commit | 1dcd2e8d24b295bc73e513acec2ed1514bb66be4 (patch) | |
| tree | 4bd13a34c251e980e1a6b13584ca1f63b0dfe670 /contrib/llvm/lib/ExecutionEngine | |
| parent | f45541ca2a56a1ba1202f94c080b04e96c1fa239 (diff) | |
| download | FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.zip FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.tar.gz | |
Merge clang, llvm, lld, lldb, compiler-rt and libc++ 5.0.0 release.
MFC r309126 (by emaste):
Correct lld llvm-tblgen dependency file name
MFC r309169:
Get rid of separate Subversion mergeinfo properties for llvm-dwarfdump
and llvm-lto. The mergeinfo confuses Subversion enormously, and these
directories will just use the mergeinfo for llvm itself.
MFC r312765:
Pull in r276136 from upstream llvm trunk (by Wei Mi):
Use ValueOffsetPair to enhance value reuse during SCEV expansion.
In D12090, the ExprValueMap was added to reuse existing value during
SCEV expansion. However, const folding and sext/zext distribution can
make the reuse still difficult.
A simplified case is: suppose we know S1 expands to V1 in
ExprValueMap, and
S1 = S2 + C_a
S3 = S2 + C_b
where C_a and C_b are different SCEVConstants. Then we'd like to
expand S3 as V1 - C_a + C_b instead of expanding S2 literally. It is
helpful when S2 is a complex SCEV expr and S2 has no entry in
ExprValueMap, which is usually caused by the fact that S3 is
generated from S1 after const folding.
In order to do that, we represent ExprValueMap as a mapping from SCEV
to ValueOffsetPair. We will save both S1->{V1, 0} and S2->{V1, C_a}
into the ExprValueMap when we create SCEV for V1. When S3 is
expanded, it will first expand S2 to V1 - C_a because of S2->{V1,
C_a} in the map, then expand S3 to V1 - C_a + C_b.
Differential Revision: https://reviews.llvm.org/D21313
This should fix assertion failures when building OpenCV >= 3.1.
PR: 215649
MFC r312831:
Revert r312765 for now, since it causes assertions when building
lang/spidermonkey24.
Reported by: antoine
PR: 215649
MFC r316511 (by jhb):
Add an implementation of __ffssi2() derived from __ffsdi2().
Newer versions of GCC include an __ffssi2() symbol in libgcc and the
compiler can emit calls to it in generated code. This is true for at
least GCC 6.2 when compiling world for mips and mips64.
Reviewed by: jmallett, dim
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D10086
MFC r318601 (by adrian):
[libcompiler-rt] add bswapdi2/bswapsi2
This is required for mips gcc 6.3 userland to build/run.
Reviewed by: emaste, dim
Approved by: emaste
Differential Revision: https://reviews.freebsd.org/D10838
MFC r318884 (by emaste):
lldb: map TRAP_CAP to a trace trap
In the absense of a more specific handler for TRAP_CAP (generated by
ENOTCAPABLE or ECAPMODE while in capability mode) treat it as a trace
trap.
Example usage (testing the bug in PR219173):
% proccontrol -m trapcap lldb usr.bin/hexdump/obj/hexdump -- -Cv -s 1 /bin/ls
...
(lldb) run
Process 12980 launching
Process 12980 launched: '.../usr.bin/hexdump/obj/hexdump' (x86_64)
Process 12980 stopped
* thread #1, stop reason = trace
frame #0: 0x0000004b80c65f1a libc.so.7`__sys_lseek + 10
...
In the future we should have LLDB control the trapcap procctl itself
(as it does with ASLR), as well as report a specific stop reason.
This change eliminates an assertion failure from LLDB for now.
MFC r319796:
Remove a few unneeded files from libllvm, libclang and liblldb.
MFC r319885 (by emaste):
lld: ELF: Fix ICF crash on absolute symbol relocations.
If two sections contained relocations to absolute symbols with the same
value we would crash when trying to access their sections. Add a check that
both symbols point to sections before accessing their sections, and treat
absolute symbols as equal if their values are equal.
Obtained from: LLD commit r292578
MFC r319918:
Revert r319796 for now, it can cause undefined references when linking
in some circumstances.
Reported by: Shawn Webb <shawn.webb@hardenedbsd.org>
MFC r319957 (by emaste):
lld: Add armelf emulation mode
Obtained from: LLD r305375
MFC r321369:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 (trunk r308421). Upstream has branched for the 5.0.0 release,
which should be in about a month. Please report bugs and regressions,
so we can get them into the release.
Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.
MFC r321420:
Add a few more object files to liblldb, which should solve errors when
linking the lldb executable in some cases. In particular, when the
-ffunction-sections -fdata-sections options are turned off, or
ineffective.
Reported by: Shawn Webb, Mark Millard
MFC r321433:
Cleanup stale Options.inc files from the previous libllvm build for
clang 4.0.0. Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.
Reported by: Mark Millard
MFC r321439 (by bdrewery):
Move llvm Options.inc hack from r321433 for NO_CLEAN to lib/clang/libllvm.
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation. So using
a beforebuild target here removes the file before the compilation
tries to include it.
MFC r321664:
Pull in r308891 from upstream llvm trunk (by Benjamin Kramer):
[CodeGenPrepare] Cut off FindAllMemoryUses if there are too many uses.
This avoids excessive compile time. The case I'm looking at is
Function.cpp from an old version of LLVM that still had the giant
memcmp string matcher in it. Before r308322 this compiled in about 2
minutes, after it, clang takes infinite* time to compile it. With
this patch we're at 5 min, which is still bad but this is a
pathological case.
The cut off at 20 uses was chosen by looking at other cut-offs in LLVM
for user scanning. It's probably too high, but does the job and is
very unlikely to regress anything.
Fixes PR33900.
* I'm impatient and aborted after 15 minutes, on the bug report it was
killed after 2h.
Pull in r308986 from upstream llvm trunk (by Simon Pilgrim):
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914)
D35067/rL308322 attempted to support up to 4 load pairs for memcmp
inlining which resulted in regressions for some optimized libc memcmp
implementations (PR33914).
Until we can match these more optimal cases, this patch reduces the
memcmp expansion to a maximum of 2 load pairs (which matches what we
do for -Os).
This patch should be considered for the 5.0.0 release branch as well
Differential Revision: https://reviews.llvm.org/D35830
These fix a hang (or extremely long compile time) when building older
LLVM ports.
Reported by: antoine
PR: 219139
MFC r321719:
Pull in r309503 from upstream clang trunk (by Richard Smith):
PR33902: Invalidate line number cache when adding more text to
existing buffer.
This led to crashes as the line number cache would report a bogus
line number for a line of code, and we'd try to find a nonexistent
column within the line when printing diagnostics.
This fixes an assertion when building the graphics/champlain port.
Reported by: antoine, kwm
PR: 219139
MFC r321723:
Upgrade our copies of clang, llvm, lld and lldb to r309439 from the
upstream release_50 branch. This is just after upstream's 5.0.0-rc1.
MFC r322320:
Upgrade our copies of clang, llvm and libc++ to r310316 from the
upstream release_50 branch.
MFC r322326 (by emaste):
lldb: Make i386-*-freebsd expression work on JIT path
* Enable i386 ABI creation for freebsd
* Added an extra argument in ABISysV_i386::PrepareTrivialCall for mmap
syscall
* Unlike linux, the last argument of mmap is actually 64-bit(off_t).
This requires us to push an additional word for the higher order bits.
* Prior to this change, ktrace dump will show mmap failures due to
invalid argument coming from the 6th mmap argument.
Submitted by: Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D34776
MFC r322360 (by emaste):
lldb: Report inferior signals as signals, not exceptions, on FreeBSD
This is the FreeBSD equivalent of LLVM r238549.
This serves 2 purposes:
* LLDB should handle inferior process signals SIGSEGV/SIGILL/SIGBUS/
SIGFPE the way it is suppose to be handled. Prior to this fix these
signals will neither create a coredump, nor exit from the debugger
or work for signal handling scenario.
* eInvalidCrashReason need not report "unknown crash reason" if we have
a valid si_signo
llvm.org/pr23699
Patch by Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D35223
Submitted by: Karnajit Wangkhem
Obtained from: LLVM r310591
MFC r322474 (by emaste):
lld: Add `-z muldefs` option.
Obtained from: LLVM r310757
MFC r322740:
Upgrade our copies of clang, llvm, lld and libc++ to r311219 from the
upstream release_50 branch.
MFC r322855:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r311606 from
the upstream release_50 branch.
As of this version, lib/msun's trig test should also work correctly
again (see bug 220989 for more information).
PR: 220989
MFC r323112:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r312293 from
the upstream release_50 branch. This corresponds to 5.0.0 rc4.
As of this version, the cad/stepcode port should now compile in a more
reasonable time on i386 (see bug 221836 for more information).
PR: 221836
MFC r323245:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 release (upstream r312559).
Release notes for llvm, clang and lld will be available here soon:
<http://releases.llvm.org/5.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/lld/docs/ReleaseNotes.html>
Relnotes: yes
(cherry picked from commit 12cd91cf4c6b96a24427c0de5374916f2808d263)
Diffstat (limited to 'contrib/llvm/lib/ExecutionEngine')
30 files changed, 724 insertions, 319 deletions
diff --git a/contrib/llvm/lib/ExecutionEngine/ExecutionEngine.cpp b/contrib/llvm/lib/ExecutionEngine/ExecutionEngine.cpp index b4bed32..2ee72f9 100644 --- a/contrib/llvm/lib/ExecutionEngine/ExecutionEngine.cpp +++ b/contrib/llvm/lib/ExecutionEngine/ExecutionEngine.cpp @@ -515,7 +515,7 @@ ExecutionEngine *EngineBuilder::create(TargetMachine *TM) { // to the function tells DynamicLibrary to load the program, not a library. if (sys::DynamicLibrary::LoadLibraryPermanently(nullptr, ErrorStr)) return nullptr; - + // If the user specified a memory manager but didn't specify which engine to // create, we assume they only want the JIT, and we fail if they only want // the interpreter. @@ -616,7 +616,7 @@ GenericValue ExecutionEngine::getConstantValue(const Constant *C) { for (unsigned int i = 0; i < elemNum; ++i) { Type *ElemTy = STy->getElementType(i); if (ElemTy->isIntegerTy()) - Result.AggregateVal[i].IntVal = + Result.AggregateVal[i].IntVal = APInt(ElemTy->getPrimitiveSizeInBits(), 0); else if (ElemTy->isAggregateType()) { const Constant *ElemUndef = UndefValue::get(ElemTy); @@ -727,7 +727,7 @@ GenericValue ExecutionEngine::getConstantValue(const Constant *C) { APFloat apf = APFloat(APFloat::x87DoubleExtended(), GV.IntVal); uint64_t v; bool ignored; - (void)apf.convertToInteger(&v, BitWidth, + (void)apf.convertToInteger(makeMutableArrayRef(v), BitWidth, CE->getOpcode()==Instruction::FPToSI, APFloat::rmTowardZero, &ignored); GV.IntVal = v; // endian? @@ -979,7 +979,7 @@ GenericValue ExecutionEngine::getConstantValue(const Constant *C) { // Check if vector holds integers. if (ElemTy->isIntegerTy()) { if (CAZ) { - GenericValue intZero; + GenericValue intZero; intZero.IntVal = APInt(ElemTy->getScalarSizeInBits(), 0ull); std::fill(Result.AggregateVal.begin(), Result.AggregateVal.end(), intZero); @@ -1079,7 +1079,7 @@ void ExecutionEngine::StoreValueToMemory(const GenericValue &Val, *(((float*)Ptr)+i) = Val.AggregateVal[i].FloatVal; if (cast<VectorType>(Ty)->getElementType()->isIntegerTy()) { unsigned numOfBytes =(Val.AggregateVal[i].IntVal.getBitWidth()+7)/8; - StoreIntToMemory(Val.AggregateVal[i].IntVal, + StoreIntToMemory(Val.AggregateVal[i].IntVal, (uint8_t*)Ptr + numOfBytes*i, numOfBytes); } } @@ -1186,7 +1186,7 @@ void ExecutionEngine::InitializeMemory(const Constant *Init, void *Addr) { DEBUG(Init->dump()); if (isa<UndefValue>(Init)) return; - + if (const ConstantVector *CP = dyn_cast<ConstantVector>(Init)) { unsigned ElementSize = getDataLayout().getTypeAllocSize(CP->getType()->getElementType()); @@ -1194,12 +1194,12 @@ void ExecutionEngine::InitializeMemory(const Constant *Init, void *Addr) { InitializeMemory(CP->getOperand(i), (char*)Addr+i*ElementSize); return; } - + if (isa<ConstantAggregateZero>(Init)) { memset(Addr, 0, (size_t)getDataLayout().getTypeAllocSize(Init->getType())); return; } - + if (const ConstantArray *CPA = dyn_cast<ConstantArray>(Init)) { unsigned ElementSize = getDataLayout().getTypeAllocSize(CPA->getType()->getElementType()); @@ -1207,7 +1207,7 @@ void ExecutionEngine::InitializeMemory(const Constant *Init, void *Addr) { InitializeMemory(CPA->getOperand(i), (char*)Addr+i*ElementSize); return; } - + if (const ConstantStruct *CPS = dyn_cast<ConstantStruct>(Init)) { const StructLayout *SL = getDataLayout().getStructLayout(cast<StructType>(CPS->getType())); diff --git a/contrib/llvm/lib/ExecutionEngine/ExecutionEngineBindings.cpp b/contrib/llvm/lib/ExecutionEngine/ExecutionEngineBindings.cpp index 1d7c6e7..e956dbe 100644 --- a/contrib/llvm/lib/ExecutionEngine/ExecutionEngineBindings.cpp +++ b/contrib/llvm/lib/ExecutionEngine/ExecutionEngineBindings.cpp @@ -188,7 +188,7 @@ LLVMBool LLVMCreateMCJITCompilerForModule( for (auto &F : *Mod) { auto Attrs = F.getAttributes(); StringRef Value(options.NoFramePointerElim ? "true" : "false"); - Attrs = Attrs.addAttribute(F.getContext(), AttributeSet::FunctionIndex, + Attrs = Attrs.addAttribute(F.getContext(), AttributeList::FunctionIndex, "no-frame-pointer-elim", Value); F.setAttributes(Attrs); } diff --git a/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/IntelJITEventListener.cpp b/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/IntelJITEventListener.cpp index 0051c69..a7b1fe2 100644 --- a/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/IntelJITEventListener.cpp +++ b/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/IntelJITEventListener.cpp @@ -12,10 +12,10 @@ // //===----------------------------------------------------------------------===// -#include "llvm/Config/config.h" #include "IntelJITEventsWrapper.h" #include "llvm/ADT/DenseMap.h" #include "llvm/CodeGen/MachineFunction.h" +#include "llvm/Config/config.h" #include "llvm/DebugInfo/DIContext.h" #include "llvm/DebugInfo/DWARF/DWARFContext.h" #include "llvm/ExecutionEngine/JITEventListener.h" diff --git a/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/jitprofiling.c b/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/jitprofiling.c index e966889..f2d36a7 100644 --- a/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/jitprofiling.c +++ b/contrib/llvm/lib/ExecutionEngine/IntelJITEvents/jitprofiling.c @@ -22,8 +22,8 @@ #include <windows.h> #pragma optimize("", off) #else /* ITT_PLATFORM==ITT_PLATFORM_WIN */ -#include <pthread.h> #include <dlfcn.h> +#include <pthread.h> #include <stdint.h> #endif /* ITT_PLATFORM==ITT_PLATFORM_WIN */ #include <malloc.h> diff --git a/contrib/llvm/lib/ExecutionEngine/Interpreter/Execution.cpp b/contrib/llvm/lib/ExecutionEngine/Interpreter/Execution.cpp index 923f6e7..9684443 100644 --- a/contrib/llvm/lib/ExecutionEngine/Interpreter/Execution.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Interpreter/Execution.cpp @@ -899,10 +899,10 @@ void Interpreter::visitSwitchInst(SwitchInst &I) { // Check to see if any of the cases match... BasicBlock *Dest = nullptr; - for (SwitchInst::CaseIt i = I.case_begin(), e = I.case_end(); i != e; ++i) { - GenericValue CaseVal = getOperandValue(i.getCaseValue(), SF); + for (auto Case : I.cases()) { + GenericValue CaseVal = getOperandValue(Case.getCaseValue(), SF); if (executeICMP_EQ(CondVal, CaseVal, ElTy).IntVal != 0) { - Dest = cast<BasicBlock>(i.getCaseSuccessor()); + Dest = cast<BasicBlock>(Case.getCaseSuccessor()); break; } } @@ -1565,7 +1565,7 @@ GenericValue Interpreter::executeBitCastInst(Value *SrcVal, Type *DstTy, Tmp = Tmp.zext(SrcBitSize); Tmp = TempSrc.AggregateVal[SrcElt++].IntVal; Tmp = Tmp.zext(DstBitSize); - Tmp = Tmp.shl(ShiftAmt); + Tmp <<= ShiftAmt; ShiftAmt += isLittleEndian ? SrcBitSize : -SrcBitSize; Elt.IntVal |= Tmp; } @@ -1580,7 +1580,7 @@ GenericValue Interpreter::executeBitCastInst(Value *SrcVal, Type *DstTy, GenericValue Elt; Elt.IntVal = Elt.IntVal.zext(SrcBitSize); Elt.IntVal = TempSrc.AggregateVal[i].IntVal; - Elt.IntVal = Elt.IntVal.lshr(ShiftAmt); + Elt.IntVal.lshrInPlace(ShiftAmt); // it could be DstBitSize == SrcBitSize, so check it if (DstBitSize < SrcBitSize) Elt.IntVal = Elt.IntVal.trunc(DstBitSize); diff --git a/contrib/llvm/lib/ExecutionEngine/Interpreter/ExternalFunctions.cpp b/contrib/llvm/lib/ExecutionEngine/Interpreter/ExternalFunctions.cpp index ee75bee..64dca93 100644 --- a/contrib/llvm/lib/ExecutionEngine/Interpreter/ExternalFunctions.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Interpreter/ExternalFunctions.cpp @@ -22,7 +22,7 @@ #include "Interpreter.h" #include "llvm/ADT/APInt.h" #include "llvm/ADT/ArrayRef.h" -#include "llvm/Config/config.h" // Detect libffi +#include "llvm/Config/config.h" // Detect libffi #include "llvm/ExecutionEngine/GenericValue.h" #include "llvm/IR/DataLayout.h" #include "llvm/IR/DerivedTypes.h" @@ -33,8 +33,8 @@ #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/Mutex.h" -#include "llvm/Support/raw_ostream.h" #include "llvm/Support/UniqueLock.h" +#include "llvm/Support/raw_ostream.h" #include <cassert> #include <cmath> #include <csignal> diff --git a/contrib/llvm/lib/ExecutionEngine/MCJIT/MCJIT.cpp b/contrib/llvm/lib/ExecutionEngine/MCJIT/MCJIT.cpp index ff8749f..1164d60 100644 --- a/contrib/llvm/lib/ExecutionEngine/MCJIT/MCJIT.cpp +++ b/contrib/llvm/lib/ExecutionEngine/MCJIT/MCJIT.cpp @@ -317,7 +317,13 @@ uint64_t MCJIT::getSymbolAddress(const std::string &Name, raw_string_ostream MangledNameStream(MangledName); Mangler::getNameWithPrefix(MangledNameStream, Name, getDataLayout()); } - return findSymbol(MangledName, CheckFunctionsOnly).getAddress(); + if (auto Sym = findSymbol(MangledName, CheckFunctionsOnly)) { + if (auto AddrOrErr = Sym.getAddress()) + return *AddrOrErr; + else + report_fatal_error(AddrOrErr.takeError()); + } else + report_fatal_error(Sym.takeError()); } JITSymbol MCJIT::findSymbol(const std::string &Name, @@ -599,11 +605,12 @@ GenericValue MCJIT::runFunction(Function *F, ArrayRef<GenericValue> ArgValues) { void *MCJIT::getPointerToNamedFunction(StringRef Name, bool AbortOnFailure) { if (!isSymbolSearchingDisabled()) { - void *ptr = - reinterpret_cast<void*>( - static_cast<uintptr_t>(Resolver.findSymbol(Name).getAddress())); - if (ptr) - return ptr; + if (auto Sym = Resolver.findSymbol(Name)) { + if (auto AddrOrErr = Sym.getAddress()) + return reinterpret_cast<void*>( + static_cast<uintptr_t>(*AddrOrErr)); + } else if (auto Err = Sym.takeError()) + report_fatal_error(std::move(Err)); } /// If a LazyFunctionCreator is installed, use it to get/create the function. diff --git a/contrib/llvm/lib/ExecutionEngine/OProfileJIT/OProfileJITEventListener.cpp b/contrib/llvm/lib/ExecutionEngine/OProfileJIT/OProfileJITEventListener.cpp index 57b5d85..3581d64 100644 --- a/contrib/llvm/lib/ExecutionEngine/OProfileJIT/OProfileJITEventListener.cpp +++ b/contrib/llvm/lib/ExecutionEngine/OProfileJIT/OProfileJITEventListener.cpp @@ -12,8 +12,8 @@ // //===----------------------------------------------------------------------===// -#include "llvm/Config/config.h" #include "llvm/CodeGen/MachineFunction.h" +#include "llvm/Config/config.h" #include "llvm/ExecutionEngine/JITEventListener.h" #include "llvm/ExecutionEngine/OProfileWrapper.h" #include "llvm/ExecutionEngine/RuntimeDyld.h" diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp b/contrib/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp index 711b887..e3a4568 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Orc/IndirectionUtils.cpp @@ -7,9 +7,9 @@ // //===----------------------------------------------------------------------===// +#include "llvm/ExecutionEngine/Orc/IndirectionUtils.h" #include "llvm/ADT/STLExtras.h" #include "llvm/ADT/Triple.h" -#include "llvm/ExecutionEngine/Orc/IndirectionUtils.h" #include "llvm/ExecutionEngine/Orc/OrcABISupport.h" #include "llvm/IR/CallSite.h" #include "llvm/IR/IRBuilder.h" diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindings.cpp b/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindings.cpp index 8dcd49a..de80cb1 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindings.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindings.cpp @@ -12,6 +12,24 @@ using namespace llvm; +LLVMSharedModuleRef LLVMOrcMakeSharedModule(LLVMModuleRef Mod) { + return wrap(new std::shared_ptr<Module>(unwrap(Mod))); +} + +void LLVMOrcDisposeSharedModuleRef(LLVMSharedModuleRef SharedMod) { + delete unwrap(SharedMod); +} + +LLVMSharedObjectBufferRef +LLVMOrcMakeSharedObjectBuffer(LLVMMemoryBufferRef ObjBuffer) { + return wrap(new std::shared_ptr<MemoryBuffer>(unwrap(ObjBuffer))); +} + +void +LLVMOrcDisposeSharedObjectBufferRef(LLVMSharedObjectBufferRef SharedObjBuffer) { + delete unwrap(SharedObjBuffer); +} + LLVMOrcJITStackRef LLVMOrcCreateInstance(LLVMTargetMachineRef TM) { TargetMachine *TM2(unwrap(TM)); @@ -42,12 +60,13 @@ void LLVMOrcGetMangledSymbol(LLVMOrcJITStackRef JITStack, char **MangledName, void LLVMOrcDisposeMangledSymbol(char *MangledName) { delete[] MangledName; } -LLVMOrcTargetAddress +LLVMOrcErrorCode LLVMOrcCreateLazyCompileCallback(LLVMOrcJITStackRef JITStack, + LLVMOrcTargetAddress *RetAddr, LLVMOrcLazyCompileCallbackFn Callback, void *CallbackCtx) { OrcCBindingsStack &J = *unwrap(JITStack); - return J.createLazyCompileCallback(Callback, CallbackCtx); + return J.createLazyCompileCallback(*RetAddr, Callback, CallbackCtx); } LLVMOrcErrorCode LLVMOrcCreateIndirectStub(LLVMOrcJITStackRef JITStack, @@ -64,36 +83,44 @@ LLVMOrcErrorCode LLVMOrcSetIndirectStubPointer(LLVMOrcJITStackRef JITStack, return J.setIndirectStubPointer(StubName, NewAddr); } -LLVMOrcModuleHandle -LLVMOrcAddEagerlyCompiledIR(LLVMOrcJITStackRef JITStack, LLVMModuleRef Mod, +LLVMOrcErrorCode +LLVMOrcAddEagerlyCompiledIR(LLVMOrcJITStackRef JITStack, + LLVMOrcModuleHandle *RetHandle, + LLVMSharedModuleRef Mod, LLVMOrcSymbolResolverFn SymbolResolver, void *SymbolResolverCtx) { OrcCBindingsStack &J = *unwrap(JITStack); - Module *M(unwrap(Mod)); - return J.addIRModuleEager(M, SymbolResolver, SymbolResolverCtx); + std::shared_ptr<Module> *M(unwrap(Mod)); + return J.addIRModuleEager(*RetHandle, *M, SymbolResolver, SymbolResolverCtx); } -LLVMOrcModuleHandle -LLVMOrcAddLazilyCompiledIR(LLVMOrcJITStackRef JITStack, LLVMModuleRef Mod, +LLVMOrcErrorCode +LLVMOrcAddLazilyCompiledIR(LLVMOrcJITStackRef JITStack, + LLVMOrcModuleHandle *RetHandle, + LLVMSharedModuleRef Mod, LLVMOrcSymbolResolverFn SymbolResolver, void *SymbolResolverCtx) { OrcCBindingsStack &J = *unwrap(JITStack); - Module *M(unwrap(Mod)); - return J.addIRModuleLazy(M, SymbolResolver, SymbolResolverCtx); + std::shared_ptr<Module> *M(unwrap(Mod)); + return J.addIRModuleLazy(*RetHandle, *M, SymbolResolver, SymbolResolverCtx); } -void LLVMOrcRemoveModule(LLVMOrcJITStackRef JITStack, LLVMOrcModuleHandle H) { +LLVMOrcErrorCode LLVMOrcRemoveModule(LLVMOrcJITStackRef JITStack, + LLVMOrcModuleHandle H) { OrcCBindingsStack &J = *unwrap(JITStack); - J.removeModule(H); + return J.removeModule(H); } -LLVMOrcTargetAddress LLVMOrcGetSymbolAddress(LLVMOrcJITStackRef JITStack, - const char *SymbolName) { +LLVMOrcErrorCode LLVMOrcGetSymbolAddress(LLVMOrcJITStackRef JITStack, + LLVMOrcTargetAddress *RetAddr, + const char *SymbolName) { OrcCBindingsStack &J = *unwrap(JITStack); - auto Sym = J.findSymbol(SymbolName, true); - return Sym.getAddress(); + return J.findSymbolAddress(*RetAddr, SymbolName, true); } -void LLVMOrcDisposeInstance(LLVMOrcJITStackRef JITStack) { - delete unwrap(JITStack); +LLVMOrcErrorCode LLVMOrcDisposeInstance(LLVMOrcJITStackRef JITStack) { + auto *J = unwrap(JITStack); + auto Err = J->shutdown(); + delete J; + return Err; } diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindingsStack.h b/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindingsStack.h index a74fae7..e38decf 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindingsStack.h +++ b/contrib/llvm/lib/ExecutionEngine/Orc/OrcCBindingsStack.h @@ -1,4 +1,4 @@ -//===--- OrcCBindingsStack.h - Orc JIT stack for C bindings ---*- C++ -*---===// +//===- OrcCBindingsStack.h - Orc JIT stack for C bindings -----*- C++ -*---===// // // The LLVM Compiler Infrastructure // @@ -11,47 +11,71 @@ #define LLVM_LIB_EXECUTIONENGINE_ORC_ORCCBINDINGSSTACK_H #include "llvm-c/OrcBindings.h" -#include "llvm/ADT/Triple.h" +#include "llvm-c/TargetMachine.h" +#include "llvm/ADT/STLExtras.h" +#include "llvm/ADT/StringRef.h" +#include "llvm/ExecutionEngine/JITSymbol.h" #include "llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h" #include "llvm/ExecutionEngine/Orc/CompileUtils.h" #include "llvm/ExecutionEngine/Orc/ExecutionUtils.h" #include "llvm/ExecutionEngine/Orc/IRCompileLayer.h" -#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h" -#include "llvm/IR/LLVMContext.h" +#include "llvm/ExecutionEngine/Orc/LambdaResolver.h" +#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h" +#include "llvm/ExecutionEngine/RuntimeDyld.h" +#include "llvm/ExecutionEngine/SectionMemoryManager.h" +#include "llvm/IR/DataLayout.h" +#include "llvm/IR/Mangler.h" +#include "llvm/IR/Module.h" +#include "llvm/Support/CBindingWrapping.h" #include "llvm/Support/Error.h" +#include "llvm/Support/raw_ostream.h" +#include "llvm/Target/TargetMachine.h" +#include <algorithm> +#include <cstdint> +#include <functional> +#include <memory> +#include <set> +#include <string> +#include <vector> namespace llvm { class OrcCBindingsStack; +DEFINE_SIMPLE_CONVERSION_FUNCTIONS(std::shared_ptr<Module>, + LLVMSharedModuleRef) +DEFINE_SIMPLE_CONVERSION_FUNCTIONS(std::shared_ptr<MemoryBuffer>, + LLVMSharedObjectBufferRef) DEFINE_SIMPLE_CONVERSION_FUNCTIONS(OrcCBindingsStack, LLVMOrcJITStackRef) DEFINE_SIMPLE_CONVERSION_FUNCTIONS(TargetMachine, LLVMTargetMachineRef) class OrcCBindingsStack { public: - typedef orc::JITCompileCallbackManager CompileCallbackMgr; - typedef orc::ObjectLinkingLayer<> ObjLayerT; - typedef orc::IRCompileLayer<ObjLayerT> CompileLayerT; - typedef orc::CompileOnDemandLayer<CompileLayerT, CompileCallbackMgr> - CODLayerT; - typedef std::function<std::unique_ptr<CompileCallbackMgr>()> - CallbackManagerBuilder; + using CompileCallbackMgr = orc::JITCompileCallbackManager; + using ObjLayerT = orc::RTDyldObjectLinkingLayer; + using CompileLayerT = orc::IRCompileLayer<ObjLayerT, orc::SimpleCompiler>; + using CODLayerT = + orc::CompileOnDemandLayer<CompileLayerT, CompileCallbackMgr>; - typedef CODLayerT::IndirectStubsManagerBuilderT IndirectStubsManagerBuilder; + using CallbackManagerBuilder = + std::function<std::unique_ptr<CompileCallbackMgr>()>; + + using IndirectStubsManagerBuilder = CODLayerT::IndirectStubsManagerBuilderT; private: class GenericHandle { public: - virtual ~GenericHandle() {} + virtual ~GenericHandle() = default; + virtual JITSymbol findSymbolIn(const std::string &Name, bool ExportedSymbolsOnly) = 0; - virtual void removeModule() = 0; + virtual Error removeModule() = 0; }; template <typename LayerT> class GenericHandleImpl : public GenericHandle { public: - GenericHandleImpl(LayerT &Layer, typename LayerT::ModuleSetHandleT Handle) + GenericHandleImpl(LayerT &Layer, typename LayerT::ModuleHandleT Handle) : Layer(Layer), Handle(std::move(Handle)) {} JITSymbol findSymbolIn(const std::string &Name, @@ -59,31 +83,32 @@ private: return Layer.findSymbolIn(Handle, Name, ExportedSymbolsOnly); } - void removeModule() override { return Layer.removeModuleSet(Handle); } + Error removeModule() override { return Layer.removeModule(Handle); } private: LayerT &Layer; - typename LayerT::ModuleSetHandleT Handle; + typename LayerT::ModuleHandleT Handle; }; template <typename LayerT> std::unique_ptr<GenericHandleImpl<LayerT>> - createGenericHandle(LayerT &Layer, typename LayerT::ModuleSetHandleT Handle) { + createGenericHandle(LayerT &Layer, typename LayerT::ModuleHandleT Handle) { return llvm::make_unique<GenericHandleImpl<LayerT>>(Layer, std::move(Handle)); } public: - // We need a 'ModuleSetHandleT' to conform to the layer concept. - typedef unsigned ModuleSetHandleT; - - typedef unsigned ModuleHandleT; + using ModuleHandleT = unsigned; OrcCBindingsStack(TargetMachine &TM, std::unique_ptr<CompileCallbackMgr> CCMgr, IndirectStubsManagerBuilder IndirectStubsMgrBuilder) : DL(TM.createDataLayout()), IndirectStubsMgr(IndirectStubsMgrBuilder()), - CCMgr(std::move(CCMgr)), ObjectLayer(), + CCMgr(std::move(CCMgr)), + ObjectLayer( + []() { + return std::make_shared<SectionMemoryManager>(); + }), CompileLayer(ObjectLayer, orc::SimpleCompiler(TM)), CODLayer(CompileLayer, [](Function &F) { return std::set<Function *>({&F}); }, @@ -91,12 +116,14 @@ public: CXXRuntimeOverrides( [this](const std::string &S) { return mangle(S); }) {} - ~OrcCBindingsStack() { + LLVMOrcErrorCode shutdown() { // Run any destructors registered with __cxa_atexit. CXXRuntimeOverrides.runDestructors(); // Run any IR destructors. for (auto &DtorRunner : IRStaticDestructorRunners) - DtorRunner.runViaLayer(*this); + if (auto Err = DtorRunner.runViaLayer(*this)) + return mapError(std::move(Err)); + return LLVMOrcErrSuccess; } std::string mangle(StringRef Name) { @@ -113,14 +140,17 @@ public: return reinterpret_cast<PtrTy>(static_cast<uintptr_t>(Addr)); } - JITTargetAddress - createLazyCompileCallback(LLVMOrcLazyCompileCallbackFn Callback, + + LLVMOrcErrorCode + createLazyCompileCallback(JITTargetAddress &RetAddr, + LLVMOrcLazyCompileCallbackFn Callback, void *CallbackCtx) { auto CCInfo = CCMgr->getCompileCallback(); CCInfo.setCompileAction([=]() -> JITTargetAddress { return Callback(wrap(this), CallbackCtx); }); - return CCInfo.getAddress(); + RetAddr = CCInfo.getAddress(); + return LLVMOrcErrSuccess; } LLVMOrcErrorCode createIndirectStub(StringRef StubName, @@ -134,12 +164,12 @@ public: return mapError(IndirectStubsMgr->updatePointer(Name, Addr)); } - std::unique_ptr<JITSymbolResolver> + std::shared_ptr<JITSymbolResolver> createResolver(LLVMOrcSymbolResolverFn ExternalResolver, void *ExternalResolverCtx) { return orc::createLambdaResolver( [this, ExternalResolver, ExternalResolverCtx](const std::string &Name) - -> JITSymbol { + -> JITSymbol { // Search order: // 1. JIT'd symbols. // 2. Runtime overrides. @@ -147,26 +177,31 @@ public: if (auto Sym = CODLayer.findSymbol(Name, true)) return Sym; + else if (auto Err = Sym.takeError()) + return Sym.takeError(); + if (auto Sym = CXXRuntimeOverrides.searchOverrides(Name)) return Sym; if (ExternalResolver) return JITSymbol( ExternalResolver(Name.c_str(), ExternalResolverCtx), - llvm::JITSymbolFlags::Exported); + JITSymbolFlags::Exported); return JITSymbol(nullptr); }, - [](const std::string &Name) { + [](const std::string &Name) -> JITSymbol { return JITSymbol(nullptr); }); } template <typename LayerT> - ModuleHandleT addIRModule(LayerT &Layer, Module *M, - std::unique_ptr<RuntimeDyld::MemoryManager> MemMgr, - LLVMOrcSymbolResolverFn ExternalResolver, - void *ExternalResolverCtx) { + LLVMOrcErrorCode + addIRModule(ModuleHandleT &RetHandle, LayerT &Layer, + std::shared_ptr<Module> M, + std::unique_ptr<RuntimeDyld::MemoryManager> MemMgr, + LLVMOrcSymbolResolverFn ExternalResolver, + void *ExternalResolverCtx) { // Attach a data-layout if one isn't already present. if (M->getDataLayout().isDefault()) @@ -184,46 +219,52 @@ public: auto Resolver = createResolver(ExternalResolver, ExternalResolverCtx); // Add the module to the JIT. - std::vector<Module *> S; - S.push_back(std::move(M)); - - auto LH = Layer.addModuleSet(std::move(S), std::move(MemMgr), - std::move(Resolver)); - ModuleHandleT H = createHandle(Layer, LH); + ModuleHandleT H; + if (auto LHOrErr = Layer.addModule(std::move(M), std::move(Resolver))) + H = createHandle(Layer, *LHOrErr); + else + return mapError(LHOrErr.takeError()); // Run the static constructors, and save the static destructor runner for // execution when the JIT is torn down. orc::CtorDtorRunner<OrcCBindingsStack> CtorRunner(std::move(CtorNames), H); - CtorRunner.runViaLayer(*this); + if (auto Err = CtorRunner.runViaLayer(*this)) + return mapError(std::move(Err)); IRStaticDestructorRunners.emplace_back(std::move(DtorNames), H); - return H; + RetHandle = H; + return LLVMOrcErrSuccess; } - ModuleHandleT addIRModuleEager(Module *M, - LLVMOrcSymbolResolverFn ExternalResolver, - void *ExternalResolverCtx) { - return addIRModule(CompileLayer, std::move(M), + LLVMOrcErrorCode addIRModuleEager(ModuleHandleT &RetHandle, + std::shared_ptr<Module> M, + LLVMOrcSymbolResolverFn ExternalResolver, + void *ExternalResolverCtx) { + return addIRModule(RetHandle, CompileLayer, std::move(M), llvm::make_unique<SectionMemoryManager>(), std::move(ExternalResolver), ExternalResolverCtx); } - ModuleHandleT addIRModuleLazy(Module *M, - LLVMOrcSymbolResolverFn ExternalResolver, - void *ExternalResolverCtx) { - return addIRModule(CODLayer, std::move(M), + LLVMOrcErrorCode addIRModuleLazy(ModuleHandleT &RetHandle, + std::shared_ptr<Module> M, + LLVMOrcSymbolResolverFn ExternalResolver, + void *ExternalResolverCtx) { + return addIRModule(RetHandle, CODLayer, std::move(M), llvm::make_unique<SectionMemoryManager>(), std::move(ExternalResolver), ExternalResolverCtx); } - void removeModule(ModuleHandleT H) { - GenericHandles[H]->removeModule(); + LLVMOrcErrorCode removeModule(ModuleHandleT H) { + if (auto Err = GenericHandles[H]->removeModule()) + return mapError(std::move(Err)); GenericHandles[H] = nullptr; FreeHandleIndexes.push_back(H); + return LLVMOrcErrSuccess; } - JITSymbol findSymbol(const std::string &Name, bool ExportedSymbolsOnly) { + JITSymbol findSymbol(const std::string &Name, + bool ExportedSymbolsOnly) { if (auto Sym = IndirectStubsMgr->findStub(Name, ExportedSymbolsOnly)) return Sym; return CODLayer.findSymbol(mangle(Name), ExportedSymbolsOnly); @@ -234,12 +275,31 @@ public: return GenericHandles[H]->findSymbolIn(Name, ExportedSymbolsOnly); } + LLVMOrcErrorCode findSymbolAddress(JITTargetAddress &RetAddr, + const std::string &Name, + bool ExportedSymbolsOnly) { + RetAddr = 0; + if (auto Sym = findSymbol(Name, ExportedSymbolsOnly)) { + // Successful lookup, non-null symbol: + if (auto AddrOrErr = Sym.getAddress()) { + RetAddr = *AddrOrErr; + return LLVMOrcErrSuccess; + } else + return mapError(AddrOrErr.takeError()); + } else if (auto Err = Sym.takeError()) { + // Lookup failure - report error. + return mapError(std::move(Err)); + } + // Otherwise we had a successful lookup but got a null result. We already + // set RetAddr to '0' above, so just return success. + return LLVMOrcErrSuccess; + } + const std::string &getErrorMessage() const { return ErrMsg; } private: template <typename LayerT> - unsigned createHandle(LayerT &Layer, - typename LayerT::ModuleSetHandleT Handle) { + unsigned createHandle(LayerT &Layer, typename LayerT::ModuleHandleT Handle) { unsigned NewHandle; if (!FreeHandleIndexes.empty()) { NewHandle = FreeHandleIndexes.back(); diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/OrcError.cpp b/contrib/llvm/lib/ExecutionEngine/Orc/OrcError.cpp index c531fe3..df2d320 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/OrcError.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Orc/OrcError.cpp @@ -39,14 +39,21 @@ public: return "Remote indirect stubs owner does not exist"; case OrcErrorCode::RemoteIndirectStubsOwnerIdAlreadyInUse: return "Remote indirect stubs owner Id already in use"; + case OrcErrorCode::RPCConnectionClosed: + return "RPC connection closed"; + case OrcErrorCode::RPCCouldNotNegotiateFunction: + return "Could not negotiate RPC function"; case OrcErrorCode::RPCResponseAbandoned: return "RPC response abandoned"; + case OrcErrorCode::JITSymbolNotFound: + return "JIT symbol not found"; case OrcErrorCode::UnexpectedRPCCall: return "Unexpected RPC call"; case OrcErrorCode::UnexpectedRPCResponse: return "Unexpected RPC response"; - case OrcErrorCode::UnknownRPCFunction: - return "Unknown RPC function"; + case OrcErrorCode::UnknownErrorCodeFromRemote: + return "Unknown error returned from remote RPC function " + "(Use StringError to get error message)"; } llvm_unreachable("Unhandled error code"); } @@ -58,10 +65,29 @@ static ManagedStatic<OrcErrorCategory> OrcErrCat; namespace llvm { namespace orc { -Error orcError(OrcErrorCode ErrCode) { +char JITSymbolNotFound::ID = 0; + +std::error_code orcError(OrcErrorCode ErrCode) { + typedef std::underlying_type<OrcErrorCode>::type UT; + return std::error_code(static_cast<UT>(ErrCode), *OrcErrCat); +} + +JITSymbolNotFound::JITSymbolNotFound(std::string SymbolName) + : SymbolName(std::move(SymbolName)) {} + +std::error_code JITSymbolNotFound::convertToErrorCode() const { typedef std::underlying_type<OrcErrorCode>::type UT; - return errorCodeToError( - std::error_code(static_cast<UT>(ErrCode), *OrcErrCat)); + return std::error_code(static_cast<UT>(OrcErrorCode::JITSymbolNotFound), + *OrcErrCat); } + +void JITSymbolNotFound::log(raw_ostream &OS) const { + OS << "Could not find symbol '" << SymbolName << "'"; +} + +const std::string &JITSymbolNotFound::getSymbolName() const { + return SymbolName; +} + } } diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.cpp b/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.cpp index b7a68e0..f89f21a 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.cpp +++ b/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.cpp @@ -124,5 +124,10 @@ OrcMCJITReplacement::runFunction(Function *F, llvm_unreachable("Full-featured argument passing not supported yet!"); } +void OrcMCJITReplacement::runStaticConstructorsDestructors(bool isDtors) { + for (auto &M : LocalModules) + ExecutionEngine::runStaticConstructorsDestructors(*M, isDtors); +} + } // End namespace orc. } // End namespace llvm. diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.h b/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.h index af70960..346a404 100644 --- a/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.h +++ b/contrib/llvm/lib/ExecutionEngine/Orc/OrcMCJITReplacement.h @@ -1,4 +1,4 @@ -//===---- OrcMCJITReplacement.h - Orc based MCJIT replacement ---*- C++ -*-===// +//===- OrcMCJITReplacement.h - Orc based MCJIT replacement ------*- C++ -*-===// // // The LLVM Compiler Infrastructure // @@ -20,13 +20,16 @@ #include "llvm/ExecutionEngine/ExecutionEngine.h" #include "llvm/ExecutionEngine/GenericValue.h" #include "llvm/ExecutionEngine/JITSymbol.h" -#include "llvm/ExecutionEngine/RuntimeDyld.h" #include "llvm/ExecutionEngine/Orc/CompileUtils.h" #include "llvm/ExecutionEngine/Orc/IRCompileLayer.h" #include "llvm/ExecutionEngine/Orc/LazyEmittingLayer.h" -#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h" +#include "llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h" +#include "llvm/ExecutionEngine/RTDyldMemoryManager.h" +#include "llvm/ExecutionEngine/RuntimeDyld.h" +#include "llvm/IR/DataLayout.h" #include "llvm/IR/Function.h" #include "llvm/IR/Mangler.h" +#include "llvm/IR/Module.h" #include "llvm/Object/Archive.h" #include "llvm/Object/Binary.h" #include "llvm/Object/ObjectFile.h" @@ -34,10 +37,10 @@ #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/raw_ostream.h" #include "llvm/Target/TargetMachine.h" +#include <algorithm> #include <cassert> #include <cstddef> #include <cstdint> -#include <algorithm> #include <map> #include <memory> #include <set> @@ -45,6 +48,9 @@ #include <vector> namespace llvm { + +class ObjectCache; + namespace orc { class OrcMCJITReplacement : public ExecutionEngine { @@ -94,9 +100,8 @@ class OrcMCJITReplacement : public ExecutionEngine { return ClientMM->registerEHFrames(Addr, LoadAddr, Size); } - void deregisterEHFrames(uint8_t *Addr, uint64_t LoadAddr, - size_t Size) override { - return ClientMM->deregisterEHFrames(Addr, LoadAddr, Size); + void deregisterEHFrames() override { + return ClientMM->deregisterEHFrames(); } void notifyObjectLoaded(RuntimeDyld &RTDyld, @@ -152,7 +157,6 @@ class OrcMCJITReplacement : public ExecutionEngine { }; private: - static ExecutionEngine * createOrcMCJITReplacement(std::string *ErrorMsg, std::shared_ptr<MCJITMemoryManager> MemMgr, @@ -163,24 +167,26 @@ private: } public: - static void Register() { - OrcMCJITReplacementCtor = createOrcMCJITReplacement; - } - OrcMCJITReplacement( std::shared_ptr<MCJITMemoryManager> MemMgr, std::shared_ptr<JITSymbolResolver> ClientResolver, std::unique_ptr<TargetMachine> TM) : ExecutionEngine(TM->createDataLayout()), TM(std::move(TM)), - MemMgr(*this, std::move(MemMgr)), Resolver(*this), + MemMgr(std::make_shared<MCJITReplacementMemMgr>(*this, + std::move(MemMgr))), + Resolver(std::make_shared<LinkingResolver>(*this)), ClientResolver(std::move(ClientResolver)), NotifyObjectLoaded(*this), NotifyFinalized(*this), - ObjectLayer(NotifyObjectLoaded, NotifyFinalized), + ObjectLayer([this]() { return this->MemMgr; }, NotifyObjectLoaded, + NotifyFinalized), CompileLayer(ObjectLayer, SimpleCompiler(*this->TM)), LazyEmitLayer(CompileLayer) {} - void addModule(std::unique_ptr<Module> M) override { + static void Register() { + OrcMCJITReplacementCtor = createOrcMCJITReplacement; + } + void addModule(std::unique_ptr<Module> M) override { // If this module doesn't have a DataLayout attached then attach the // default. if (M->getDataLayout().isDefault()) { @@ -188,32 +194,47 @@ public: } else { assert(M->getDataLayout() == getDataLayout() && "DataLayout Mismatch"); } - Modules.push_back(std::move(M)); - std::vector<Module *> Ms; - Ms.push_back(&*Modules.back()); - LazyEmitLayer.addModuleSet(std::move(Ms), &MemMgr, &Resolver); + auto *MPtr = M.release(); + ShouldDelete[MPtr] = true; + auto Deleter = [this](Module *Mod) { + auto I = ShouldDelete.find(Mod); + if (I != ShouldDelete.end() && I->second) + delete Mod; + }; + LocalModules.push_back(std::shared_ptr<Module>(MPtr, std::move(Deleter))); + cantFail(LazyEmitLayer.addModule(LocalModules.back(), Resolver)); } void addObjectFile(std::unique_ptr<object::ObjectFile> O) override { - std::vector<std::unique_ptr<object::ObjectFile>> Objs; - Objs.push_back(std::move(O)); - ObjectLayer.addObjectSet(std::move(Objs), &MemMgr, &Resolver); + auto Obj = + std::make_shared<object::OwningBinary<object::ObjectFile>>(std::move(O), + nullptr); + cantFail(ObjectLayer.addObject(std::move(Obj), Resolver)); } void addObjectFile(object::OwningBinary<object::ObjectFile> O) override { - std::vector<std::unique_ptr<object::OwningBinary<object::ObjectFile>>> Objs; - Objs.push_back( - llvm::make_unique<object::OwningBinary<object::ObjectFile>>( - std::move(O))); - ObjectLayer.addObjectSet(std::move(Objs), &MemMgr, &Resolver); + auto Obj = + std::make_shared<object::OwningBinary<object::ObjectFile>>(std::move(O)); + cantFail(ObjectLayer.addObject(std::move(Obj), Resolver)); } void addArchive(object::OwningBinary<object::Archive> A) override { Archives.push_back(std::move(A)); } + + bool removeModule(Module *M) override { + for (auto I = LocalModules.begin(), E = LocalModules.end(); I != E; ++I) { + if (I->get() == M) { + ShouldDelete[M] = false; + LocalModules.erase(I); + return true; + } + } + return false; + } uint64_t getSymbolAddress(StringRef Name) { - return findSymbol(Name).getAddress(); + return cantFail(findSymbol(Name).getAddress()); } JITSymbol findSymbol(StringRef Name) { @@ -257,13 +278,15 @@ public: ArrayRef<GenericValue> ArgValues) override; void setObjectCache(ObjectCache *NewCache) override { - CompileLayer.setObjectCache(NewCache); + CompileLayer.getCompiler().setObjectCache(NewCache); } void setProcessAllSections(bool ProcessAllSections) override { ObjectLayer.setProcessAllSections(ProcessAllSections); } + void runStaticConstructorsDestructors(bool isDtors) override; + private: JITSymbol findMangledSymbol(StringRef Name) { if (auto Sym = LazyEmitLayer.findSymbol(Name, false)) @@ -295,10 +318,12 @@ private: } std::unique_ptr<object::Binary> &ChildBin = ChildBinOrErr.get(); if (ChildBin->isObject()) { - std::vector<std::unique_ptr<object::ObjectFile>> ObjSet; - ObjSet.push_back(std::unique_ptr<object::ObjectFile>( - static_cast<object::ObjectFile *>(ChildBin.release()))); - ObjectLayer.addObjectSet(std::move(ObjSet), &MemMgr, &Resolver); + std::unique_ptr<object::ObjectFile> ChildObj( + static_cast<object::ObjectFile*>(ChildBinOrErr->release())); + auto Obj = + std::make_shared<object::OwningBinary<object::ObjectFile>>( + std::move(ChildObj), nullptr); + cantFail(ObjectLayer.addObject(std::move(Obj), Resolver)); if (auto Sym = ObjectLayer.findSymbol(Name, true)) return Sym; } @@ -309,34 +334,19 @@ private: class NotifyObjectLoadedT { public: - typedef std::vector<std::unique_ptr<RuntimeDyld::LoadedObjectInfo>> - LoadedObjInfoListT; + using LoadedObjInfoListT = + std::vector<std::unique_ptr<RuntimeDyld::LoadedObjectInfo>>; NotifyObjectLoadedT(OrcMCJITReplacement &M) : M(M) {} - template <typename ObjListT> - void operator()(ObjectLinkingLayerBase::ObjSetHandleT H, - const ObjListT &Objects, - const LoadedObjInfoListT &Infos) const { + void operator()(RTDyldObjectLinkingLayerBase::ObjHandleT H, + const RTDyldObjectLinkingLayer::ObjectPtr &Obj, + const LoadedObjectInfo &Info) const { M.UnfinalizedSections[H] = std::move(M.SectionsAllocatedSinceLastLoad); M.SectionsAllocatedSinceLastLoad = SectionAddrSet(); - assert(Objects.size() == Infos.size() && - "Incorrect number of Infos for Objects."); - for (unsigned I = 0; I < Objects.size(); ++I) - M.MemMgr.notifyObjectLoaded(&M, getObject(*Objects[I])); + M.MemMgr->notifyObjectLoaded(&M, *Obj->getBinary()); } - private: - static const object::ObjectFile& getObject(const object::ObjectFile &Obj) { - return Obj; - } - - template <typename ObjT> - static const object::ObjectFile& - getObject(const object::OwningBinary<ObjT> &Obj) { - return *Obj.getBinary(); - } - OrcMCJITReplacement &M; }; @@ -344,7 +354,7 @@ private: public: NotifyFinalizedT(OrcMCJITReplacement &M) : M(M) {} - void operator()(ObjectLinkingLayerBase::ObjSetHandleT H) { + void operator()(RTDyldObjectLinkingLayerBase::ObjHandleT H) { M.UnfinalizedSections.erase(H); } @@ -361,13 +371,13 @@ private: return MangledName; } - typedef ObjectLinkingLayer<NotifyObjectLoadedT> ObjectLayerT; - typedef IRCompileLayer<ObjectLayerT> CompileLayerT; - typedef LazyEmittingLayer<CompileLayerT> LazyEmitLayerT; + using ObjectLayerT = RTDyldObjectLinkingLayer; + using CompileLayerT = IRCompileLayer<ObjectLayerT, orc::SimpleCompiler>; + using LazyEmitLayerT = LazyEmittingLayer<CompileLayerT>; std::unique_ptr<TargetMachine> TM; - MCJITReplacementMemMgr MemMgr; - LinkingResolver Resolver; + std::shared_ptr<MCJITReplacementMemMgr> MemMgr; + std::shared_ptr<LinkingResolver> Resolver; std::shared_ptr<JITSymbolResolver> ClientResolver; Mangler Mang; @@ -381,21 +391,24 @@ private: // We need to store ObjLayerT::ObjSetHandles for each of the object sets // that have been emitted but not yet finalized so that we can forward the // mapSectionAddress calls appropriately. - typedef std::set<const void *> SectionAddrSet; - struct ObjSetHandleCompare { - bool operator()(ObjectLayerT::ObjSetHandleT H1, - ObjectLayerT::ObjSetHandleT H2) const { + using SectionAddrSet = std::set<const void *>; + struct ObjHandleCompare { + bool operator()(ObjectLayerT::ObjHandleT H1, + ObjectLayerT::ObjHandleT H2) const { return &*H1 < &*H2; } }; SectionAddrSet SectionsAllocatedSinceLastLoad; - std::map<ObjectLayerT::ObjSetHandleT, SectionAddrSet, ObjSetHandleCompare> + std::map<ObjectLayerT::ObjHandleT, SectionAddrSet, ObjHandleCompare> UnfinalizedSections; + std::map<Module*, bool> ShouldDelete; + std::vector<std::shared_ptr<Module>> LocalModules; std::vector<object::OwningBinary<object::Archive>> Archives; }; } // end namespace orc + } // end namespace llvm #endif // LLVM_LIB_EXECUTIONENGINE_ORC_MCJITREPLACEMENT_H diff --git a/contrib/llvm/lib/ExecutionEngine/Orc/RPCUtils.cpp b/contrib/llvm/lib/ExecutionEngine/Orc/RPCUtils.cpp new file mode 100644 index 0000000..2a7ab5c --- /dev/null +++ b/contrib/llvm/lib/ExecutionEngine/Orc/RPCUtils.cpp @@ -0,0 +1,55 @@ +//===--------------- RPCUtils.cpp - RPCUtils implementation ---------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// RPCUtils implementation. +// +//===----------------------------------------------------------------------===// + +#include "llvm/ExecutionEngine/Orc/RPCUtils.h" + +char llvm::orc::rpc::RPCFatalError::ID = 0; +char llvm::orc::rpc::ConnectionClosed::ID = 0; +char llvm::orc::rpc::ResponseAbandoned::ID = 0; +char llvm::orc::rpc::CouldNotNegotiate::ID = 0; + +namespace llvm { +namespace orc { +namespace rpc { + +std::error_code ConnectionClosed::convertToErrorCode() const { + return orcError(OrcErrorCode::RPCConnectionClosed); +} + +void ConnectionClosed::log(raw_ostream &OS) const { + OS << "RPC connection already closed"; +} + +std::error_code ResponseAbandoned::convertToErrorCode() const { + return orcError(OrcErrorCode::RPCResponseAbandoned); +} + +void ResponseAbandoned::log(raw_ostream &OS) const { + OS << "RPC response abandoned"; +} + +CouldNotNegotiate::CouldNotNegotiate(std::string Signature) + : Signature(std::move(Signature)) {} + +std::error_code CouldNotNegotiate::convertToErrorCode() const { + return orcError(OrcErrorCode::RPCCouldNotNegotiateFunction); +} + +void CouldNotNegotiate::log(raw_ostream &OS) const { + OS << "Could not negotiate RPC function " << Signature; +} + + +} // end namespace rpc +} // end namespace orc +} // end namespace llvm diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp index de73fbd..99e84b7 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp @@ -134,6 +134,18 @@ void RTDyldMemoryManager::deregisterEHFramesInProcess(uint8_t *Addr, #endif +void RTDyldMemoryManager::registerEHFrames(uint8_t *Addr, uint64_t LoadAddr, + size_t Size) { + registerEHFramesInProcess(Addr, Size); + EHFrames.push_back({Addr, Size}); +} + +void RTDyldMemoryManager::deregisterEHFrames() { + for (auto &Frame : EHFrames) + deregisterEHFramesInProcess(Frame.Addr, Frame.Size); + EHFrames.clear(); +} + static int jit_noop() { return 0; } diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp index 63b56f7..8198836 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp @@ -12,13 +12,13 @@ //===----------------------------------------------------------------------===// #include "llvm/ExecutionEngine/RuntimeDyld.h" -#include "RuntimeDyldCheckerImpl.h" #include "RuntimeDyldCOFF.h" +#include "RuntimeDyldCheckerImpl.h" #include "RuntimeDyldELF.h" #include "RuntimeDyldImpl.h" #include "RuntimeDyldMachO.h" -#include "llvm/Object/ELFObjectFile.h" #include "llvm/Object/COFF.h" +#include "llvm/Object/ELFObjectFile.h" #include "llvm/Support/ManagedStatic.h" #include "llvm/Support/MathExtras.h" #include "llvm/Support/MutexGuard.h" @@ -73,7 +73,9 @@ namespace llvm { void RuntimeDyldImpl::registerEHFrames() {} -void RuntimeDyldImpl::deregisterEHFrames() {} +void RuntimeDyldImpl::deregisterEHFrames() { + MemMgr.deregisterEHFrames(); +} #ifndef NDEBUG static void dumpSectionMemory(const SectionEntry &S, StringRef State) { @@ -126,7 +128,10 @@ void RuntimeDyldImpl::resolveRelocations() { ); // First, resolve relocations associated with external symbols. - resolveExternalSymbols(); + if (auto Err = resolveExternalSymbols()) { + HasError = true; + ErrorStr = toString(std::move(Err)); + } // Iterate over all outstanding relocations for (auto it = Relocations.begin(), e = Relocations.end(); it != e; ++it) { @@ -241,9 +246,11 @@ RuntimeDyldImpl::loadObjectImpl(const object::ObjectFile &Obj) { continue; // Then check the symbol resolver to see if there's a definition // elsewhere in this logical dylib. - if (auto Sym = Resolver.findSymbolInLogicalDylib(Name)) + if (auto Sym = Resolver.findSymbolInLogicalDylib(Name)) { if (Sym.getFlags().isStrongDefinition()) continue; + } else if (auto Err = Sym.takeError()) + return std::move(Err); // else JITSymFlags &= ~JITSymbolFlags::Weak; } @@ -443,7 +450,7 @@ Error RuntimeDyldImpl::computeTotalAllocSize(const ObjectFile &Obj, SI != SE; ++SI) { const SectionRef &Section = *SI; - bool IsRequired = isRequiredForExecution(Section); + bool IsRequired = isRequiredForExecution(Section) || ProcessAllSections; // Consider only the sections that are required to be loaded for execution if (IsRequired) { @@ -484,6 +491,14 @@ Error RuntimeDyldImpl::computeTotalAllocSize(const ObjectFile &Obj, } } + // Compute Global Offset Table size. If it is not zero we + // also update alignment, which is equal to a size of a + // single GOT entry. + if (unsigned GotSize = computeGOTSize(Obj)) { + RWSectionSizes.push_back(GotSize); + RWDataAlign = std::max<uint32_t>(RWDataAlign, getGOTEntrySize()); + } + // Compute the size of all common symbols uint64_t CommonSize = 0; uint32_t CommonAlign = 1; @@ -518,6 +533,24 @@ Error RuntimeDyldImpl::computeTotalAllocSize(const ObjectFile &Obj, return Error::success(); } +// compute GOT size +unsigned RuntimeDyldImpl::computeGOTSize(const ObjectFile &Obj) { + size_t GotEntrySize = getGOTEntrySize(); + if (!GotEntrySize) + return 0; + + size_t GotSize = 0; + for (section_iterator SI = Obj.section_begin(), SE = Obj.section_end(); + SI != SE; ++SI) { + + for (const RelocationRef &Reloc : SI->relocations()) + if (relocationNeedsGot(Reloc)) + GotSize += GotEntrySize; + } + + return GotSize; +} + // compute stub buffer size for the given section unsigned RuntimeDyldImpl::computeSectionStubBufSize(const ObjectFile &Obj, const SectionRef &Section) { @@ -717,8 +750,8 @@ RuntimeDyldImpl::emitSection(const ObjectFile &Obj, Alignment = std::max(Alignment, getStubAlignment()); // Some sections, such as debug info, don't need to be loaded for execution. - // Leave those where they are. - if (IsRequired) { + // Process those only if explicitly requested. + if (IsRequired || ProcessAllSections) { Allocate = DataSize + PaddingSize + StubBufSize; if (!Allocate) Allocate = 1; @@ -762,6 +795,10 @@ RuntimeDyldImpl::emitSection(const ObjectFile &Obj, Sections.push_back( SectionEntry(Name, Addr, DataSize, Allocate, (uintptr_t)pData)); + // Debug info sections are linked as if their load address was zero + if (!IsRequired) + Sections.back().setLoadAddress(0); + if (Checker) Checker->registerSection(Obj.getFileName(), SectionID); @@ -921,7 +958,7 @@ void RuntimeDyldImpl::resolveRelocationList(const RelocationList &Relocs, } } -void RuntimeDyldImpl::resolveExternalSymbols() { +Error RuntimeDyldImpl::resolveExternalSymbols() { while (!ExternalSymbolRelocations.empty()) { StringMap<RelocationList>::iterator i = ExternalSymbolRelocations.begin(); @@ -939,10 +976,24 @@ void RuntimeDyldImpl::resolveExternalSymbols() { // This is an external symbol, try to get its address from the symbol // resolver. // First search for the symbol in this logical dylib. - Addr = Resolver.findSymbolInLogicalDylib(Name.data()).getAddress(); + if (auto Sym = Resolver.findSymbolInLogicalDylib(Name.data())) { + if (auto AddrOrErr = Sym.getAddress()) + Addr = *AddrOrErr; + else + return AddrOrErr.takeError(); + } else if (auto Err = Sym.takeError()) + return Err; + // If that fails, try searching for an external symbol. - if (!Addr) - Addr = Resolver.findSymbol(Name.data()).getAddress(); + if (!Addr) { + if (auto Sym = Resolver.findSymbol(Name.data())) { + if (auto AddrOrErr = Sym.getAddress()) + Addr = *AddrOrErr; + else + return AddrOrErr.takeError(); + } else if (auto Err = Sym.takeError()) + return Err; + } // The call to getSymbolAddress may have caused additional modules to // be loaded, which may have added new entries to the // ExternalSymbolRelocations map. Consquently, we need to update our @@ -977,6 +1028,8 @@ void RuntimeDyldImpl::resolveExternalSymbols() { ExternalSymbolRelocations.erase(i); } + + return Error::success(); } //===----------------------------------------------------------------------===// diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp index 1bd28ef..1c54ad6 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp @@ -27,9 +27,12 @@ using namespace llvm::object; namespace { class LoadedCOFFObjectInfo final - : public RuntimeDyld::LoadedObjectInfoHelper<LoadedCOFFObjectInfo> { + : public LoadedObjectInfoHelper<LoadedCOFFObjectInfo, + RuntimeDyld::LoadedObjectInfo> { public: - LoadedCOFFObjectInfo(RuntimeDyldImpl &RTDyld, ObjSectionToIDMap ObjSecToIDMap) + LoadedCOFFObjectInfo( + RuntimeDyldImpl &RTDyld, + RuntimeDyld::LoadedObjectInfo::ObjSectionToIDMap ObjSecToIDMap) : LoadedObjectInfoHelper(RTDyld, std::move(ObjSecToIDMap)) {} OwningBinary<ObjectFile> diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldChecker.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldChecker.cpp index 7bfa794..5bc7434 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldChecker.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldChecker.cpp @@ -742,7 +742,7 @@ uint64_t RuntimeDyldCheckerImpl::getSymbolLocalAddr(StringRef Symbol) const { uint64_t RuntimeDyldCheckerImpl::getSymbolRemoteAddr(StringRef Symbol) const { if (auto InternalSymbol = getRTDyld().getSymbol(Symbol)) return InternalSymbol.getAddress(); - return getRTDyld().Resolver.findSymbol(Symbol).getAddress(); + return cantFail(getRTDyld().Resolver.findSymbol(Symbol).getAddress()); } uint64_t RuntimeDyldCheckerImpl::readMemoryAtAddr(uint64_t SrcAddr, @@ -861,6 +861,15 @@ RuntimeDyldCheckerImpl::getSubsectionStartingAt(StringRef Name) const { SymInfo.getOffset()); } +Optional<uint64_t> +RuntimeDyldCheckerImpl::getSectionLoadAddress(void *LocalAddress) const { + for (auto &S : getRTDyld().Sections) { + if (S.getAddress() == LocalAddress) + return S.getLoadAddress(); + } + return Optional<uint64_t>(); +} + void RuntimeDyldCheckerImpl::registerSection( StringRef FilePath, unsigned SectionID) { StringRef FileName = sys::path::filename(FilePath); @@ -935,3 +944,8 @@ RuntimeDyldChecker::getSectionAddr(StringRef FileName, StringRef SectionName, bool LocalAddress) { return Impl->getSectionAddr(FileName, SectionName, LocalAddress); } + +Optional<uint64_t> +RuntimeDyldChecker::getSectionLoadAddress(void *LocalAddress) const { + return Impl->getSectionLoadAddress(LocalAddress); +} diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCheckerImpl.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCheckerImpl.h index b7263be..b462ef2 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCheckerImpl.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCheckerImpl.h @@ -60,6 +60,8 @@ private: bool IsInsideLoad) const; StringRef getSubsectionStartingAt(StringRef Name) const; + Optional<uint64_t> getSectionLoadAddress(void *LocalAddr) const; + void registerSection(StringRef FilePath, unsigned SectionID); void registerStubMap(StringRef FilePath, unsigned SectionID, const RuntimeDyldImpl::StubMap &RTDyldStubs); diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp index 05615d3..77c9684 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp @@ -18,10 +18,10 @@ #include "llvm/ADT/STLExtras.h" #include "llvm/ADT/StringRef.h" #include "llvm/ADT/Triple.h" +#include "llvm/BinaryFormat/ELF.h" #include "llvm/MC/MCStreamer.h" #include "llvm/Object/ELFObjectFile.h" #include "llvm/Object/ObjectFile.h" -#include "llvm/Support/ELF.h" #include "llvm/Support/Endian.h" #include "llvm/Support/MemoryBuffer.h" #include "llvm/Support/TargetRegistry.h" @@ -78,11 +78,11 @@ public: void updateSymbolAddress(const SymbolRef &SymRef, uint64_t Addr); // Methods for type inquiry through isa, cast and dyn_cast - static inline bool classof(const Binary *v) { + static bool classof(const Binary *v) { return (isa<ELFObjectFile<ELFT>>(v) && classof(cast<ELFObjectFile<ELFT>>(v))); } - static inline bool classof(const ELFObjectFile<ELFT> *v) { + static bool classof(const ELFObjectFile<ELFT> *v) { return v->isDyldType(); } }; @@ -123,7 +123,8 @@ void DyldELFObject<ELFT>::updateSymbolAddress(const SymbolRef &SymRef, } class LoadedELFObjectInfo final - : public RuntimeDyld::LoadedObjectInfoHelper<LoadedELFObjectInfo> { + : public LoadedObjectInfoHelper<LoadedELFObjectInfo, + RuntimeDyld::LoadedObjectInfo> { public: LoadedELFObjectInfo(RuntimeDyldImpl &RTDyld, ObjSectionToIDMap ObjSecToIDMap) : LoadedObjectInfoHelper(RTDyld, std::move(ObjSecToIDMap)) {} @@ -221,22 +222,10 @@ void RuntimeDyldELF::registerEHFrames() { uint64_t EHFrameLoadAddr = Sections[EHFrameSID].getLoadAddress(); size_t EHFrameSize = Sections[EHFrameSID].getSize(); MemMgr.registerEHFrames(EHFrameAddr, EHFrameLoadAddr, EHFrameSize); - RegisteredEHFrameSections.push_back(EHFrameSID); } UnregisteredEHFrameSections.clear(); } -void RuntimeDyldELF::deregisterEHFrames() { - for (int i = 0, e = RegisteredEHFrameSections.size(); i != e; ++i) { - SID EHFrameSID = RegisteredEHFrameSections[i]; - uint8_t *EHFrameAddr = Sections[EHFrameSID].getAddress(); - uint64_t EHFrameLoadAddr = Sections[EHFrameSID].getLoadAddress(); - size_t EHFrameSize = Sections[EHFrameSID].getSize(); - MemMgr.deregisterEHFrames(EHFrameAddr, EHFrameLoadAddr, EHFrameSize); - } - RegisteredEHFrameSections.clear(); -} - std::unique_ptr<RuntimeDyldELF> llvm::RuntimeDyldELF::create(Triple::ArchType Arch, RuntimeDyld::MemoryManager &MemMgr, @@ -272,6 +261,8 @@ void RuntimeDyldELF::resolveX86_64Relocation(const SectionEntry &Section, default: llvm_unreachable("Relocation type not implemented yet!"); break; + case ELF::R_X86_64_NONE: + break; case ELF::R_X86_64_64: { support::ulittle64_t::ref(Section.getAddressWithOffset(Offset)) = Value + Addend; @@ -419,6 +410,18 @@ void RuntimeDyldELF::resolveAArch64Relocation(const SectionEntry &Section, // from bits 11:0 of X or32AArch64Imm(TargetPtr, Value + Addend); break; + case ELF::R_AARCH64_LDST8_ABS_LO12_NC: + // Operation: S + A + // Immediate goes in bits 21:10 of LD/ST instruction, taken + // from bits 11:0 of X + or32AArch64Imm(TargetPtr, getBits(Value + Addend, 0, 11)); + break; + case ELF::R_AARCH64_LDST16_ABS_LO12_NC: + // Operation: S + A + // Immediate goes in bits 21:10 of LD/ST instruction, taken + // from bits 11:1 of X + or32AArch64Imm(TargetPtr, getBits(Value + Addend, 1, 11)); + break; case ELF::R_AARCH64_LDST32_ABS_LO12_NC: // Operation: S + A // Immediate goes in bits 21:10 of LD/ST instruction, taken @@ -431,6 +434,12 @@ void RuntimeDyldELF::resolveAArch64Relocation(const SectionEntry &Section, // from bits 11:3 of X or32AArch64Imm(TargetPtr, getBits(Value + Addend, 3, 11)); break; + case ELF::R_AARCH64_LDST128_ABS_LO12_NC: + // Operation: S + A + // Immediate goes in bits 21:10 of LD/ST instruction, taken + // from bits 11:4 of X + or32AArch64Imm(TargetPtr, getBits(Value + Addend, 4, 11)); + break; } } @@ -729,23 +738,23 @@ void RuntimeDyldELF::resolvePPC64Relocation(const SectionEntry &Section, writeInt16BE(LocalAddress, applyPPCha(Delta)); } break; case ELF::R_PPC64_ADDR32: { - int32_t Result = static_cast<int32_t>(Value + Addend); - if (SignExtend32<32>(Result) != Result) + int64_t Result = static_cast<int64_t>(Value + Addend); + if (SignExtend64<32>(Result) != Result) llvm_unreachable("Relocation R_PPC64_ADDR32 overflow"); writeInt32BE(LocalAddress, Result); } break; case ELF::R_PPC64_REL24: { uint64_t FinalAddress = Section.getLoadAddressWithOffset(Offset); - int32_t delta = static_cast<int32_t>(Value - FinalAddress + Addend); - if (SignExtend32<26>(delta) != delta) + int64_t delta = static_cast<int64_t>(Value - FinalAddress + Addend); + if (SignExtend64<26>(delta) != delta) llvm_unreachable("Relocation R_PPC64_REL24 overflow"); // Generates a 'bl <address>' instruction writeInt32BE(LocalAddress, 0x48000001 | (delta & 0x03FFFFFC)); } break; case ELF::R_PPC64_REL32: { uint64_t FinalAddress = Section.getLoadAddressWithOffset(Offset); - int32_t delta = static_cast<int32_t>(Value - FinalAddress + Addend); - if (SignExtend32<32>(delta) != delta) + int64_t delta = static_cast<int64_t>(Value - FinalAddress + Addend); + if (SignExtend64<32>(delta) != delta) llvm_unreachable("Relocation R_PPC64_REL32 overflow"); writeInt32BE(LocalAddress, delta); } break; @@ -782,20 +791,63 @@ void RuntimeDyldELF::resolveSystemZRelocation(const SectionEntry &Section, writeInt32BE(LocalAddress, Delta / 2); break; } + case ELF::R_390_PC16: { + int64_t Delta = (Value + Addend) - Section.getLoadAddressWithOffset(Offset); + assert(int16_t(Delta) == Delta && "R_390_PC16 overflow"); + writeInt16BE(LocalAddress, Delta); + break; + } case ELF::R_390_PC32: { int64_t Delta = (Value + Addend) - Section.getLoadAddressWithOffset(Offset); assert(int32_t(Delta) == Delta && "R_390_PC32 overflow"); writeInt32BE(LocalAddress, Delta); break; } - case ELF::R_390_64: - writeInt64BE(LocalAddress, Value + Addend); - break; case ELF::R_390_PC64: { int64_t Delta = (Value + Addend) - Section.getLoadAddressWithOffset(Offset); writeInt64BE(LocalAddress, Delta); break; } + case ELF::R_390_8: + *LocalAddress = (uint8_t)(Value + Addend); + break; + case ELF::R_390_16: + writeInt16BE(LocalAddress, Value + Addend); + break; + case ELF::R_390_32: + writeInt32BE(LocalAddress, Value + Addend); + break; + case ELF::R_390_64: + writeInt64BE(LocalAddress, Value + Addend); + break; + } +} + +void RuntimeDyldELF::resolveBPFRelocation(const SectionEntry &Section, + uint64_t Offset, uint64_t Value, + uint32_t Type, int64_t Addend) { + bool isBE = Arch == Triple::bpfeb; + + switch (Type) { + default: + llvm_unreachable("Relocation type not implemented yet!"); + break; + case ELF::R_BPF_NONE: + break; + case ELF::R_BPF_64_64: { + write(isBE, Section.getAddressWithOffset(Offset), Value + Addend); + DEBUG(dbgs() << "Writing " << format("%p", (Value + Addend)) << " at " + << format("%p\n", Section.getAddressWithOffset(Offset))); + break; + } + case ELF::R_BPF_64_32: { + Value += Addend; + assert(Value <= UINT32_MAX); + write(isBE, Section.getAddressWithOffset(Offset), static_cast<uint32_t>(Value)); + DEBUG(dbgs() << "Writing " << format("%p", Value) << " at " + << format("%p\n", Section.getAddressWithOffset(Offset))); + break; + } } } @@ -859,6 +911,10 @@ void RuntimeDyldELF::resolveRelocation(const SectionEntry &Section, case Triple::systemz: resolveSystemZRelocation(Section, Offset, Value, Type, Addend); break; + case Triple::bpfel: + case Triple::bpfeb: + resolveBPFRelocation(Section, Offset, Value, Type, Addend); + break; default: llvm_unreachable("Unsupported CPU type!"); } @@ -900,7 +956,7 @@ uint32_t RuntimeDyldELF::getMatchingLoRelocation(uint32_t RelType, } // Sometimes we don't need to create thunk for a branch. -// This typically happens when branch target is located +// This typically happens when branch target is located // in the same object file. In such case target is either // a weak symbol or symbol in a different executable section. // This function checks if branch target is located in the @@ -941,6 +997,61 @@ bool RuntimeDyldELF::resolveAArch64ShortBranch( return true; } +void RuntimeDyldELF::resolveAArch64Branch(unsigned SectionID, + const RelocationValueRef &Value, + relocation_iterator RelI, + StubMap &Stubs) { + + DEBUG(dbgs() << "\t\tThis is an AArch64 branch relocation."); + SectionEntry &Section = Sections[SectionID]; + + uint64_t Offset = RelI->getOffset(); + unsigned RelType = RelI->getType(); + // Look for an existing stub. + StubMap::const_iterator i = Stubs.find(Value); + if (i != Stubs.end()) { + resolveRelocation(Section, Offset, + (uint64_t)Section.getAddressWithOffset(i->second), + RelType, 0); + DEBUG(dbgs() << " Stub function found\n"); + } else if (!resolveAArch64ShortBranch(SectionID, RelI, Value)) { + // Create a new stub function. + DEBUG(dbgs() << " Create a new stub function\n"); + Stubs[Value] = Section.getStubOffset(); + uint8_t *StubTargetAddr = createStubFunction( + Section.getAddressWithOffset(Section.getStubOffset())); + + RelocationEntry REmovz_g3(SectionID, StubTargetAddr - Section.getAddress(), + ELF::R_AARCH64_MOVW_UABS_G3, Value.Addend); + RelocationEntry REmovk_g2(SectionID, + StubTargetAddr - Section.getAddress() + 4, + ELF::R_AARCH64_MOVW_UABS_G2_NC, Value.Addend); + RelocationEntry REmovk_g1(SectionID, + StubTargetAddr - Section.getAddress() + 8, + ELF::R_AARCH64_MOVW_UABS_G1_NC, Value.Addend); + RelocationEntry REmovk_g0(SectionID, + StubTargetAddr - Section.getAddress() + 12, + ELF::R_AARCH64_MOVW_UABS_G0_NC, Value.Addend); + + if (Value.SymbolName) { + addRelocationForSymbol(REmovz_g3, Value.SymbolName); + addRelocationForSymbol(REmovk_g2, Value.SymbolName); + addRelocationForSymbol(REmovk_g1, Value.SymbolName); + addRelocationForSymbol(REmovk_g0, Value.SymbolName); + } else { + addRelocationForSection(REmovz_g3, Value.SectionID); + addRelocationForSection(REmovk_g2, Value.SectionID); + addRelocationForSection(REmovk_g1, Value.SectionID); + addRelocationForSection(REmovk_g0, Value.SectionID); + } + resolveRelocation(Section, Offset, + reinterpret_cast<uint64_t>(Section.getAddressWithOffset( + Section.getStubOffset())), + RelType, 0); + Section.advanceStubOffset(getMaxStubSize()); + } +} + Expected<relocation_iterator> RuntimeDyldELF::processRelocationRef( unsigned SectionID, relocation_iterator RelI, const ObjectFile &O, @@ -1035,55 +1146,22 @@ RuntimeDyldELF::processRelocationRef( DEBUG(dbgs() << "\t\tSectionID: " << SectionID << " Offset: " << Offset << "\n"); - if ((Arch == Triple::aarch64 || Arch == Triple::aarch64_be) && - (RelType == ELF::R_AARCH64_CALL26 || RelType == ELF::R_AARCH64_JUMP26)) { - // This is an AArch64 branch relocation, need to use a stub function. - DEBUG(dbgs() << "\t\tThis is an AArch64 branch relocation."); - SectionEntry &Section = Sections[SectionID]; - - // Look for an existing stub. - StubMap::const_iterator i = Stubs.find(Value); - if (i != Stubs.end()) { - resolveRelocation(Section, Offset, - (uint64_t)Section.getAddressWithOffset(i->second), - RelType, 0); - DEBUG(dbgs() << " Stub function found\n"); - } else if (!resolveAArch64ShortBranch(SectionID, RelI, Value)) { - // Create a new stub function. - DEBUG(dbgs() << " Create a new stub function\n"); - Stubs[Value] = Section.getStubOffset(); - uint8_t *StubTargetAddr = createStubFunction( - Section.getAddressWithOffset(Section.getStubOffset())); - - RelocationEntry REmovz_g3(SectionID, - StubTargetAddr - Section.getAddress(), - ELF::R_AARCH64_MOVW_UABS_G3, Value.Addend); - RelocationEntry REmovk_g2(SectionID, StubTargetAddr - - Section.getAddress() + 4, - ELF::R_AARCH64_MOVW_UABS_G2_NC, Value.Addend); - RelocationEntry REmovk_g1(SectionID, StubTargetAddr - - Section.getAddress() + 8, - ELF::R_AARCH64_MOVW_UABS_G1_NC, Value.Addend); - RelocationEntry REmovk_g0(SectionID, StubTargetAddr - - Section.getAddress() + 12, - ELF::R_AARCH64_MOVW_UABS_G0_NC, Value.Addend); - - if (Value.SymbolName) { - addRelocationForSymbol(REmovz_g3, Value.SymbolName); - addRelocationForSymbol(REmovk_g2, Value.SymbolName); - addRelocationForSymbol(REmovk_g1, Value.SymbolName); - addRelocationForSymbol(REmovk_g0, Value.SymbolName); - } else { - addRelocationForSection(REmovz_g3, Value.SectionID); - addRelocationForSection(REmovk_g2, Value.SectionID); - addRelocationForSection(REmovk_g1, Value.SectionID); - addRelocationForSection(REmovk_g0, Value.SectionID); - } - resolveRelocation(Section, Offset, - reinterpret_cast<uint64_t>(Section.getAddressWithOffset( - Section.getStubOffset())), - RelType, 0); - Section.advanceStubOffset(getMaxStubSize()); + if ((Arch == Triple::aarch64 || Arch == Triple::aarch64_be)) { + if (RelType == ELF::R_AARCH64_CALL26 || RelType == ELF::R_AARCH64_JUMP26) { + resolveAArch64Branch(SectionID, Value, RelI, Stubs); + } else if (RelType == ELF::R_AARCH64_ADR_GOT_PAGE) { + // Craete new GOT entry or find existing one. If GOT entry is + // to be created, then we also emit ABS64 relocation for it. + uint64_t GOTOffset = findOrAllocGOTEntry(Value, ELF::R_AARCH64_ABS64); + resolveGOTOffsetRelocation(SectionID, Offset, GOTOffset + Addend, + ELF::R_AARCH64_ADR_PREL_PG_HI21); + + } else if (RelType == ELF::R_AARCH64_LD64_GOT_LO12_NC) { + uint64_t GOTOffset = findOrAllocGOTEntry(Value, ELF::R_AARCH64_ABS64); + resolveGOTOffsetRelocation(SectionID, Offset, GOTOffset + Addend, + ELF::R_AARCH64_LDST64_ABS_LO12_NC); + } else { + processSimpleRelocation(SectionID, Offset, RelType, Value); } } else if (Arch == Triple::arm) { if (RelType == ELF::R_ARM_PC24 || RelType == ELF::R_ARM_CALL || @@ -1232,7 +1310,7 @@ RuntimeDyldELF::processRelocationRef( if (i != GOTSymbolOffsets.end()) RE.SymOffset = i->second; else { - RE.SymOffset = allocateGOTEntries(SectionID, 1); + RE.SymOffset = allocateGOTEntries(1); GOTSymbolOffsets[TargetName] = RE.SymOffset; } } @@ -1247,12 +1325,13 @@ RuntimeDyldELF::processRelocationRef( Obj.getPlatformFlags(AbiVariant); AbiVariant &= ELF::EF_PPC64_ABI; // A PPC branch relocation will need a stub function if the target is - // an external symbol (Symbol::ST_Unknown) or if the target address - // is not within the signed 24-bits branch address. + // an external symbol (either Value.SymbolName is set, or SymType is + // Symbol::ST_Unknown) or if the target address is not within the + // signed 24-bits branch address. SectionEntry &Section = Sections[SectionID]; uint8_t *Target = Section.getAddressWithOffset(Offset); bool RangeOverflow = false; - if (SymType != SymbolRef::ST_Unknown) { + if (!Value.SymbolName && SymType != SymbolRef::ST_Unknown) { if (AbiVariant != 2) { // In the ELFv1 ABI, a function call may point to the .opd entry, // so the final symbol value is calculated based on the relocation @@ -1267,21 +1346,19 @@ RuntimeDyldELF::processRelocationRef( } uint8_t *RelocTarget = Sections[Value.SectionID].getAddressWithOffset(Value.Addend); - int32_t delta = static_cast<int32_t>(Target - RelocTarget); + int64_t delta = static_cast<int64_t>(Target - RelocTarget); // If it is within 26-bits branch range, just set the branch target - if (SignExtend32<26>(delta) == delta) { + if (SignExtend64<26>(delta) == delta) { RelocationEntry RE(SectionID, Offset, RelType, Value.Addend); - if (Value.SymbolName) - addRelocationForSymbol(RE, Value.SymbolName); - else - addRelocationForSection(RE, Value.SectionID); + addRelocationForSection(RE, Value.SectionID); } else { RangeOverflow = true; } } - if (SymType == SymbolRef::ST_Unknown || RangeOverflow) { - // It is an external symbol (SymbolRef::ST_Unknown) or within a range - // larger than 24-bits. + if (Value.SymbolName || SymType == SymbolRef::ST_Unknown || + RangeOverflow) { + // It is an external symbol (either Value.SymbolName is set, or + // SymType is SymbolRef::ST_Unknown) or out of range. StubMap::const_iterator i = Stubs.find(Value); if (i != Stubs.end()) { // Symbol function stub already created, just relocate to it @@ -1335,7 +1412,7 @@ RuntimeDyldELF::processRelocationRef( RelType, 0); Section.advanceStubOffset(getMaxStubSize()); } - if (SymType == SymbolRef::ST_Unknown) { + if (Value.SymbolName || SymType == SymbolRef::ST_Unknown) { // Restore the TOC for external calls if (AbiVariant == 2) writeInt32BE(Target + 4, 0xE8410018); // ld r2,28(r1) @@ -1489,14 +1566,15 @@ RuntimeDyldELF::processRelocationRef( Section.advanceStubOffset(getMaxStubSize()); // Allocate a GOT Entry - uint64_t GOTOffset = allocateGOTEntries(SectionID, 1); + uint64_t GOTOffset = allocateGOTEntries(1); // The load of the GOT address has an addend of -4 - resolveGOTOffsetRelocation(SectionID, StubOffset + 2, GOTOffset - 4); + resolveGOTOffsetRelocation(SectionID, StubOffset + 2, GOTOffset - 4, + ELF::R_X86_64_PC32); // Fill in the value of the symbol we're targeting into the GOT addRelocationForSymbol( - computeGOTOffsetRE(SectionID, GOTOffset, 0, ELF::R_X86_64_64), + computeGOTOffsetRE(GOTOffset, 0, ELF::R_X86_64_64), Value.SymbolName); } @@ -1511,11 +1589,13 @@ RuntimeDyldELF::processRelocationRef( } else if (RelType == ELF::R_X86_64_GOTPCREL || RelType == ELF::R_X86_64_GOTPCRELX || RelType == ELF::R_X86_64_REX_GOTPCRELX) { - uint64_t GOTOffset = allocateGOTEntries(SectionID, 1); - resolveGOTOffsetRelocation(SectionID, Offset, GOTOffset + Addend); + uint64_t GOTOffset = allocateGOTEntries(1); + resolveGOTOffsetRelocation(SectionID, Offset, GOTOffset + Addend, + ELF::R_X86_64_PC32); // Fill in the value of the symbol we're targeting into the GOT - RelocationEntry RE = computeGOTOffsetRE(SectionID, GOTOffset, Value.Offset, ELF::R_X86_64_64); + RelocationEntry RE = + computeGOTOffsetRE(GOTOffset, Value.Offset, ELF::R_X86_64_64); if (Value.SymbolName) addRelocationForSymbol(RE, Value.SymbolName); else @@ -1573,9 +1653,7 @@ size_t RuntimeDyldELF::getGOTEntrySize() { return Result; } -uint64_t RuntimeDyldELF::allocateGOTEntries(unsigned SectionID, unsigned no) -{ - (void)SectionID; // The GOT Section is the same for all section in the object file +uint64_t RuntimeDyldELF::allocateGOTEntries(unsigned no) { if (GOTSectionID == 0) { GOTSectionID = Sections.size(); // Reserve a section id. We'll allocate the section later @@ -1587,17 +1665,38 @@ uint64_t RuntimeDyldELF::allocateGOTEntries(unsigned SectionID, unsigned no) return StartOffset; } -void RuntimeDyldELF::resolveGOTOffsetRelocation(unsigned SectionID, uint64_t Offset, uint64_t GOTOffset) -{ +uint64_t RuntimeDyldELF::findOrAllocGOTEntry(const RelocationValueRef &Value, + unsigned GOTRelType) { + auto E = GOTOffsetMap.insert({Value, 0}); + if (E.second) { + uint64_t GOTOffset = allocateGOTEntries(1); + + // Create relocation for newly created GOT entry + RelocationEntry RE = + computeGOTOffsetRE(GOTOffset, Value.Offset, GOTRelType); + if (Value.SymbolName) + addRelocationForSymbol(RE, Value.SymbolName); + else + addRelocationForSection(RE, Value.SectionID); + + E.first->second = GOTOffset; + } + + return E.first->second; +} + +void RuntimeDyldELF::resolveGOTOffsetRelocation(unsigned SectionID, + uint64_t Offset, + uint64_t GOTOffset, + uint32_t Type) { // Fill in the relative address of the GOT Entry into the stub - RelocationEntry GOTRE(SectionID, Offset, ELF::R_X86_64_PC32, GOTOffset); + RelocationEntry GOTRE(SectionID, Offset, Type, GOTOffset); addRelocationForSection(GOTRE, GOTSectionID); } -RelocationEntry RuntimeDyldELF::computeGOTOffsetRE(unsigned SectionID, uint64_t GOTOffset, uint64_t SymbolOffset, - uint32_t Type) -{ - (void)SectionID; // The GOT Section is the same for all section in the object file +RelocationEntry RuntimeDyldELF::computeGOTOffsetRE(uint64_t GOTOffset, + uint64_t SymbolOffset, + uint32_t Type) { return RelocationEntry(GOTSectionID, GOTOffset, Type, SymbolOffset); } @@ -1663,6 +1762,19 @@ bool RuntimeDyldELF::isCompatibleFile(const object::ObjectFile &Obj) const { return Obj.isELF(); } +bool RuntimeDyldELF::relocationNeedsGot(const RelocationRef &R) const { + unsigned RelTy = R.getType(); + if (Arch == Triple::aarch64 || Arch == Triple::aarch64_be) + return RelTy == ELF::R_AARCH64_ADR_GOT_PAGE || + RelTy == ELF::R_AARCH64_LD64_GOT_LO12_NC; + + if (Arch == Triple::x86_64) + return RelTy == ELF::R_X86_64_GOTPCREL || + RelTy == ELF::R_X86_64_GOTPCRELX || + RelTy == ELF::R_X86_64_REX_GOTPCRELX; + return false; +} + bool RuntimeDyldELF::relocationNeedsStub(const RelocationRef &R) const { if (Arch != Triple::x86_64) return true; // Conservative answer diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h index d1867d0..fb5da6d 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h @@ -43,6 +43,9 @@ class RuntimeDyldELF : public RuntimeDyldImpl { bool resolveAArch64ShortBranch(unsigned SectionID, relocation_iterator RelI, const RelocationValueRef &Value); + void resolveAArch64Branch(unsigned SectionID, const RelocationValueRef &Value, + relocation_iterator RelI, StubMap &Stubs); + void resolveARMRelocation(const SectionEntry &Section, uint64_t Offset, uint32_t Value, uint32_t Type, int32_t Addend); @@ -55,6 +58,9 @@ class RuntimeDyldELF : public RuntimeDyldImpl { void resolveSystemZRelocation(const SectionEntry &Section, uint64_t Offset, uint64_t Value, uint32_t Type, int64_t Addend); + void resolveBPFRelocation(const SectionEntry &Section, uint64_t Offset, + uint64_t Value, uint32_t Type, int64_t Addend); + unsigned getMaxStubSize() override { if (Arch == Triple::aarch64 || Arch == Triple::aarch64_be) return 20; // movz; movk; movk; movk; br @@ -88,24 +94,26 @@ class RuntimeDyldELF : public RuntimeDyldImpl { ObjSectionToIDMap &LocalSections, RelocationValueRef &Rel); protected: - size_t getGOTEntrySize(); + size_t getGOTEntrySize() override; private: SectionEntry &getSection(unsigned SectionID) { return Sections[SectionID]; } // Allocate no GOT entries for use in the given section. - uint64_t allocateGOTEntries(unsigned SectionID, unsigned no); + uint64_t allocateGOTEntries(unsigned no); + + // Find GOT entry corresponding to relocation or create new one. + uint64_t findOrAllocGOTEntry(const RelocationValueRef &Value, + unsigned GOTRelType); // Resolve the relvative address of GOTOffset in Section ID and place // it at the given Offset void resolveGOTOffsetRelocation(unsigned SectionID, uint64_t Offset, - uint64_t GOTOffset); + uint64_t GOTOffset, uint32_t Type); // For a GOT entry referenced from SectionID, compute a relocation entry // that will place the final resolved value in the GOT slot - RelocationEntry computeGOTOffsetRE(unsigned SectionID, - uint64_t GOTOffset, - uint64_t SymbolOffset, + RelocationEntry computeGOTOffsetRE(uint64_t GOTOffset, uint64_t SymbolOffset, unsigned Type); // Compute the address in memory where we can find the placeholder @@ -144,8 +152,11 @@ private: // in a table until we receive a request to register all unregistered // EH frame sections with the memory manager. SmallVector<SID, 2> UnregisteredEHFrameSections; - SmallVector<SID, 2> RegisteredEHFrameSections; + // Map between GOT relocation value and corresponding GOT offset + std::map<RelocationValueRef, uint64_t> GOTOffsetMap; + + bool relocationNeedsGot(const RelocationRef &R) const override; bool relocationNeedsStub(const RelocationRef &R) const override; public: @@ -168,7 +179,6 @@ public: StubMap &Stubs) override; bool isCompatibleFile(const object::ObjectFile &Obj) const override; void registerEHFrames() override; - void deregisterEHFrames() override; Error finalizeLoad(const ObjectFile &Obj, ObjSectionToIDMap &SectionMap) override; }; diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h index 279d0de..95b04fd 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h @@ -28,8 +28,8 @@ #include "llvm/Support/Mutex.h" #include "llvm/Support/SwapByteOrder.h" #include <map> -#include <unordered_map> #include <system_error> +#include <unordered_map> using namespace llvm; using namespace llvm::object; @@ -213,7 +213,7 @@ public: } }; -/// @brief Symbol info for RuntimeDyld. +/// @brief Symbol info for RuntimeDyld. class SymbolTableEntry { public: SymbolTableEntry() @@ -417,7 +417,7 @@ protected: StubMap &Stubs) = 0; /// \brief Resolve relocations to external symbols. - void resolveExternalSymbols(); + Error resolveExternalSymbols(); // \brief Compute an upper bound of the memory that is required to load all // sections @@ -426,6 +426,9 @@ protected: uint64_t &RODataSize, uint32_t &RODataAlign, uint64_t &RWDataSize, uint32_t &RWDataAlign); + // \brief Compute GOT size + unsigned computeGOTSize(const ObjectFile &Obj); + // \brief Compute the stub buffer size required for a section unsigned computeSectionStubBufSize(const ObjectFile &Obj, const SectionRef &Section); @@ -433,6 +436,14 @@ protected: // \brief Implementation of the generic part of the loadObject algorithm. Expected<ObjSectionToIDMap> loadObjectImpl(const object::ObjectFile &Obj); + // \brief Return size of Global Offset Table (GOT) entry + virtual size_t getGOTEntrySize() { return 0; } + + // \brief Return true if the relocation R may require allocating a GOT entry. + virtual bool relocationNeedsGot(const RelocationRef &R) const { + return false; + } + // \brief Return true if the relocation R may require allocating a stub. virtual bool relocationNeedsStub(const RelocationRef &R) const { return true; // Conservative answer @@ -504,7 +515,7 @@ public: virtual void registerEHFrames(); - virtual void deregisterEHFrames(); + void deregisterEHFrames(); virtual Error finalizeLoad(const ObjectFile &ObjImg, ObjSectionToIDMap &SectionMap) { diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp index 00541e8..80e9c7a 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldMachO.cpp @@ -27,7 +27,8 @@ using namespace llvm::object; namespace { class LoadedMachOObjectInfo final - : public RuntimeDyld::LoadedObjectInfoHelper<LoadedMachOObjectInfo> { + : public LoadedObjectInfoHelper<LoadedMachOObjectInfo, + RuntimeDyld::LoadedObjectInfo> { public: LoadedMachOObjectInfo(RuntimeDyldImpl &RTDyld, ObjSectionToIDMap ObjSecToIDMap) diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFI386.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFI386.h index 0398413..901f778 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFI386.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFI386.h @@ -14,9 +14,9 @@ #ifndef LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFI386_H #define LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFI386_H -#include "llvm/Object/COFF.h" -#include "llvm/Support/COFF.h" #include "../RuntimeDyldCOFF.h" +#include "llvm/BinaryFormat/COFF.h" +#include "llvm/Object/COFF.h" #define DEBUG_TYPE "dyld" @@ -217,7 +217,6 @@ public: } void registerEHFrames() override {} - void deregisterEHFrames() override {} }; } diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFThumb.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFThumb.h index 8c6af0b..3e4b0c8 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFThumb.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFThumb.h @@ -14,9 +14,9 @@ #ifndef LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFTHUMB_H #define LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFTHUMB_H -#include "llvm/Object/COFF.h" -#include "llvm/Support/COFF.h" #include "../RuntimeDyldCOFF.h" +#include "llvm/BinaryFormat/COFF.h" +#include "llvm/Object/COFF.h" #define DEBUG_TYPE "dyld" @@ -316,7 +316,6 @@ public: } void registerEHFrames() override {} - void deregisterEHFrames() override {} }; } diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFX86_64.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFX86_64.h index 109beb3..7cbb438 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFX86_64.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFX86_64.h @@ -14,9 +14,9 @@ #ifndef LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFF86_64_H #define LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFF86_64_H -#include "llvm/Object/COFF.h" -#include "llvm/Support/COFF.h" #include "../RuntimeDyldCOFF.h" +#include "llvm/BinaryFormat/COFF.h" +#include "llvm/Object/COFF.h" #define DEBUG_TYPE "dyld" @@ -194,9 +194,6 @@ public: } UnregisteredEHFrameSections.clear(); } - void deregisterEHFrames() override { - // Stub - } Error finalizeLoad(const ObjectFile &Obj, ObjSectionToIDMap &SectionMap) override { // Look for and record the EH frame section IDs. diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldELFMips.cpp b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldELFMips.cpp index cae4d69..926996d 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldELFMips.cpp +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldELFMips.cpp @@ -8,7 +8,7 @@ //===----------------------------------------------------------------------===// #include "RuntimeDyldELFMips.h" -#include "llvm/Support/ELF.h" +#include "llvm/BinaryFormat/ELF.h" #define DEBUG_TYPE "dyld" diff --git a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldMachOARM.h b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldMachOARM.h index adca0ee..43461de 100644 --- a/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldMachOARM.h +++ b/contrib/llvm/lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldMachOARM.h @@ -288,7 +288,6 @@ private: HalfDiffKindBits); addRelocationForSection(R, SectionAID); - addRelocationForSection(R, SectionBID); return ++RelI; } diff --git a/contrib/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp b/contrib/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp index 50478ea..8904475 100644 --- a/contrib/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp +++ b/contrib/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp @@ -12,8 +12,8 @@ // //===----------------------------------------------------------------------===// -#include "llvm/Config/config.h" #include "llvm/ExecutionEngine/SectionMemoryManager.h" +#include "llvm/Config/config.h" #include "llvm/Support/MathExtras.h" #include "llvm/Support/Process.h" |
