diff options
| author | dim <dim@FreeBSD.org> | 2017-09-26 19:56:36 +0000 |
|---|---|---|
| committer | Luiz Souza <luiz@netgate.com> | 2018-02-21 15:12:19 -0300 |
| commit | 1dcd2e8d24b295bc73e513acec2ed1514bb66be4 (patch) | |
| tree | 4bd13a34c251e980e1a6b13584ca1f63b0dfe670 /contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp | |
| parent | f45541ca2a56a1ba1202f94c080b04e96c1fa239 (diff) | |
| download | FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.zip FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.tar.gz | |
Merge clang, llvm, lld, lldb, compiler-rt and libc++ 5.0.0 release.
MFC r309126 (by emaste):
Correct lld llvm-tblgen dependency file name
MFC r309169:
Get rid of separate Subversion mergeinfo properties for llvm-dwarfdump
and llvm-lto. The mergeinfo confuses Subversion enormously, and these
directories will just use the mergeinfo for llvm itself.
MFC r312765:
Pull in r276136 from upstream llvm trunk (by Wei Mi):
Use ValueOffsetPair to enhance value reuse during SCEV expansion.
In D12090, the ExprValueMap was added to reuse existing value during
SCEV expansion. However, const folding and sext/zext distribution can
make the reuse still difficult.
A simplified case is: suppose we know S1 expands to V1 in
ExprValueMap, and
S1 = S2 + C_a
S3 = S2 + C_b
where C_a and C_b are different SCEVConstants. Then we'd like to
expand S3 as V1 - C_a + C_b instead of expanding S2 literally. It is
helpful when S2 is a complex SCEV expr and S2 has no entry in
ExprValueMap, which is usually caused by the fact that S3 is
generated from S1 after const folding.
In order to do that, we represent ExprValueMap as a mapping from SCEV
to ValueOffsetPair. We will save both S1->{V1, 0} and S2->{V1, C_a}
into the ExprValueMap when we create SCEV for V1. When S3 is
expanded, it will first expand S2 to V1 - C_a because of S2->{V1,
C_a} in the map, then expand S3 to V1 - C_a + C_b.
Differential Revision: https://reviews.llvm.org/D21313
This should fix assertion failures when building OpenCV >= 3.1.
PR: 215649
MFC r312831:
Revert r312765 for now, since it causes assertions when building
lang/spidermonkey24.
Reported by: antoine
PR: 215649
MFC r316511 (by jhb):
Add an implementation of __ffssi2() derived from __ffsdi2().
Newer versions of GCC include an __ffssi2() symbol in libgcc and the
compiler can emit calls to it in generated code. This is true for at
least GCC 6.2 when compiling world for mips and mips64.
Reviewed by: jmallett, dim
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D10086
MFC r318601 (by adrian):
[libcompiler-rt] add bswapdi2/bswapsi2
This is required for mips gcc 6.3 userland to build/run.
Reviewed by: emaste, dim
Approved by: emaste
Differential Revision: https://reviews.freebsd.org/D10838
MFC r318884 (by emaste):
lldb: map TRAP_CAP to a trace trap
In the absense of a more specific handler for TRAP_CAP (generated by
ENOTCAPABLE or ECAPMODE while in capability mode) treat it as a trace
trap.
Example usage (testing the bug in PR219173):
% proccontrol -m trapcap lldb usr.bin/hexdump/obj/hexdump -- -Cv -s 1 /bin/ls
...
(lldb) run
Process 12980 launching
Process 12980 launched: '.../usr.bin/hexdump/obj/hexdump' (x86_64)
Process 12980 stopped
* thread #1, stop reason = trace
frame #0: 0x0000004b80c65f1a libc.so.7`__sys_lseek + 10
...
In the future we should have LLDB control the trapcap procctl itself
(as it does with ASLR), as well as report a specific stop reason.
This change eliminates an assertion failure from LLDB for now.
MFC r319796:
Remove a few unneeded files from libllvm, libclang and liblldb.
MFC r319885 (by emaste):
lld: ELF: Fix ICF crash on absolute symbol relocations.
If two sections contained relocations to absolute symbols with the same
value we would crash when trying to access their sections. Add a check that
both symbols point to sections before accessing their sections, and treat
absolute symbols as equal if their values are equal.
Obtained from: LLD commit r292578
MFC r319918:
Revert r319796 for now, it can cause undefined references when linking
in some circumstances.
Reported by: Shawn Webb <shawn.webb@hardenedbsd.org>
MFC r319957 (by emaste):
lld: Add armelf emulation mode
Obtained from: LLD r305375
MFC r321369:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 (trunk r308421). Upstream has branched for the 5.0.0 release,
which should be in about a month. Please report bugs and regressions,
so we can get them into the release.
Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.
MFC r321420:
Add a few more object files to liblldb, which should solve errors when
linking the lldb executable in some cases. In particular, when the
-ffunction-sections -fdata-sections options are turned off, or
ineffective.
Reported by: Shawn Webb, Mark Millard
MFC r321433:
Cleanup stale Options.inc files from the previous libllvm build for
clang 4.0.0. Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.
Reported by: Mark Millard
MFC r321439 (by bdrewery):
Move llvm Options.inc hack from r321433 for NO_CLEAN to lib/clang/libllvm.
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation. So using
a beforebuild target here removes the file before the compilation
tries to include it.
MFC r321664:
Pull in r308891 from upstream llvm trunk (by Benjamin Kramer):
[CodeGenPrepare] Cut off FindAllMemoryUses if there are too many uses.
This avoids excessive compile time. The case I'm looking at is
Function.cpp from an old version of LLVM that still had the giant
memcmp string matcher in it. Before r308322 this compiled in about 2
minutes, after it, clang takes infinite* time to compile it. With
this patch we're at 5 min, which is still bad but this is a
pathological case.
The cut off at 20 uses was chosen by looking at other cut-offs in LLVM
for user scanning. It's probably too high, but does the job and is
very unlikely to regress anything.
Fixes PR33900.
* I'm impatient and aborted after 15 minutes, on the bug report it was
killed after 2h.
Pull in r308986 from upstream llvm trunk (by Simon Pilgrim):
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914)
D35067/rL308322 attempted to support up to 4 load pairs for memcmp
inlining which resulted in regressions for some optimized libc memcmp
implementations (PR33914).
Until we can match these more optimal cases, this patch reduces the
memcmp expansion to a maximum of 2 load pairs (which matches what we
do for -Os).
This patch should be considered for the 5.0.0 release branch as well
Differential Revision: https://reviews.llvm.org/D35830
These fix a hang (or extremely long compile time) when building older
LLVM ports.
Reported by: antoine
PR: 219139
MFC r321719:
Pull in r309503 from upstream clang trunk (by Richard Smith):
PR33902: Invalidate line number cache when adding more text to
existing buffer.
This led to crashes as the line number cache would report a bogus
line number for a line of code, and we'd try to find a nonexistent
column within the line when printing diagnostics.
This fixes an assertion when building the graphics/champlain port.
Reported by: antoine, kwm
PR: 219139
MFC r321723:
Upgrade our copies of clang, llvm, lld and lldb to r309439 from the
upstream release_50 branch. This is just after upstream's 5.0.0-rc1.
MFC r322320:
Upgrade our copies of clang, llvm and libc++ to r310316 from the
upstream release_50 branch.
MFC r322326 (by emaste):
lldb: Make i386-*-freebsd expression work on JIT path
* Enable i386 ABI creation for freebsd
* Added an extra argument in ABISysV_i386::PrepareTrivialCall for mmap
syscall
* Unlike linux, the last argument of mmap is actually 64-bit(off_t).
This requires us to push an additional word for the higher order bits.
* Prior to this change, ktrace dump will show mmap failures due to
invalid argument coming from the 6th mmap argument.
Submitted by: Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D34776
MFC r322360 (by emaste):
lldb: Report inferior signals as signals, not exceptions, on FreeBSD
This is the FreeBSD equivalent of LLVM r238549.
This serves 2 purposes:
* LLDB should handle inferior process signals SIGSEGV/SIGILL/SIGBUS/
SIGFPE the way it is suppose to be handled. Prior to this fix these
signals will neither create a coredump, nor exit from the debugger
or work for signal handling scenario.
* eInvalidCrashReason need not report "unknown crash reason" if we have
a valid si_signo
llvm.org/pr23699
Patch by Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D35223
Submitted by: Karnajit Wangkhem
Obtained from: LLVM r310591
MFC r322474 (by emaste):
lld: Add `-z muldefs` option.
Obtained from: LLVM r310757
MFC r322740:
Upgrade our copies of clang, llvm, lld and libc++ to r311219 from the
upstream release_50 branch.
MFC r322855:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r311606 from
the upstream release_50 branch.
As of this version, lib/msun's trig test should also work correctly
again (see bug 220989 for more information).
PR: 220989
MFC r323112:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r312293 from
the upstream release_50 branch. This corresponds to 5.0.0 rc4.
As of this version, the cad/stepcode port should now compile in a more
reasonable time on i386 (see bug 221836 for more information).
PR: 221836
MFC r323245:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 release (upstream r312559).
Release notes for llvm, clang and lld will be available here soon:
<http://releases.llvm.org/5.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/lld/docs/ReleaseNotes.html>
Relnotes: yes
(cherry picked from commit 12cd91cf4c6b96a24427c0de5374916f2808d263)
Diffstat (limited to 'contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp')
| -rw-r--r-- | contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp | 765 |
1 files changed, 503 insertions, 262 deletions
diff --git a/contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp b/contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp index e5e34a5..9572bd3 100644 --- a/contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp +++ b/contrib/llvm/tools/clang/lib/CodeGen/CGExpr.cpp @@ -61,17 +61,39 @@ llvm::Value *CodeGenFunction::EmitCastToVoidPtr(llvm::Value *value) { /// CreateTempAlloca - This creates a alloca and inserts it into the entry /// block. Address CodeGenFunction::CreateTempAlloca(llvm::Type *Ty, CharUnits Align, - const Twine &Name) { - auto Alloca = CreateTempAlloca(Ty, Name); + const Twine &Name, + llvm::Value *ArraySize, + bool CastToDefaultAddrSpace) { + auto Alloca = CreateTempAlloca(Ty, Name, ArraySize); Alloca->setAlignment(Align.getQuantity()); - return Address(Alloca, Align); -} - -/// CreateTempAlloca - This creates a alloca and inserts it into the entry -/// block. + llvm::Value *V = Alloca; + // Alloca always returns a pointer in alloca address space, which may + // be different from the type defined by the language. For example, + // in C++ the auto variables are in the default address space. Therefore + // cast alloca to the default address space when necessary. + if (CastToDefaultAddrSpace && getASTAllocaAddressSpace() != LangAS::Default) { + auto DestAddrSpace = getContext().getTargetAddressSpace(LangAS::Default); + auto CurIP = Builder.saveIP(); + Builder.SetInsertPoint(AllocaInsertPt); + V = getTargetHooks().performAddrSpaceCast( + *this, V, getASTAllocaAddressSpace(), LangAS::Default, + Ty->getPointerTo(DestAddrSpace), /*non-null*/ true); + Builder.restoreIP(CurIP); + } + + return Address(V, Align); +} + +/// CreateTempAlloca - This creates an alloca and inserts it into the entry +/// block if \p ArraySize is nullptr, otherwise inserts it at the current +/// insertion point of the builder. llvm::AllocaInst *CodeGenFunction::CreateTempAlloca(llvm::Type *Ty, - const Twine &Name) { - return new llvm::AllocaInst(Ty, nullptr, Name, AllocaInsertPt); + const Twine &Name, + llvm::Value *ArraySize) { + if (ArraySize) + return Builder.CreateAlloca(Ty, ArraySize, Name); + return new llvm::AllocaInst(Ty, CGM.getDataLayout().getAllocaAddrSpace(), + ArraySize, Name, AllocaInsertPt); } /// CreateDefaultAlignTempAlloca - This creates an alloca with the @@ -98,14 +120,18 @@ Address CodeGenFunction::CreateIRTemp(QualType Ty, const Twine &Name) { return CreateTempAlloca(ConvertType(Ty), Align, Name); } -Address CodeGenFunction::CreateMemTemp(QualType Ty, const Twine &Name) { +Address CodeGenFunction::CreateMemTemp(QualType Ty, const Twine &Name, + bool CastToDefaultAddrSpace) { // FIXME: Should we prefer the preferred type alignment here? - return CreateMemTemp(Ty, getContext().getTypeAlignInChars(Ty), Name); + return CreateMemTemp(Ty, getContext().getTypeAlignInChars(Ty), Name, + CastToDefaultAddrSpace); } Address CodeGenFunction::CreateMemTemp(QualType Ty, CharUnits Align, - const Twine &Name) { - return CreateTempAlloca(ConvertTypeForMem(Ty), Align, Name); + const Twine &Name, + bool CastToDefaultAddrSpace) { + return CreateTempAlloca(ConvertTypeForMem(Ty), Align, Name, nullptr, + CastToDefaultAddrSpace); } /// EvaluateExprAsBool - Perform the usual unary conversions on the specified @@ -315,9 +341,10 @@ pushTemporaryCleanup(CodeGenFunction &CGF, const MaterializeTemporaryExpr *M, } } -static Address -createReferenceTemporary(CodeGenFunction &CGF, - const MaterializeTemporaryExpr *M, const Expr *Inner) { +static Address createReferenceTemporary(CodeGenFunction &CGF, + const MaterializeTemporaryExpr *M, + const Expr *Inner) { + auto &TCG = CGF.getTargetHooks(); switch (M->getStorageDuration()) { case SD_FullExpression: case SD_Automatic: { @@ -330,13 +357,24 @@ createReferenceTemporary(CodeGenFunction &CGF, (Ty->isArrayType() || Ty->isRecordType()) && CGF.CGM.isTypeConstant(Ty, true)) if (llvm::Constant *Init = CGF.CGM.EmitConstantExpr(Inner, Ty, &CGF)) { - auto *GV = new llvm::GlobalVariable( - CGF.CGM.getModule(), Init->getType(), /*isConstant=*/true, - llvm::GlobalValue::PrivateLinkage, Init, ".ref.tmp"); - CharUnits alignment = CGF.getContext().getTypeAlignInChars(Ty); - GV->setAlignment(alignment.getQuantity()); - // FIXME: Should we put the new global into a COMDAT? - return Address(GV, alignment); + if (auto AddrSpace = CGF.getTarget().getConstantAddressSpace()) { + auto AS = AddrSpace.getValue(); + auto *GV = new llvm::GlobalVariable( + CGF.CGM.getModule(), Init->getType(), /*isConstant=*/true, + llvm::GlobalValue::PrivateLinkage, Init, ".ref.tmp", nullptr, + llvm::GlobalValue::NotThreadLocal, + CGF.getContext().getTargetAddressSpace(AS)); + CharUnits alignment = CGF.getContext().getTypeAlignInChars(Ty); + GV->setAlignment(alignment.getQuantity()); + llvm::Constant *C = GV; + if (AS != LangAS::Default) + C = TCG.performAddrSpaceCast( + CGF.CGM, GV, AS, LangAS::Default, + GV->getValueType()->getPointerTo( + CGF.getContext().getTargetAddressSpace(LangAS::Default))); + // FIXME: Should we put the new global into a COMDAT? + return Address(C, alignment); + } } return CGF.CreateMemTemp(Ty, "ref.tmp"); } @@ -373,12 +411,14 @@ EmitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *M) { // dynamic initialization or a cleanup and we can just return the address // of the temporary. if (Var->hasInitializer()) - return MakeAddrLValue(Object, M->getType(), AlignmentSource::Decl); + return MakeAddrLValue(Object, M->getType(), + LValueBaseInfo(AlignmentSource::Decl, false)); Var->setInitializer(CGM.EmitNullConstant(E->getType())); } LValue RefTempDst = MakeAddrLValue(Object, M->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, + false)); switch (getEvaluationKind(E->getType())) { default: llvm_unreachable("expected scalar or aggregate expression"); @@ -415,9 +455,11 @@ EmitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *M) { // Create and initialize the reference temporary. Address Object = createReferenceTemporary(*this, M, E); - if (auto *Var = dyn_cast<llvm::GlobalVariable>(Object.getPointer())) { + if (auto *Var = dyn_cast<llvm::GlobalVariable>( + Object.getPointer()->stripPointerCasts())) { Object = Address(llvm::ConstantExpr::getBitCast( - Var, ConvertTypeForMem(E->getType())->getPointerTo()), + cast<llvm::Constant>(Object.getPointer()), + ConvertTypeForMem(E->getType())->getPointerTo()), Object.getAlignment()); // If the temporary is a global and has a constant initializer or is a // constant temporary that we promoted to a global, we may have already @@ -464,7 +506,7 @@ EmitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *M) { case SubobjectAdjustment::FieldAdjustment: { LValue LV = MakeAddrLValue(Object, E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); LV = EmitLValueForField(LV, Adjustment.Field); assert(LV.isSimple() && "materialized temporary field is not a simple lvalue"); @@ -481,7 +523,8 @@ EmitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *M) { } } - return MakeAddrLValue(Object, M->getType(), AlignmentSource::Decl); + return MakeAddrLValue(Object, M->getType(), + LValueBaseInfo(AlignmentSource::Decl, false)); } RValue @@ -534,7 +577,8 @@ bool CodeGenFunction::sanitizePerformTypeCheck() const { void CodeGenFunction::EmitTypeCheck(TypeCheckKind TCK, SourceLocation Loc, llvm::Value *Ptr, QualType Ty, - CharUnits Alignment, bool SkipNullCheck) { + CharUnits Alignment, + SanitizerSet SkippedChecks) { if (!sanitizePerformTypeCheck()) return; @@ -544,31 +588,52 @@ void CodeGenFunction::EmitTypeCheck(TypeCheckKind TCK, SourceLocation Loc, if (Ptr->getType()->getPointerAddressSpace()) return; + // Don't check pointers to volatile data. The behavior here is implementation- + // defined. + if (Ty.isVolatileQualified()) + return; + SanitizerScope SanScope(this); SmallVector<std::pair<llvm::Value *, SanitizerMask>, 3> Checks; llvm::BasicBlock *Done = nullptr; + // Quickly determine whether we have a pointer to an alloca. It's possible + // to skip null checks, and some alignment checks, for these pointers. This + // can reduce compile-time significantly. + auto PtrToAlloca = + dyn_cast<llvm::AllocaInst>(Ptr->stripPointerCastsNoFollowAliases()); + bool AllowNullPointers = TCK == TCK_DowncastPointer || TCK == TCK_Upcast || TCK == TCK_UpcastToVirtualBase; if ((SanOpts.has(SanitizerKind::Null) || AllowNullPointers) && - !SkipNullCheck) { + !SkippedChecks.has(SanitizerKind::Null) && !PtrToAlloca) { // The glvalue must not be an empty glvalue. llvm::Value *IsNonNull = Builder.CreateIsNotNull(Ptr); - if (AllowNullPointers) { - // When performing pointer casts, it's OK if the value is null. - // Skip the remaining checks in that case. - Done = createBasicBlock("null"); - llvm::BasicBlock *Rest = createBasicBlock("not.null"); - Builder.CreateCondBr(IsNonNull, Rest, Done); - EmitBlock(Rest); - } else { - Checks.push_back(std::make_pair(IsNonNull, SanitizerKind::Null)); + // The IR builder can constant-fold the null check if the pointer points to + // a constant. + bool PtrIsNonNull = + IsNonNull == llvm::ConstantInt::getTrue(getLLVMContext()); + + // Skip the null check if the pointer is known to be non-null. + if (!PtrIsNonNull) { + if (AllowNullPointers) { + // When performing pointer casts, it's OK if the value is null. + // Skip the remaining checks in that case. + Done = createBasicBlock("null"); + llvm::BasicBlock *Rest = createBasicBlock("not.null"); + Builder.CreateCondBr(IsNonNull, Rest, Done); + EmitBlock(Rest); + } else { + Checks.push_back(std::make_pair(IsNonNull, SanitizerKind::Null)); + } } } - if (SanOpts.has(SanitizerKind::ObjectSize) && !Ty->isIncompleteType()) { + if (SanOpts.has(SanitizerKind::ObjectSize) && + !SkippedChecks.has(SanitizerKind::ObjectSize) && + !Ty->isIncompleteType()) { uint64_t Size = getContext().getTypeSizeInChars(Ty).getQuantity(); // The glvalue must refer to a large enough storage region. @@ -578,22 +643,25 @@ void CodeGenFunction::EmitTypeCheck(TypeCheckKind TCK, SourceLocation Loc, llvm::Type *Tys[2] = { IntPtrTy, Int8PtrTy }; llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::objectsize, Tys); llvm::Value *Min = Builder.getFalse(); + llvm::Value *NullIsUnknown = Builder.getFalse(); llvm::Value *CastAddr = Builder.CreateBitCast(Ptr, Int8PtrTy); - llvm::Value *LargeEnough = - Builder.CreateICmpUGE(Builder.CreateCall(F, {CastAddr, Min}), - llvm::ConstantInt::get(IntPtrTy, Size)); + llvm::Value *LargeEnough = Builder.CreateICmpUGE( + Builder.CreateCall(F, {CastAddr, Min, NullIsUnknown}), + llvm::ConstantInt::get(IntPtrTy, Size)); Checks.push_back(std::make_pair(LargeEnough, SanitizerKind::ObjectSize)); } uint64_t AlignVal = 0; - if (SanOpts.has(SanitizerKind::Alignment)) { + if (SanOpts.has(SanitizerKind::Alignment) && + !SkippedChecks.has(SanitizerKind::Alignment)) { AlignVal = Alignment.getQuantity(); if (!Ty->isIncompleteType() && !AlignVal) AlignVal = getContext().getTypeAlignInChars(Ty).getQuantity(); // The glvalue must be suitably aligned. - if (AlignVal) { + if (AlignVal > 1 && + (!PtrToAlloca || PtrToAlloca->getAlignment() < AlignVal)) { llvm::Value *Align = Builder.CreateAnd(Builder.CreatePtrToInt(Ptr, IntPtrTy), llvm::ConstantInt::get(IntPtrTy, AlignVal - 1)); @@ -624,6 +692,7 @@ void CodeGenFunction::EmitTypeCheck(TypeCheckKind TCK, SourceLocation Loc, // or call a non-static member function CXXRecordDecl *RD = Ty->getAsCXXRecordDecl(); if (SanOpts.has(SanitizerKind::Vptr) && + !SkippedChecks.has(SanitizerKind::Vptr) && (TCK == TCK_MemberAccess || TCK == TCK_MemberCall || TCK == TCK_DowncastPointer || TCK == TCK_DowncastReference || TCK == TCK_UpcastToVirtualBase) && @@ -825,7 +894,7 @@ void CodeGenModule::EmitExplicitCastExprType(const ExplicitCastExpr *E, /// EmitPointerWithAlignment - Given an expression of pointer type, try to /// derive a more accurate bound on the alignment of the pointer. Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E, - AlignmentSource *Source) { + LValueBaseInfo *BaseInfo) { // We allow this with ObjC object pointers because of fragile ABIs. assert(E->getType()->isPointerType() || E->getType()->isObjCObjectPointerType()); @@ -844,16 +913,20 @@ Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E, if (PtrTy->getPointeeType()->isVoidType()) break; - AlignmentSource InnerSource; - Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), &InnerSource); - if (Source) *Source = InnerSource; + LValueBaseInfo InnerInfo; + Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), &InnerInfo); + if (BaseInfo) *BaseInfo = InnerInfo; // If this is an explicit bitcast, and the source l-value is // opaque, honor the alignment of the casted-to type. if (isa<ExplicitCastExpr>(CE) && - InnerSource != AlignmentSource::Decl) { - Addr = Address(Addr.getPointer(), - getNaturalPointeeTypeAlignment(E->getType(), Source)); + InnerInfo.getAlignmentSource() != AlignmentSource::Decl) { + LValueBaseInfo ExpInfo; + CharUnits Align = getNaturalPointeeTypeAlignment(E->getType(), + &ExpInfo); + if (BaseInfo) + BaseInfo->mergeForCast(ExpInfo); + Addr = Address(Addr.getPointer(), Align); } if (SanOpts.has(SanitizerKind::CFIUnrelatedCast) && @@ -871,12 +944,12 @@ Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E, // Array-to-pointer decay. case CK_ArrayToPointerDecay: - return EmitArrayToPointerDecay(CE->getSubExpr(), Source); + return EmitArrayToPointerDecay(CE->getSubExpr(), BaseInfo); // Derived-to-base conversions. case CK_UncheckedDerivedToBase: case CK_DerivedToBase: { - Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), Source); + Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), BaseInfo); auto Derived = CE->getSubExpr()->getType()->getPointeeCXXRecordDecl(); return GetAddressOfBaseClass(Addr, Derived, CE->path_begin(), CE->path_end(), @@ -895,7 +968,7 @@ Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E, if (const UnaryOperator *UO = dyn_cast<UnaryOperator>(E)) { if (UO->getOpcode() == UO_AddrOf) { LValue LV = EmitLValue(UO->getSubExpr()); - if (Source) *Source = LV.getAlignmentSource(); + if (BaseInfo) *BaseInfo = LV.getBaseInfo(); return LV.getAddress(); } } @@ -903,7 +976,7 @@ Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E, // TODO: conditional operators, comma. // Otherwise, use the alignment of the type. - CharUnits Align = getNaturalPointeeTypeAlignment(E->getType(), Source); + CharUnits Align = getNaturalPointeeTypeAlignment(E->getType(), BaseInfo); return Address(EmitScalarExpr(E), Align); } @@ -947,15 +1020,47 @@ LValue CodeGenFunction::EmitUnsupportedLValue(const Expr *E, E->getType()); } +bool CodeGenFunction::IsWrappedCXXThis(const Expr *Obj) { + const Expr *Base = Obj; + while (!isa<CXXThisExpr>(Base)) { + // The result of a dynamic_cast can be null. + if (isa<CXXDynamicCastExpr>(Base)) + return false; + + if (const auto *CE = dyn_cast<CastExpr>(Base)) { + Base = CE->getSubExpr(); + } else if (const auto *PE = dyn_cast<ParenExpr>(Base)) { + Base = PE->getSubExpr(); + } else if (const auto *UO = dyn_cast<UnaryOperator>(Base)) { + if (UO->getOpcode() == UO_Extension) + Base = UO->getSubExpr(); + else + return false; + } else { + return false; + } + } + return true; +} + LValue CodeGenFunction::EmitCheckedLValue(const Expr *E, TypeCheckKind TCK) { LValue LV; if (SanOpts.has(SanitizerKind::ArrayBounds) && isa<ArraySubscriptExpr>(E)) LV = EmitArraySubscriptExpr(cast<ArraySubscriptExpr>(E), /*Accessed*/true); else LV = EmitLValue(E); - if (!isa<DeclRefExpr>(E) && !LV.isBitField() && LV.isSimple()) + if (!isa<DeclRefExpr>(E) && !LV.isBitField() && LV.isSimple()) { + SanitizerSet SkippedChecks; + if (const auto *ME = dyn_cast<MemberExpr>(E)) { + bool IsBaseCXXThis = IsWrappedCXXThis(ME->getBase()); + if (IsBaseCXXThis) + SkippedChecks.set(SanitizerKind::Alignment, true); + if (IsBaseCXXThis || isa<DeclRefExpr>(ME->getBase())) + SkippedChecks.set(SanitizerKind::Null, true); + } EmitTypeCheck(TCK, E->getExprLoc(), LV.getPointer(), - E->getType(), LV.getAlignment()); + E->getType(), LV.getAlignment(), SkippedChecks); + } return LV; } @@ -1033,7 +1138,19 @@ LValue CodeGenFunction::EmitLValue(const Expr *E) { const auto *cleanups = cast<ExprWithCleanups>(E); enterFullExpression(cleanups); RunCleanupsScope Scope(*this); - return EmitLValue(cleanups->getSubExpr()); + LValue LV = EmitLValue(cleanups->getSubExpr()); + if (LV.isSimple()) { + // Defend against branches out of gnu statement expressions surrounded by + // cleanups. + llvm::Value *V = LV.getPointer(); + Scope.ForceCleanup({&V}); + return LValue::MakeAddr(Address(V, LV.getAlignment()), LV.getType(), + getContext(), LV.getBaseInfo(), + LV.getTBAAInfo()); + } + // FIXME: Is it possible to create an ExprWithCleanups that produces a + // bitfield lvalue or some other non-simple lvalue? + return LV; } case Expr::CXXDefaultArgExprClass: @@ -1085,6 +1202,11 @@ LValue CodeGenFunction::EmitLValue(const Expr *E) { case Expr::MaterializeTemporaryExprClass: return EmitMaterializeTemporaryExpr(cast<MaterializeTemporaryExpr>(E)); + + case Expr::CoawaitExprClass: + return EmitCoawaitLValue(cast<CoawaitExpr>(E)); + case Expr::CoyieldExprClass: + return EmitCoyieldLValue(cast<CoyieldExpr>(E)); } } @@ -1203,7 +1325,7 @@ CodeGenFunction::tryEmitAsConstant(DeclRefExpr *refExpr) { llvm::Value *CodeGenFunction::EmitLoadOfScalar(LValue lvalue, SourceLocation Loc) { return EmitLoadOfScalar(lvalue.getAddress(), lvalue.isVolatile(), - lvalue.getType(), Loc, lvalue.getAlignmentSource(), + lvalue.getType(), Loc, lvalue.getBaseInfo(), lvalue.getTBAAInfo(), lvalue.getTBAABaseType(), lvalue.getTBAAOffset(), lvalue.isNontemporal()); @@ -1265,40 +1387,89 @@ llvm::MDNode *CodeGenFunction::getRangeForLoadFromType(QualType Ty) { return MDHelper.createRange(Min, End); } +bool CodeGenFunction::EmitScalarRangeCheck(llvm::Value *Value, QualType Ty, + SourceLocation Loc) { + bool HasBoolCheck = SanOpts.has(SanitizerKind::Bool); + bool HasEnumCheck = SanOpts.has(SanitizerKind::Enum); + if (!HasBoolCheck && !HasEnumCheck) + return false; + + bool IsBool = hasBooleanRepresentation(Ty) || + NSAPI(CGM.getContext()).isObjCBOOLType(Ty); + bool NeedsBoolCheck = HasBoolCheck && IsBool; + bool NeedsEnumCheck = HasEnumCheck && Ty->getAs<EnumType>(); + if (!NeedsBoolCheck && !NeedsEnumCheck) + return false; + + // Single-bit booleans don't need to be checked. Special-case this to avoid + // a bit width mismatch when handling bitfield values. This is handled by + // EmitFromMemory for the non-bitfield case. + if (IsBool && + cast<llvm::IntegerType>(Value->getType())->getBitWidth() == 1) + return false; + + llvm::APInt Min, End; + if (!getRangeForType(*this, Ty, Min, End, /*StrictEnums=*/true, IsBool)) + return true; + + SanitizerScope SanScope(this); + llvm::Value *Check; + --End; + if (!Min) { + Check = Builder.CreateICmpULE( + Value, llvm::ConstantInt::get(getLLVMContext(), End)); + } else { + llvm::Value *Upper = Builder.CreateICmpSLE( + Value, llvm::ConstantInt::get(getLLVMContext(), End)); + llvm::Value *Lower = Builder.CreateICmpSGE( + Value, llvm::ConstantInt::get(getLLVMContext(), Min)); + Check = Builder.CreateAnd(Upper, Lower); + } + llvm::Constant *StaticArgs[] = {EmitCheckSourceLocation(Loc), + EmitCheckTypeDescriptor(Ty)}; + SanitizerMask Kind = + NeedsEnumCheck ? SanitizerKind::Enum : SanitizerKind::Bool; + EmitCheck(std::make_pair(Check, Kind), SanitizerHandler::LoadInvalidValue, + StaticArgs, EmitCheckValue(Value)); + return true; +} + llvm::Value *CodeGenFunction::EmitLoadOfScalar(Address Addr, bool Volatile, QualType Ty, SourceLocation Loc, - AlignmentSource AlignSource, + LValueBaseInfo BaseInfo, llvm::MDNode *TBAAInfo, QualType TBAABaseType, uint64_t TBAAOffset, bool isNontemporal) { - // For better performance, handle vector loads differently. - if (Ty->isVectorType()) { - const llvm::Type *EltTy = Addr.getElementType(); - - const auto *VTy = cast<llvm::VectorType>(EltTy); - - // Handle vectors of size 3 like size 4 for better performance. - if (VTy->getNumElements() == 3) { - - // Bitcast to vec4 type. - llvm::VectorType *vec4Ty = llvm::VectorType::get(VTy->getElementType(), - 4); - Address Cast = Builder.CreateElementBitCast(Addr, vec4Ty, "castToVec4"); - // Now load value. - llvm::Value *V = Builder.CreateLoad(Cast, Volatile, "loadVec4"); - - // Shuffle vector to get vec3. - V = Builder.CreateShuffleVector(V, llvm::UndefValue::get(vec4Ty), - {0, 1, 2}, "extractVec"); - return EmitFromMemory(V, Ty); + if (!CGM.getCodeGenOpts().PreserveVec3Type) { + // For better performance, handle vector loads differently. + if (Ty->isVectorType()) { + const llvm::Type *EltTy = Addr.getElementType(); + + const auto *VTy = cast<llvm::VectorType>(EltTy); + + // Handle vectors of size 3 like size 4 for better performance. + if (VTy->getNumElements() == 3) { + + // Bitcast to vec4 type. + llvm::VectorType *vec4Ty = + llvm::VectorType::get(VTy->getElementType(), 4); + Address Cast = Builder.CreateElementBitCast(Addr, vec4Ty, "castToVec4"); + // Now load value. + llvm::Value *V = Builder.CreateLoad(Cast, Volatile, "loadVec4"); + + // Shuffle vector to get vec3. + V = Builder.CreateShuffleVector(V, llvm::UndefValue::get(vec4Ty), + {0, 1, 2}, "extractVec"); + return EmitFromMemory(V, Ty); + } } } // Atomic operations have to be done on integral types. LValue AtomicLValue = - LValue::MakeAddr(Addr, Ty, getContext(), AlignSource, TBAAInfo); + LValue::MakeAddr(Addr, Ty, getContext(), BaseInfo, TBAAInfo); if (Ty->isAtomicType() || LValueIsSuitableForInlineAtomic(AtomicLValue)) { return EmitAtomicLoad(AtomicLValue, Loc).getScalarVal(); } @@ -1310,42 +1481,17 @@ llvm::Value *CodeGenFunction::EmitLoadOfScalar(Address Addr, bool Volatile, Load->setMetadata(CGM.getModule().getMDKindID("nontemporal"), Node); } if (TBAAInfo) { - llvm::MDNode *TBAAPath = CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo, - TBAAOffset); - if (TBAAPath) - CGM.DecorateInstructionWithTBAA(Load, TBAAPath, - false /*ConvertTypeToTag*/); + bool MayAlias = BaseInfo.getMayAlias(); + llvm::MDNode *TBAA = MayAlias + ? CGM.getTBAAInfo(getContext().CharTy) + : CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo, TBAAOffset); + if (TBAA) + CGM.DecorateInstructionWithTBAA(Load, TBAA, MayAlias); } - bool IsBool = hasBooleanRepresentation(Ty) || - NSAPI(CGM.getContext()).isObjCBOOLType(Ty); - bool NeedsBoolCheck = SanOpts.has(SanitizerKind::Bool) && IsBool; - bool NeedsEnumCheck = - SanOpts.has(SanitizerKind::Enum) && Ty->getAs<EnumType>(); - if (NeedsBoolCheck || NeedsEnumCheck) { - SanitizerScope SanScope(this); - llvm::APInt Min, End; - if (getRangeForType(*this, Ty, Min, End, /*StrictEnums=*/true, IsBool)) { - --End; - llvm::Value *Check; - if (!Min) - Check = Builder.CreateICmpULE( - Load, llvm::ConstantInt::get(getLLVMContext(), End)); - else { - llvm::Value *Upper = Builder.CreateICmpSLE( - Load, llvm::ConstantInt::get(getLLVMContext(), End)); - llvm::Value *Lower = Builder.CreateICmpSGE( - Load, llvm::ConstantInt::get(getLLVMContext(), Min)); - Check = Builder.CreateAnd(Upper, Lower); - } - llvm::Constant *StaticArgs[] = { - EmitCheckSourceLocation(Loc), - EmitCheckTypeDescriptor(Ty) - }; - SanitizerMask Kind = NeedsEnumCheck ? SanitizerKind::Enum : SanitizerKind::Bool; - EmitCheck(std::make_pair(Check, Kind), SanitizerHandler::LoadInvalidValue, - StaticArgs, EmitCheckValue(Load)); - } + if (EmitScalarRangeCheck(Load, Ty, Loc)) { + // In order to prevent the optimizer from throwing away the check, don't + // attach range metadata to the load. } else if (CGM.getCodeGenOpts().OptimizationLevel > 0) if (llvm::MDNode *RangeInfo = getRangeForLoadFromType(Ty)) Load->setMetadata(llvm::LLVMContext::MD_range, RangeInfo); @@ -1380,37 +1526,38 @@ llvm::Value *CodeGenFunction::EmitFromMemory(llvm::Value *Value, QualType Ty) { void CodeGenFunction::EmitStoreOfScalar(llvm::Value *Value, Address Addr, bool Volatile, QualType Ty, - AlignmentSource AlignSource, + LValueBaseInfo BaseInfo, llvm::MDNode *TBAAInfo, bool isInit, QualType TBAABaseType, uint64_t TBAAOffset, bool isNontemporal) { - // Handle vectors differently to get better performance. - if (Ty->isVectorType()) { - llvm::Type *SrcTy = Value->getType(); - auto *VecTy = cast<llvm::VectorType>(SrcTy); - // Handle vec3 special. - if (VecTy->getNumElements() == 3) { - // Our source is a vec3, do a shuffle vector to make it a vec4. - llvm::Constant *Mask[] = {Builder.getInt32(0), Builder.getInt32(1), - Builder.getInt32(2), - llvm::UndefValue::get(Builder.getInt32Ty())}; - llvm::Value *MaskV = llvm::ConstantVector::get(Mask); - Value = Builder.CreateShuffleVector(Value, - llvm::UndefValue::get(VecTy), - MaskV, "extractVec"); - SrcTy = llvm::VectorType::get(VecTy->getElementType(), 4); - } - if (Addr.getElementType() != SrcTy) { - Addr = Builder.CreateElementBitCast(Addr, SrcTy, "storetmp"); + if (!CGM.getCodeGenOpts().PreserveVec3Type) { + // Handle vectors differently to get better performance. + if (Ty->isVectorType()) { + llvm::Type *SrcTy = Value->getType(); + auto *VecTy = dyn_cast<llvm::VectorType>(SrcTy); + // Handle vec3 special. + if (VecTy && VecTy->getNumElements() == 3) { + // Our source is a vec3, do a shuffle vector to make it a vec4. + llvm::Constant *Mask[] = {Builder.getInt32(0), Builder.getInt32(1), + Builder.getInt32(2), + llvm::UndefValue::get(Builder.getInt32Ty())}; + llvm::Value *MaskV = llvm::ConstantVector::get(Mask); + Value = Builder.CreateShuffleVector(Value, llvm::UndefValue::get(VecTy), + MaskV, "extractVec"); + SrcTy = llvm::VectorType::get(VecTy->getElementType(), 4); + } + if (Addr.getElementType() != SrcTy) { + Addr = Builder.CreateElementBitCast(Addr, SrcTy, "storetmp"); + } } } Value = EmitToMemory(Value, Ty); LValue AtomicLValue = - LValue::MakeAddr(Addr, Ty, getContext(), AlignSource, TBAAInfo); + LValue::MakeAddr(Addr, Ty, getContext(), BaseInfo, TBAAInfo); if (Ty->isAtomicType() || (!isInit && LValueIsSuitableForInlineAtomic(AtomicLValue))) { EmitAtomicStore(RValue::get(Value), AtomicLValue, isInit); @@ -1425,18 +1572,19 @@ void CodeGenFunction::EmitStoreOfScalar(llvm::Value *Value, Address Addr, Store->setMetadata(CGM.getModule().getMDKindID("nontemporal"), Node); } if (TBAAInfo) { - llvm::MDNode *TBAAPath = CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo, - TBAAOffset); - if (TBAAPath) - CGM.DecorateInstructionWithTBAA(Store, TBAAPath, - false /*ConvertTypeToTag*/); + bool MayAlias = BaseInfo.getMayAlias(); + llvm::MDNode *TBAA = MayAlias + ? CGM.getTBAAInfo(getContext().CharTy) + : CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo, TBAAOffset); + if (TBAA) + CGM.DecorateInstructionWithTBAA(Store, TBAA, MayAlias); } } void CodeGenFunction::EmitStoreOfScalar(llvm::Value *value, LValue lvalue, bool isInit) { EmitStoreOfScalar(value, lvalue.getAddress(), lvalue.isVolatile(), - lvalue.getType(), lvalue.getAlignmentSource(), + lvalue.getType(), lvalue.getBaseInfo(), lvalue.getTBAAInfo(), isInit, lvalue.getTBAABaseType(), lvalue.getTBAAOffset(), lvalue.isNontemporal()); } @@ -1487,10 +1635,11 @@ RValue CodeGenFunction::EmitLoadOfLValue(LValue LV, SourceLocation Loc) { return EmitLoadOfGlobalRegLValue(LV); assert(LV.isBitField() && "Unknown LValue type!"); - return EmitLoadOfBitfieldLValue(LV); + return EmitLoadOfBitfieldLValue(LV, Loc); } -RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV) { +RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV, + SourceLocation Loc) { const CGBitFieldInfo &Info = LV.getBitFieldInfo(); // Get the output type. @@ -1515,7 +1664,7 @@ RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV) { "bf.clear"); } Val = Builder.CreateIntCast(Val, ResLTy, Info.IsSigned, "bf.cast"); - + EmitScalarRangeCheck(Val, LV.getType(), Loc); return RValue::get(Val); } @@ -1967,38 +2116,39 @@ static LValue EmitThreadPrivateVarDeclLValue( llvm::Type *RealVarTy, SourceLocation Loc) { Addr = CGF.CGM.getOpenMPRuntime().getAddrOfThreadPrivate(CGF, VD, Addr, Loc); Addr = CGF.Builder.CreateElementBitCast(Addr, RealVarTy); - return CGF.MakeAddrLValue(Addr, T, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + return CGF.MakeAddrLValue(Addr, T, BaseInfo); } Address CodeGenFunction::EmitLoadOfReference(Address Addr, const ReferenceType *RefTy, - AlignmentSource *Source) { + LValueBaseInfo *BaseInfo) { llvm::Value *Ptr = Builder.CreateLoad(Addr); return Address(Ptr, getNaturalTypeAlignment(RefTy->getPointeeType(), - Source, /*forPointee*/ true)); - + BaseInfo, /*forPointee*/ true)); } LValue CodeGenFunction::EmitLoadOfReferenceLValue(Address RefAddr, const ReferenceType *RefTy) { - AlignmentSource Source; - Address Addr = EmitLoadOfReference(RefAddr, RefTy, &Source); - return MakeAddrLValue(Addr, RefTy->getPointeeType(), Source); + LValueBaseInfo BaseInfo; + Address Addr = EmitLoadOfReference(RefAddr, RefTy, &BaseInfo); + return MakeAddrLValue(Addr, RefTy->getPointeeType(), BaseInfo); } Address CodeGenFunction::EmitLoadOfPointer(Address Ptr, const PointerType *PtrTy, - AlignmentSource *Source) { + LValueBaseInfo *BaseInfo) { llvm::Value *Addr = Builder.CreateLoad(Ptr); - return Address(Addr, getNaturalTypeAlignment(PtrTy->getPointeeType(), Source, + return Address(Addr, getNaturalTypeAlignment(PtrTy->getPointeeType(), + BaseInfo, /*forPointeeType=*/true)); } LValue CodeGenFunction::EmitLoadOfPointerLValue(Address PtrAddr, const PointerType *PtrTy) { - AlignmentSource Source; - Address Addr = EmitLoadOfPointer(PtrAddr, PtrTy, &Source); - return MakeAddrLValue(Addr, PtrTy->getPointeeType(), Source); + LValueBaseInfo BaseInfo; + Address Addr = EmitLoadOfPointer(PtrAddr, PtrTy, &BaseInfo); + return MakeAddrLValue(Addr, PtrTy->getPointeeType(), BaseInfo); } static LValue EmitGlobalVarDeclLValue(CodeGenFunction &CGF, @@ -2024,7 +2174,8 @@ static LValue EmitGlobalVarDeclLValue(CodeGenFunction &CGF, if (auto RefTy = VD->getType()->getAs<ReferenceType>()) { LV = CGF.EmitLoadOfReferenceLValue(Addr, RefTy); } else { - LV = CGF.MakeAddrLValue(Addr, T, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + LV = CGF.MakeAddrLValue(Addr, T, BaseInfo); } setObjCGCLValueClass(CGF.getContext(), E, LV); return LV; @@ -2058,7 +2209,8 @@ static LValue EmitFunctionDeclLValue(CodeGenFunction &CGF, const Expr *E, const FunctionDecl *FD) { llvm::Value *V = EmitFunctionDeclPointer(CGF.CGM, FD); CharUnits Alignment = CGF.getContext().getDeclAlign(FD); - return CGF.MakeAddrLValue(V, E->getType(), Alignment, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + return CGF.MakeAddrLValue(V, E->getType(), Alignment, BaseInfo); } static LValue EmitCapturedFieldLValue(CodeGenFunction &CGF, const FieldDecl *FD, @@ -2123,8 +2275,8 @@ LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) { // Should we be using the alignment of the constant pointer we emitted? CharUnits Alignment = getNaturalTypeAlignment(E->getType(), nullptr, /*pointee*/ true); - - return MakeAddrLValue(Address(Val, Alignment), T, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + return MakeAddrLValue(Address(Val, Alignment), T, BaseInfo); } // Check for captured variables. @@ -2141,14 +2293,16 @@ LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) { LValue CapLVal = EmitCapturedFieldLValue(*this, CapturedStmtInfo->lookup(VD), CapturedStmtInfo->getContextValue()); + bool MayAlias = CapLVal.getBaseInfo().getMayAlias(); return MakeAddrLValue( Address(CapLVal.getPointer(), getContext().getDeclAlign(VD)), - CapLVal.getType(), AlignmentSource::Decl); + CapLVal.getType(), LValueBaseInfo(AlignmentSource::Decl, MayAlias)); } assert(isa<BlockDecl>(CurCodeDecl)); Address addr = GetAddrOfBlockDecl(VD, VD->hasAttr<BlocksAttr>()); - return MakeAddrLValue(addr, T, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + return MakeAddrLValue(addr, T, BaseInfo); } } @@ -2162,7 +2316,8 @@ LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) { if (ND->hasAttr<WeakRefAttr>()) { const auto *VD = cast<ValueDecl>(ND); ConstantAddress Aliasee = CGM.GetWeakRefReference(VD); - return MakeAddrLValue(Aliasee, T, AlignmentSource::Decl); + return MakeAddrLValue(Aliasee, T, + LValueBaseInfo(AlignmentSource::Decl, false)); } if (const auto *VD = dyn_cast<VarDecl>(ND)) { @@ -2208,7 +2363,8 @@ LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) { if (auto RefTy = VD->getType()->getAs<ReferenceType>()) { LV = EmitLoadOfReferenceLValue(addr, RefTy); } else { - LV = MakeAddrLValue(addr, T, AlignmentSource::Decl); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); + LV = MakeAddrLValue(addr, T, BaseInfo); } bool isLocalStorage = VD->hasLocalStorage(); @@ -2253,9 +2409,9 @@ LValue CodeGenFunction::EmitUnaryOpLValue(const UnaryOperator *E) { QualType T = E->getSubExpr()->getType()->getPointeeType(); assert(!T.isNull() && "CodeGenFunction::EmitUnaryOpLValue: Illegal type"); - AlignmentSource AlignSource; - Address Addr = EmitPointerWithAlignment(E->getSubExpr(), &AlignSource); - LValue LV = MakeAddrLValue(Addr, T, AlignSource); + LValueBaseInfo BaseInfo; + Address Addr = EmitPointerWithAlignment(E->getSubExpr(), &BaseInfo); + LValue LV = MakeAddrLValue(Addr, T, BaseInfo); LV.getQuals().setAddressSpace(ExprTy.getAddressSpace()); // We should not generate __weak write barrier on indirect reference @@ -2287,7 +2443,7 @@ LValue CodeGenFunction::EmitUnaryOpLValue(const UnaryOperator *E) { (E->getOpcode() == UO_Real ? emitAddrOfRealComponent(LV.getAddress(), LV.getType()) : emitAddrOfImagComponent(LV.getAddress(), LV.getType())); - LValue ElemLV = MakeAddrLValue(Component, T, LV.getAlignmentSource()); + LValue ElemLV = MakeAddrLValue(Component, T, LV.getBaseInfo()); ElemLV.getQuals().addQualifiers(LV.getQuals()); return ElemLV; } @@ -2307,12 +2463,14 @@ LValue CodeGenFunction::EmitUnaryOpLValue(const UnaryOperator *E) { LValue CodeGenFunction::EmitStringLiteralLValue(const StringLiteral *E) { return MakeAddrLValue(CGM.GetAddrOfConstantStringFromLiteral(E), - E->getType(), AlignmentSource::Decl); + E->getType(), + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue CodeGenFunction::EmitObjCEncodeExprLValue(const ObjCEncodeExpr *E) { return MakeAddrLValue(CGM.GetAddrOfConstantStringFromObjCEncode(E), - E->getType(), AlignmentSource::Decl); + E->getType(), + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue CodeGenFunction::EmitPredefinedLValue(const PredefinedExpr *E) { @@ -2324,6 +2482,7 @@ LValue CodeGenFunction::EmitPredefinedLValue(const PredefinedExpr *E) { StringRef NameItems[] = { PredefinedExpr::getIdentTypeName(E->getIdentType()), FnName}; std::string GVName = llvm::join(NameItems, NameItems + 2, "."); + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); if (auto *BD = dyn_cast<BlockDecl>(CurCodeDecl)) { std::string Name = SL->getString(); if (!Name.empty()) { @@ -2332,14 +2491,14 @@ LValue CodeGenFunction::EmitPredefinedLValue(const PredefinedExpr *E) { if (Discriminator) Name += "_" + Twine(Discriminator + 1).str(); auto C = CGM.GetAddrOfConstantCString(Name, GVName.c_str()); - return MakeAddrLValue(C, E->getType(), AlignmentSource::Decl); + return MakeAddrLValue(C, E->getType(), BaseInfo); } else { auto C = CGM.GetAddrOfConstantCString(FnName, GVName.c_str()); - return MakeAddrLValue(C, E->getType(), AlignmentSource::Decl); + return MakeAddrLValue(C, E->getType(), BaseInfo); } } auto C = CGM.GetAddrOfConstantStringFromLiteral(SL, GVName); - return MakeAddrLValue(C, E->getType(), AlignmentSource::Decl); + return MakeAddrLValue(C, E->getType(), BaseInfo); } /// Emit a type description suitable for use by a runtime sanitizer library. The @@ -2545,8 +2704,8 @@ static void emitCheckHandlerCall(CodeGenFunction &CGF, llvm::Value *Fn = CGF.CGM.CreateRuntimeFunction( FnType, FnName, - llvm::AttributeSet::get(CGF.getLLVMContext(), - llvm::AttributeSet::FunctionIndex, B), + llvm::AttributeList::get(CGF.getLLVMContext(), + llvm::AttributeList::FunctionIndex, B), /*Local=*/true); llvm::CallInst *HandlerCall = CGF.EmitNounwindRuntimeCall(Fn, FnArgs); if (!MayReturn) { @@ -2709,6 +2868,24 @@ void CodeGenFunction::EmitCfiSlowPathCheck( EmitBlock(Cont); } +// Emit a stub for __cfi_check function so that the linker knows about this +// symbol in LTO mode. +void CodeGenFunction::EmitCfiCheckStub() { + llvm::Module *M = &CGM.getModule(); + auto &Ctx = M->getContext(); + llvm::Function *F = llvm::Function::Create( + llvm::FunctionType::get(VoidTy, {Int64Ty, Int8PtrTy, Int8PtrTy}, false), + llvm::GlobalValue::WeakAnyLinkage, "__cfi_check", M); + llvm::BasicBlock *BB = llvm::BasicBlock::Create(Ctx, "entry", F); + // FIXME: consider emitting an intrinsic call like + // call void @llvm.cfi_check(i64 %0, i8* %1, i8* %2) + // which can be lowered in CrossDSOCFI pass to the actual contents of + // __cfi_check. This would allow inlining of __cfi_check calls. + llvm::CallInst::Create( + llvm::Intrinsic::getDeclaration(M, llvm::Intrinsic::trap), "", BB); + llvm::ReturnInst::Create(Ctx, nullptr, BB); +} + // This function is basically a switch over the CFI failure kind, which is // extracted from CFICheckFailData (1st function argument). Each case is either // llvm.trap or a call to one of the two runtime handlers, based on @@ -2719,10 +2896,10 @@ void CodeGenFunction::EmitCfiSlowPathCheck( void CodeGenFunction::EmitCfiCheckFail() { SanitizerScope SanScope(this); FunctionArgList Args; - ImplicitParamDecl ArgData(getContext(), nullptr, SourceLocation(), nullptr, - getContext().VoidPtrTy); - ImplicitParamDecl ArgAddr(getContext(), nullptr, SourceLocation(), nullptr, - getContext().VoidPtrTy); + ImplicitParamDecl ArgData(getContext(), getContext().VoidPtrTy, + ImplicitParamDecl::Other); + ImplicitParamDecl ArgAddr(getContext(), getContext().VoidPtrTy, + ImplicitParamDecl::Other); Args.push_back(&ArgData); Args.push_back(&ArgAddr); @@ -2750,9 +2927,9 @@ void CodeGenFunction::EmitCfiCheckFail() { EmitTrapCheck(DataIsNotNullPtr); llvm::StructType *SourceLocationTy = - llvm::StructType::get(VoidPtrTy, Int32Ty, Int32Ty, nullptr); + llvm::StructType::get(VoidPtrTy, Int32Ty, Int32Ty); llvm::StructType *CfiCheckFailDataTy = - llvm::StructType::get(Int8Ty, SourceLocationTy, VoidPtrTy, nullptr); + llvm::StructType::get(Int8Ty, SourceLocationTy, VoidPtrTy); llvm::Value *V = Builder.CreateConstGEP2_32( CfiCheckFailDataTy, @@ -2821,21 +2998,21 @@ llvm::CallInst *CodeGenFunction::EmitTrapCall(llvm::Intrinsic::ID IntrID) { if (!CGM.getCodeGenOpts().TrapFuncName.empty()) { auto A = llvm::Attribute::get(getLLVMContext(), "trap-func-name", CGM.getCodeGenOpts().TrapFuncName); - TrapCall->addAttribute(llvm::AttributeSet::FunctionIndex, A); + TrapCall->addAttribute(llvm::AttributeList::FunctionIndex, A); } return TrapCall; } Address CodeGenFunction::EmitArrayToPointerDecay(const Expr *E, - AlignmentSource *AlignSource) { + LValueBaseInfo *BaseInfo) { assert(E->getType()->isArrayType() && "Array to pointer decay must have array source type!"); // Expressions of array type can't be bitfields or vector elements. LValue LV = EmitLValue(E); Address Addr = LV.getAddress(); - if (AlignSource) *AlignSource = LV.getAlignmentSource(); + if (BaseInfo) *BaseInfo = LV.getBaseInfo(); // If the array type was an incomplete type, we need to make sure // the decay ends up being the right type. @@ -2874,9 +3051,13 @@ static llvm::Value *emitArraySubscriptGEP(CodeGenFunction &CGF, llvm::Value *ptr, ArrayRef<llvm::Value*> indices, bool inbounds, + bool signedIndices, + SourceLocation loc, const llvm::Twine &name = "arrayidx") { if (inbounds) { - return CGF.Builder.CreateInBoundsGEP(ptr, indices, name); + return CGF.EmitCheckedInBoundsGEP(ptr, indices, signedIndices, + CodeGenFunction::NotSubtraction, loc, + name); } else { return CGF.Builder.CreateGEP(ptr, indices, name); } @@ -2907,8 +3088,9 @@ static QualType getFixedSizeElementType(const ASTContext &ctx, } static Address emitArraySubscriptGEP(CodeGenFunction &CGF, Address addr, - ArrayRef<llvm::Value*> indices, + ArrayRef<llvm::Value *> indices, QualType eltType, bool inbounds, + bool signedIndices, SourceLocation loc, const llvm::Twine &name = "arrayidx") { // All the indices except that last must be zero. #ifndef NDEBUG @@ -2928,8 +3110,8 @@ static Address emitArraySubscriptGEP(CodeGenFunction &CGF, Address addr, CharUnits eltAlign = getArrayElementAlign(addr.getAlignment(), indices.back(), eltSize); - llvm::Value *eltPtr = - emitArraySubscriptGEP(CGF, addr.getPointer(), indices, inbounds, name); + llvm::Value *eltPtr = emitArraySubscriptGEP( + CGF, addr.getPointer(), indices, inbounds, signedIndices, loc, name); return Address(eltPtr, eltAlign); } @@ -2939,6 +3121,7 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, // in lexical order (this complexity is, sadly, required by C++17). llvm::Value *IdxPre = (E->getLHS() == E->getIdx()) ? EmitScalarExpr(E->getIdx()) : nullptr; + bool SignedIndices = false; auto EmitIdxAfterBase = [&, IdxPre](bool Promote) -> llvm::Value * { auto *Idx = IdxPre; if (E->getLHS() != E->getIdx()) { @@ -2948,6 +3131,7 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, QualType IdxTy = E->getIdx()->getType(); bool IdxSigned = IdxTy->isSignedIntegerOrEnumerationType(); + SignedIndices |= IdxSigned; if (SanOpts.has(SanitizerKind::ArrayBounds)) EmitBoundsCheck(E, E->getBase(), Idx, IdxTy, Accessed); @@ -2970,7 +3154,7 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, assert(LHS.isSimple() && "Can only subscript lvalue vectors here!"); return LValue::MakeVectorElt(LHS.getAddress(), Idx, E->getBase()->getType(), - LHS.getAlignmentSource()); + LHS.getBaseInfo()); } // All the other cases basically behave like simple offsetting. @@ -2982,18 +3166,19 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, Address Addr = EmitExtVectorElementLValue(LV); QualType EltType = LV.getType()->castAs<VectorType>()->getElementType(); - Addr = emitArraySubscriptGEP(*this, Addr, Idx, EltType, /*inbounds*/ true); - return MakeAddrLValue(Addr, EltType, LV.getAlignmentSource()); + Addr = emitArraySubscriptGEP(*this, Addr, Idx, EltType, /*inbounds*/ true, + SignedIndices, E->getExprLoc()); + return MakeAddrLValue(Addr, EltType, LV.getBaseInfo()); } - AlignmentSource AlignSource; + LValueBaseInfo BaseInfo; Address Addr = Address::invalid(); if (const VariableArrayType *vla = getContext().getAsVariableArrayType(E->getType())) { // The base must be a pointer, which is not an aggregate. Emit // it. It needs to be emitted first in case it's what captures // the VLA bounds. - Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource); + Addr = EmitPointerWithAlignment(E->getBase(), &BaseInfo); auto *Idx = EmitIdxAfterBase(/*Promote*/true); // The element count here is the total number of non-VLA elements. @@ -3010,13 +3195,14 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, } Addr = emitArraySubscriptGEP(*this, Addr, Idx, vla->getElementType(), - !getLangOpts().isSignedOverflowDefined()); + !getLangOpts().isSignedOverflowDefined(), + SignedIndices, E->getExprLoc()); } else if (const ObjCObjectType *OIT = E->getType()->getAs<ObjCObjectType>()){ // Indexing over an interface, as in "NSString *P; P[4];" // Emit the base pointer. - Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource); + Addr = EmitPointerWithAlignment(E->getBase(), &BaseInfo); auto *Idx = EmitIdxAfterBase(/*Promote*/true); CharUnits InterfaceSize = getContext().getTypeSizeInChars(OIT); @@ -3036,7 +3222,8 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, CharUnits EltAlign = getArrayElementAlign(Addr.getAlignment(), Idx, InterfaceSize); llvm::Value *EltPtr = - emitArraySubscriptGEP(*this, Addr.getPointer(), ScaledIdx, false); + emitArraySubscriptGEP(*this, Addr.getPointer(), ScaledIdx, false, + SignedIndices, E->getExprLoc()); Addr = Address(EltPtr, EltAlign); // Cast back. @@ -3058,20 +3245,21 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, auto *Idx = EmitIdxAfterBase(/*Promote*/true); // Propagate the alignment from the array itself to the result. - Addr = emitArraySubscriptGEP(*this, ArrayLV.getAddress(), - {CGM.getSize(CharUnits::Zero()), Idx}, - E->getType(), - !getLangOpts().isSignedOverflowDefined()); - AlignSource = ArrayLV.getAlignmentSource(); + Addr = emitArraySubscriptGEP( + *this, ArrayLV.getAddress(), {CGM.getSize(CharUnits::Zero()), Idx}, + E->getType(), !getLangOpts().isSignedOverflowDefined(), SignedIndices, + E->getExprLoc()); + BaseInfo = ArrayLV.getBaseInfo(); } else { // The base must be a pointer; emit it with an estimate of its alignment. - Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource); + Addr = EmitPointerWithAlignment(E->getBase(), &BaseInfo); auto *Idx = EmitIdxAfterBase(/*Promote*/true); Addr = emitArraySubscriptGEP(*this, Addr, Idx, E->getType(), - !getLangOpts().isSignedOverflowDefined()); + !getLangOpts().isSignedOverflowDefined(), + SignedIndices, E->getExprLoc()); } - LValue LV = MakeAddrLValue(Addr, E->getType(), AlignSource); + LValue LV = MakeAddrLValue(Addr, E->getType(), BaseInfo); // TODO: Preserve/extend path TBAA metadata? @@ -3084,7 +3272,7 @@ LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E, } static Address emitOMPArraySectionBase(CodeGenFunction &CGF, const Expr *Base, - AlignmentSource &AlignSource, + LValueBaseInfo &BaseInfo, QualType BaseTy, QualType ElTy, bool IsLowerBound) { LValue BaseLVal; @@ -3092,7 +3280,7 @@ static Address emitOMPArraySectionBase(CodeGenFunction &CGF, const Expr *Base, BaseLVal = CGF.EmitOMPArraySectionExpr(ASE, IsLowerBound); if (BaseTy->isArrayType()) { Address Addr = BaseLVal.getAddress(); - AlignSource = BaseLVal.getAlignmentSource(); + BaseInfo = BaseLVal.getBaseInfo(); // If the array type was an incomplete type, we need to make sure // the decay ends up being the right type. @@ -3111,10 +3299,12 @@ static Address emitOMPArraySectionBase(CodeGenFunction &CGF, const Expr *Base, return CGF.Builder.CreateElementBitCast(Addr, CGF.ConvertTypeForMem(ElTy)); } - CharUnits Align = CGF.getNaturalTypeAlignment(ElTy, &AlignSource); + LValueBaseInfo TypeInfo; + CharUnits Align = CGF.getNaturalTypeAlignment(ElTy, &TypeInfo); + BaseInfo.mergeForCast(TypeInfo); return Address(CGF.Builder.CreateLoad(BaseLVal.getAddress()), Align); } - return CGF.EmitPointerWithAlignment(Base, &AlignSource); + return CGF.EmitPointerWithAlignment(Base, &BaseInfo); } LValue CodeGenFunction::EmitOMPArraySectionExpr(const OMPArraySectionExpr *E, @@ -3218,13 +3408,13 @@ LValue CodeGenFunction::EmitOMPArraySectionExpr(const OMPArraySectionExpr *E, assert(Idx); Address EltPtr = Address::invalid(); - AlignmentSource AlignSource; + LValueBaseInfo BaseInfo; if (auto *VLA = getContext().getAsVariableArrayType(ResultExprTy)) { // The base must be a pointer, which is not an aggregate. Emit // it. It needs to be emitted first in case it's what captures // the VLA bounds. Address Base = - emitOMPArraySectionBase(*this, E->getBase(), AlignSource, BaseTy, + emitOMPArraySectionBase(*this, E->getBase(), BaseInfo, BaseTy, VLA->getElementType(), IsLowerBound); // The element count here is the total number of non-VLA elements. llvm::Value *NumElements = getVLASize(VLA).first; @@ -3238,7 +3428,8 @@ LValue CodeGenFunction::EmitOMPArraySectionExpr(const OMPArraySectionExpr *E, else Idx = Builder.CreateNSWMul(Idx, NumElements); EltPtr = emitArraySubscriptGEP(*this, Base, Idx, VLA->getElementType(), - !getLangOpts().isSignedOverflowDefined()); + !getLangOpts().isSignedOverflowDefined(), + /*SignedIndices=*/false, E->getExprLoc()); } else if (const Expr *Array = isSimpleArrayDecayOperand(E->getBase())) { // If this is A[i] where A is an array, the frontend will have decayed the // base to be a ArrayToPointerDecay implicit cast. While correct, it is @@ -3257,16 +3448,18 @@ LValue CodeGenFunction::EmitOMPArraySectionExpr(const OMPArraySectionExpr *E, // Propagate the alignment from the array itself to the result. EltPtr = emitArraySubscriptGEP( *this, ArrayLV.getAddress(), {CGM.getSize(CharUnits::Zero()), Idx}, - ResultExprTy, !getLangOpts().isSignedOverflowDefined()); - AlignSource = ArrayLV.getAlignmentSource(); + ResultExprTy, !getLangOpts().isSignedOverflowDefined(), + /*SignedIndices=*/false, E->getExprLoc()); + BaseInfo = ArrayLV.getBaseInfo(); } else { - Address Base = emitOMPArraySectionBase(*this, E->getBase(), AlignSource, + Address Base = emitOMPArraySectionBase(*this, E->getBase(), BaseInfo, BaseTy, ResultExprTy, IsLowerBound); EltPtr = emitArraySubscriptGEP(*this, Base, Idx, ResultExprTy, - !getLangOpts().isSignedOverflowDefined()); + !getLangOpts().isSignedOverflowDefined(), + /*SignedIndices=*/false, E->getExprLoc()); } - return MakeAddrLValue(EltPtr, ResultExprTy, AlignSource); + return MakeAddrLValue(EltPtr, ResultExprTy, BaseInfo); } LValue CodeGenFunction:: @@ -3278,10 +3471,10 @@ EmitExtVectorElementExpr(const ExtVectorElementExpr *E) { if (E->isArrow()) { // If it is a pointer to a vector, emit the address and form an lvalue with // it. - AlignmentSource AlignSource; - Address Ptr = EmitPointerWithAlignment(E->getBase(), &AlignSource); + LValueBaseInfo BaseInfo; + Address Ptr = EmitPointerWithAlignment(E->getBase(), &BaseInfo); const PointerType *PT = E->getBase()->getType()->getAs<PointerType>(); - Base = MakeAddrLValue(Ptr, PT->getPointeeType(), AlignSource); + Base = MakeAddrLValue(Ptr, PT->getPointeeType(), BaseInfo); Base.getQuals().removeObjCGCAttr(); } else if (E->getBase()->isGLValue()) { // Otherwise, if the base is an lvalue ( as in the case of foo.x.x), @@ -3298,7 +3491,7 @@ EmitExtVectorElementExpr(const ExtVectorElementExpr *E) { Address VecMem = CreateMemTemp(E->getBase()->getType()); Builder.CreateStore(Vec, VecMem); Base = MakeAddrLValue(VecMem, E->getBase()->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } QualType type = @@ -3312,7 +3505,7 @@ EmitExtVectorElementExpr(const ExtVectorElementExpr *E) { llvm::Constant *CV = llvm::ConstantDataVector::get(getLLVMContext(), Indices); return LValue::MakeExtVectorElt(Base.getAddress(), CV, type, - Base.getAlignmentSource()); + Base.getBaseInfo()); } assert(Base.isExtVectorElt() && "Can only subscript lvalue vec elts here!"); @@ -3323,20 +3516,26 @@ EmitExtVectorElementExpr(const ExtVectorElementExpr *E) { CElts.push_back(BaseElts->getAggregateElement(Indices[i])); llvm::Constant *CV = llvm::ConstantVector::get(CElts); return LValue::MakeExtVectorElt(Base.getExtVectorAddress(), CV, type, - Base.getAlignmentSource()); + Base.getBaseInfo()); } LValue CodeGenFunction::EmitMemberExpr(const MemberExpr *E) { Expr *BaseExpr = E->getBase(); - // If this is s.x, emit s as an lvalue. If it is s->x, emit s as a scalar. LValue BaseLV; if (E->isArrow()) { - AlignmentSource AlignSource; - Address Addr = EmitPointerWithAlignment(BaseExpr, &AlignSource); + LValueBaseInfo BaseInfo; + Address Addr = EmitPointerWithAlignment(BaseExpr, &BaseInfo); QualType PtrTy = BaseExpr->getType()->getPointeeType(); - EmitTypeCheck(TCK_MemberAccess, E->getExprLoc(), Addr.getPointer(), PtrTy); - BaseLV = MakeAddrLValue(Addr, PtrTy, AlignSource); + SanitizerSet SkippedChecks; + bool IsBaseCXXThis = IsWrappedCXXThis(BaseExpr); + if (IsBaseCXXThis) + SkippedChecks.set(SanitizerKind::Alignment, true); + if (IsBaseCXXThis || isa<DeclRefExpr>(BaseExpr)) + SkippedChecks.set(SanitizerKind::Null, true); + EmitTypeCheck(TCK_MemberAccess, E->getExprLoc(), Addr.getPointer(), PtrTy, + /*Alignment=*/CharUnits::Zero(), SkippedChecks); + BaseLV = MakeAddrLValue(Addr, PtrTy, BaseInfo); } else BaseLV = EmitCheckedLValue(BaseExpr, TCK_MemberAccess); @@ -3394,10 +3593,36 @@ static Address emitAddrOfFieldStorage(CodeGenFunction &CGF, Address base, return CGF.Builder.CreateStructGEP(base, idx, offset, field->getName()); } +static bool hasAnyVptr(const QualType Type, const ASTContext &Context) { + const auto *RD = Type.getTypePtr()->getAsCXXRecordDecl(); + if (!RD) + return false; + + if (RD->isDynamicClass()) + return true; + + for (const auto &Base : RD->bases()) + if (hasAnyVptr(Base.getType(), Context)) + return true; + + for (const FieldDecl *Field : RD->fields()) + if (hasAnyVptr(Field->getType(), Context)) + return true; + + return false; +} + LValue CodeGenFunction::EmitLValueForField(LValue base, const FieldDecl *field) { + LValueBaseInfo BaseInfo = base.getBaseInfo(); AlignmentSource fieldAlignSource = - getFieldAlignmentSource(base.getAlignmentSource()); + getFieldAlignmentSource(BaseInfo.getAlignmentSource()); + LValueBaseInfo FieldBaseInfo(fieldAlignSource, BaseInfo.getMayAlias()); + + const RecordDecl *rec = field->getParent(); + if (rec->isUnion() || rec->hasAttr<MayAliasAttr>()) + FieldBaseInfo.setMayAlias(true); + bool mayAlias = FieldBaseInfo.getMayAlias(); if (field->isBitField()) { const CGRecordLayout &RL = @@ -3417,14 +3642,10 @@ LValue CodeGenFunction::EmitLValueForField(LValue base, QualType fieldType = field->getType().withCVRQualifiers(base.getVRQualifiers()); - return LValue::MakeBitfield(Addr, Info, fieldType, fieldAlignSource); + return LValue::MakeBitfield(Addr, Info, fieldType, FieldBaseInfo); } - const RecordDecl *rec = field->getParent(); QualType type = field->getType(); - - bool mayAlias = rec->hasAttr<MayAliasAttr>(); - Address addr = base.getAddress(); unsigned cvr = base.getVRQualifiers(); bool TBAAPath = CGM.getCodeGenOpts().StructPathTBAA; @@ -3433,6 +3654,14 @@ LValue CodeGenFunction::EmitLValueForField(LValue base, assert(!type->isReferenceType() && "union has reference member"); // TODO: handle path-aware TBAA for union. TBAAPath = false; + + const auto FieldType = field->getType(); + if (CGM.getCodeGenOpts().StrictVTablePointers && + hasAnyVptr(FieldType, getContext())) + // Because unions can easily skip invariant.barriers, we need to add + // a barrier every time CXXRecord field with vptr is referenced. + addr = Address(Builder.CreateInvariantGroupBarrier(addr.getPointer()), + addr.getAlignment()); } else { // For structs, we GEP to the field that the record layout suggests. addr = emitAddrOfFieldStorage(*this, addr, field); @@ -3458,7 +3687,8 @@ LValue CodeGenFunction::EmitLValueForField(LValue base, type = refType->getPointeeType(); CharUnits alignment = - getNaturalTypeAlignment(type, &fieldAlignSource, /*pointee*/ true); + getNaturalTypeAlignment(type, &FieldBaseInfo, /*pointee*/ true); + FieldBaseInfo.setMayAlias(false); addr = Address(load, alignment); // Qualifiers on the struct don't apply to the referencee, and @@ -3479,7 +3709,7 @@ LValue CodeGenFunction::EmitLValueForField(LValue base, if (field->hasAttr<AnnotateAttr>()) addr = EmitFieldAnnotations(field, addr); - LValue LV = MakeAddrLValue(addr, type, fieldAlignSource); + LValue LV = MakeAddrLValue(addr, type, FieldBaseInfo); LV.getQuals().addCVRQualifiers(cvr); if (TBAAPath) { const ASTRecordLayout &Layout = @@ -3520,14 +3750,18 @@ CodeGenFunction::EmitLValueForFieldInitialization(LValue Base, V = Builder.CreateElementBitCast(V, llvmType, Field->getName()); // TODO: access-path TBAA? - auto FieldAlignSource = getFieldAlignmentSource(Base.getAlignmentSource()); - return MakeAddrLValue(V, FieldType, FieldAlignSource); + LValueBaseInfo BaseInfo = Base.getBaseInfo(); + LValueBaseInfo FieldBaseInfo( + getFieldAlignmentSource(BaseInfo.getAlignmentSource()), + BaseInfo.getMayAlias()); + return MakeAddrLValue(V, FieldType, FieldBaseInfo); } LValue CodeGenFunction::EmitCompoundLiteralLValue(const CompoundLiteralExpr *E){ + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); if (E->isFileScope()) { ConstantAddress GlobalPtr = CGM.GetAddrOfConstantCompoundLiteral(E); - return MakeAddrLValue(GlobalPtr, E->getType(), AlignmentSource::Decl); + return MakeAddrLValue(GlobalPtr, E->getType(), BaseInfo); } if (E->getType()->isVariablyModifiedType()) // make sure to emit the VLA size. @@ -3535,7 +3769,7 @@ LValue CodeGenFunction::EmitCompoundLiteralLValue(const CompoundLiteralExpr *E){ Address DeclPtr = CreateMemTemp(E->getType(), ".compoundliteral"); const Expr *InitExpr = E->getInitializer(); - LValue Result = MakeAddrLValue(DeclPtr, E->getType(), AlignmentSource::Decl); + LValue Result = MakeAddrLValue(DeclPtr, E->getType(), BaseInfo); EmitAnyExprToMem(InitExpr, DeclPtr, E->getType().getQualifiers(), /*Init*/ true); @@ -3632,8 +3866,12 @@ EmitConditionalOperatorLValue(const AbstractConditionalOperator *expr) { phi->addIncoming(rhs->getPointer(), rhsBlock); Address result(phi, std::min(lhs->getAlignment(), rhs->getAlignment())); AlignmentSource alignSource = - std::max(lhs->getAlignmentSource(), rhs->getAlignmentSource()); - return MakeAddrLValue(result, expr->getType(), alignSource); + std::max(lhs->getBaseInfo().getAlignmentSource(), + rhs->getBaseInfo().getAlignmentSource()); + bool MayAlias = lhs->getBaseInfo().getMayAlias() || + rhs->getBaseInfo().getMayAlias(); + return MakeAddrLValue(result, expr->getType(), + LValueBaseInfo(alignSource, MayAlias)); } else { assert((lhs || rhs) && "both operands of glvalue conditional are throw-expressions?"); @@ -3731,7 +3969,7 @@ LValue CodeGenFunction::EmitCastLValue(const CastExpr *E) { This, DerivedClassDecl, E->path_begin(), E->path_end(), /*NullCheckValue=*/false, E->getExprLoc()); - return MakeAddrLValue(Base, E->getType(), LV.getAlignmentSource()); + return MakeAddrLValue(Base, E->getType(), LV.getBaseInfo()); } case CK_ToUnion: return EmitAggExprToLValue(E); @@ -3758,7 +3996,7 @@ LValue CodeGenFunction::EmitCastLValue(const CastExpr *E) { /*MayBeNull=*/false, CFITCK_DerivedCast, E->getLocStart()); - return MakeAddrLValue(Derived, E->getType(), LV.getAlignmentSource()); + return MakeAddrLValue(Derived, E->getType(), LV.getBaseInfo()); } case CK_LValueBitCast: { // This must be a reinterpret_cast (or c-style equivalent). @@ -3774,13 +4012,13 @@ LValue CodeGenFunction::EmitCastLValue(const CastExpr *E) { /*MayBeNull=*/false, CFITCK_UnrelatedCast, E->getLocStart()); - return MakeAddrLValue(V, E->getType(), LV.getAlignmentSource()); + return MakeAddrLValue(V, E->getType(), LV.getBaseInfo()); } case CK_ObjCObjectLValueCast: { LValue LV = EmitLValue(E->getSubExpr()); Address V = Builder.CreateElementBitCast(LV.getAddress(), ConvertType(E->getType())); - return MakeAddrLValue(V, E->getType(), LV.getAlignmentSource()); + return MakeAddrLValue(V, E->getType(), LV.getBaseInfo()); } case CK_ZeroToOCLQueue: llvm_unreachable("NULL to OpenCL queue lvalue cast is not valid"); @@ -3949,6 +4187,8 @@ LValue CodeGenFunction::EmitBinaryOperatorLValue(const BinaryOperator *E) { RValue RV = EmitAnyExpr(E->getRHS()); LValue LV = EmitCheckedLValue(E->getLHS(), TCK_Store); + if (RV.isScalar()) + EmitNullabilityCheck(LV, RV.getScalarVal(), E->getExprLoc()); EmitStoreThroughLValue(RV, LV); return LV; } @@ -3967,7 +4207,7 @@ LValue CodeGenFunction::EmitCallExprLValue(const CallExpr *E) { if (!RV.isScalar()) return MakeAddrLValue(RV.getAggregateAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); assert(E->getCallReturnType(getContext())->isReferenceType() && "Can't have a scalar return unless the return type is a " @@ -3987,7 +4227,7 @@ LValue CodeGenFunction::EmitCXXConstructLValue(const CXXConstructExpr *E) { AggValueSlot Slot = CreateAggTemp(E->getType()); EmitCXXConstructExpr(E, Slot); return MakeAddrLValue(Slot.getAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue @@ -4002,7 +4242,7 @@ Address CodeGenFunction::EmitCXXUuidofExpr(const CXXUuidofExpr *E) { LValue CodeGenFunction::EmitCXXUuidofLValue(const CXXUuidofExpr *E) { return MakeAddrLValue(EmitCXXUuidofExpr(E), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue @@ -4012,7 +4252,7 @@ CodeGenFunction::EmitCXXBindTemporaryLValue(const CXXBindTemporaryExpr *E) { EmitAggExpr(E->getSubExpr(), Slot); EmitCXXTemporary(E->getTemporary(), E->getType(), Slot.getAddress()); return MakeAddrLValue(Slot.getAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue @@ -4020,7 +4260,7 @@ CodeGenFunction::EmitLambdaLValue(const LambdaExpr *E) { AggValueSlot Slot = CreateAggTemp(E->getType(), "temp.lvalue"); EmitLambdaExpr(E, Slot); return MakeAddrLValue(Slot.getAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } LValue CodeGenFunction::EmitObjCMessageExprLValue(const ObjCMessageExpr *E) { @@ -4028,7 +4268,7 @@ LValue CodeGenFunction::EmitObjCMessageExprLValue(const ObjCMessageExpr *E) { if (!RV.isScalar()) return MakeAddrLValue(RV.getAggregateAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); assert(E->getMethodDecl()->getReturnType()->isReferenceType() && "Can't have a scalar return unless the return type is a " @@ -4040,7 +4280,8 @@ LValue CodeGenFunction::EmitObjCMessageExprLValue(const ObjCMessageExpr *E) { LValue CodeGenFunction::EmitObjCSelectorLValue(const ObjCSelectorExpr *E) { Address V = CGM.getObjCRuntime().GetAddrOfSelector(*this, E->getSelector()); - return MakeAddrLValue(V, E->getType(), AlignmentSource::Decl); + return MakeAddrLValue(V, E->getType(), + LValueBaseInfo(AlignmentSource::Decl, false)); } llvm::Value *CodeGenFunction::EmitIvarOffset(const ObjCInterfaceDecl *Interface, @@ -4084,7 +4325,7 @@ LValue CodeGenFunction::EmitStmtExprLValue(const StmtExpr *E) { // Can only get l-value for message expression returning aggregate type RValue RV = EmitAnyExprToTemp(E); return MakeAddrLValue(RV.getAggregateAddress(), E->getType(), - AlignmentSource::Decl); + LValueBaseInfo(AlignmentSource::Decl, false)); } RValue CodeGenFunction::EmitCall(QualType CalleeType, const CGCallee &OrigCallee, @@ -4276,12 +4517,11 @@ EmitPointerToDataMemberBinaryExpr(const BinaryOperator *E) { const MemberPointerType *MPT = E->getRHS()->getType()->getAs<MemberPointerType>(); - AlignmentSource AlignSource; + LValueBaseInfo BaseInfo; Address MemberAddr = - EmitCXXMemberDataPointerAddress(E, BaseAddr, OffsetV, MPT, - &AlignSource); + EmitCXXMemberDataPointerAddress(E, BaseAddr, OffsetV, MPT, &BaseInfo); - return MakeAddrLValue(MemberAddr, MPT->getPointeeType(), AlignSource); + return MakeAddrLValue(MemberAddr, MPT->getPointeeType(), BaseInfo); } /// Given the address of a temporary variable, produce an r-value of @@ -4289,7 +4529,8 @@ EmitPointerToDataMemberBinaryExpr(const BinaryOperator *E) { RValue CodeGenFunction::convertTempToRValue(Address addr, QualType type, SourceLocation loc) { - LValue lvalue = MakeAddrLValue(addr, type, AlignmentSource::Decl); + LValue lvalue = MakeAddrLValue(addr, type, + LValueBaseInfo(AlignmentSource::Decl, false)); switch (getEvaluationKind(type)) { case TEK_Complex: return RValue::getComplex(EmitLoadOfComplex(lvalue, loc)); @@ -4344,9 +4585,9 @@ static LValueOrRValue emitPseudoObjectExpr(CodeGenFunction &CGF, if (ov == resultExpr && ov->isRValue() && !forLValue && CodeGenFunction::hasAggregateEvaluationKind(ov->getType())) { CGF.EmitAggExpr(ov->getSourceExpr(), slot); - + LValueBaseInfo BaseInfo(AlignmentSource::Decl, false); LValue LV = CGF.MakeAddrLValue(slot.getAddress(), ov->getType(), - AlignmentSource::Decl); + BaseInfo); opaqueData = OVMA::bind(CGF, ov, LV); result.RV = slot.asRValue(); |
