diff options
| author | dim <dim@FreeBSD.org> | 2017-09-26 19:56:36 +0000 |
|---|---|---|
| committer | Luiz Souza <luiz@netgate.com> | 2018-02-21 15:12:19 -0300 |
| commit | 1dcd2e8d24b295bc73e513acec2ed1514bb66be4 (patch) | |
| tree | 4bd13a34c251e980e1a6b13584ca1f63b0dfe670 /contrib/llvm/tools/clang/lib/Serialization | |
| parent | f45541ca2a56a1ba1202f94c080b04e96c1fa239 (diff) | |
| download | FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.zip FreeBSD-src-1dcd2e8d24b295bc73e513acec2ed1514bb66be4.tar.gz | |
Merge clang, llvm, lld, lldb, compiler-rt and libc++ 5.0.0 release.
MFC r309126 (by emaste):
Correct lld llvm-tblgen dependency file name
MFC r309169:
Get rid of separate Subversion mergeinfo properties for llvm-dwarfdump
and llvm-lto. The mergeinfo confuses Subversion enormously, and these
directories will just use the mergeinfo for llvm itself.
MFC r312765:
Pull in r276136 from upstream llvm trunk (by Wei Mi):
Use ValueOffsetPair to enhance value reuse during SCEV expansion.
In D12090, the ExprValueMap was added to reuse existing value during
SCEV expansion. However, const folding and sext/zext distribution can
make the reuse still difficult.
A simplified case is: suppose we know S1 expands to V1 in
ExprValueMap, and
S1 = S2 + C_a
S3 = S2 + C_b
where C_a and C_b are different SCEVConstants. Then we'd like to
expand S3 as V1 - C_a + C_b instead of expanding S2 literally. It is
helpful when S2 is a complex SCEV expr and S2 has no entry in
ExprValueMap, which is usually caused by the fact that S3 is
generated from S1 after const folding.
In order to do that, we represent ExprValueMap as a mapping from SCEV
to ValueOffsetPair. We will save both S1->{V1, 0} and S2->{V1, C_a}
into the ExprValueMap when we create SCEV for V1. When S3 is
expanded, it will first expand S2 to V1 - C_a because of S2->{V1,
C_a} in the map, then expand S3 to V1 - C_a + C_b.
Differential Revision: https://reviews.llvm.org/D21313
This should fix assertion failures when building OpenCV >= 3.1.
PR: 215649
MFC r312831:
Revert r312765 for now, since it causes assertions when building
lang/spidermonkey24.
Reported by: antoine
PR: 215649
MFC r316511 (by jhb):
Add an implementation of __ffssi2() derived from __ffsdi2().
Newer versions of GCC include an __ffssi2() symbol in libgcc and the
compiler can emit calls to it in generated code. This is true for at
least GCC 6.2 when compiling world for mips and mips64.
Reviewed by: jmallett, dim
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D10086
MFC r318601 (by adrian):
[libcompiler-rt] add bswapdi2/bswapsi2
This is required for mips gcc 6.3 userland to build/run.
Reviewed by: emaste, dim
Approved by: emaste
Differential Revision: https://reviews.freebsd.org/D10838
MFC r318884 (by emaste):
lldb: map TRAP_CAP to a trace trap
In the absense of a more specific handler for TRAP_CAP (generated by
ENOTCAPABLE or ECAPMODE while in capability mode) treat it as a trace
trap.
Example usage (testing the bug in PR219173):
% proccontrol -m trapcap lldb usr.bin/hexdump/obj/hexdump -- -Cv -s 1 /bin/ls
...
(lldb) run
Process 12980 launching
Process 12980 launched: '.../usr.bin/hexdump/obj/hexdump' (x86_64)
Process 12980 stopped
* thread #1, stop reason = trace
frame #0: 0x0000004b80c65f1a libc.so.7`__sys_lseek + 10
...
In the future we should have LLDB control the trapcap procctl itself
(as it does with ASLR), as well as report a specific stop reason.
This change eliminates an assertion failure from LLDB for now.
MFC r319796:
Remove a few unneeded files from libllvm, libclang and liblldb.
MFC r319885 (by emaste):
lld: ELF: Fix ICF crash on absolute symbol relocations.
If two sections contained relocations to absolute symbols with the same
value we would crash when trying to access their sections. Add a check that
both symbols point to sections before accessing their sections, and treat
absolute symbols as equal if their values are equal.
Obtained from: LLD commit r292578
MFC r319918:
Revert r319796 for now, it can cause undefined references when linking
in some circumstances.
Reported by: Shawn Webb <shawn.webb@hardenedbsd.org>
MFC r319957 (by emaste):
lld: Add armelf emulation mode
Obtained from: LLD r305375
MFC r321369:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 (trunk r308421). Upstream has branched for the 5.0.0 release,
which should be in about a month. Please report bugs and regressions,
so we can get them into the release.
Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.
MFC r321420:
Add a few more object files to liblldb, which should solve errors when
linking the lldb executable in some cases. In particular, when the
-ffunction-sections -fdata-sections options are turned off, or
ineffective.
Reported by: Shawn Webb, Mark Millard
MFC r321433:
Cleanup stale Options.inc files from the previous libllvm build for
clang 4.0.0. Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.
Reported by: Mark Millard
MFC r321439 (by bdrewery):
Move llvm Options.inc hack from r321433 for NO_CLEAN to lib/clang/libllvm.
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation. So using
a beforebuild target here removes the file before the compilation
tries to include it.
MFC r321664:
Pull in r308891 from upstream llvm trunk (by Benjamin Kramer):
[CodeGenPrepare] Cut off FindAllMemoryUses if there are too many uses.
This avoids excessive compile time. The case I'm looking at is
Function.cpp from an old version of LLVM that still had the giant
memcmp string matcher in it. Before r308322 this compiled in about 2
minutes, after it, clang takes infinite* time to compile it. With
this patch we're at 5 min, which is still bad but this is a
pathological case.
The cut off at 20 uses was chosen by looking at other cut-offs in LLVM
for user scanning. It's probably too high, but does the job and is
very unlikely to regress anything.
Fixes PR33900.
* I'm impatient and aborted after 15 minutes, on the bug report it was
killed after 2h.
Pull in r308986 from upstream llvm trunk (by Simon Pilgrim):
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914)
D35067/rL308322 attempted to support up to 4 load pairs for memcmp
inlining which resulted in regressions for some optimized libc memcmp
implementations (PR33914).
Until we can match these more optimal cases, this patch reduces the
memcmp expansion to a maximum of 2 load pairs (which matches what we
do for -Os).
This patch should be considered for the 5.0.0 release branch as well
Differential Revision: https://reviews.llvm.org/D35830
These fix a hang (or extremely long compile time) when building older
LLVM ports.
Reported by: antoine
PR: 219139
MFC r321719:
Pull in r309503 from upstream clang trunk (by Richard Smith):
PR33902: Invalidate line number cache when adding more text to
existing buffer.
This led to crashes as the line number cache would report a bogus
line number for a line of code, and we'd try to find a nonexistent
column within the line when printing diagnostics.
This fixes an assertion when building the graphics/champlain port.
Reported by: antoine, kwm
PR: 219139
MFC r321723:
Upgrade our copies of clang, llvm, lld and lldb to r309439 from the
upstream release_50 branch. This is just after upstream's 5.0.0-rc1.
MFC r322320:
Upgrade our copies of clang, llvm and libc++ to r310316 from the
upstream release_50 branch.
MFC r322326 (by emaste):
lldb: Make i386-*-freebsd expression work on JIT path
* Enable i386 ABI creation for freebsd
* Added an extra argument in ABISysV_i386::PrepareTrivialCall for mmap
syscall
* Unlike linux, the last argument of mmap is actually 64-bit(off_t).
This requires us to push an additional word for the higher order bits.
* Prior to this change, ktrace dump will show mmap failures due to
invalid argument coming from the 6th mmap argument.
Submitted by: Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D34776
MFC r322360 (by emaste):
lldb: Report inferior signals as signals, not exceptions, on FreeBSD
This is the FreeBSD equivalent of LLVM r238549.
This serves 2 purposes:
* LLDB should handle inferior process signals SIGSEGV/SIGILL/SIGBUS/
SIGFPE the way it is suppose to be handled. Prior to this fix these
signals will neither create a coredump, nor exit from the debugger
or work for signal handling scenario.
* eInvalidCrashReason need not report "unknown crash reason" if we have
a valid si_signo
llvm.org/pr23699
Patch by Karnajit Wangkhem
Differential Revision: https://reviews.llvm.org/D35223
Submitted by: Karnajit Wangkhem
Obtained from: LLVM r310591
MFC r322474 (by emaste):
lld: Add `-z muldefs` option.
Obtained from: LLVM r310757
MFC r322740:
Upgrade our copies of clang, llvm, lld and libc++ to r311219 from the
upstream release_50 branch.
MFC r322855:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r311606 from
the upstream release_50 branch.
As of this version, lib/msun's trig test should also work correctly
again (see bug 220989 for more information).
PR: 220989
MFC r323112:
Upgrade our copies of clang, llvm, lldb and compiler-rt to r312293 from
the upstream release_50 branch. This corresponds to 5.0.0 rc4.
As of this version, the cad/stepcode port should now compile in a more
reasonable time on i386 (see bug 221836 for more information).
PR: 221836
MFC r323245:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
5.0.0 release (upstream r312559).
Release notes for llvm, clang and lld will be available here soon:
<http://releases.llvm.org/5.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/5.0.0/tools/lld/docs/ReleaseNotes.html>
Relnotes: yes
(cherry picked from commit 12cd91cf4c6b96a24427c0de5374916f2808d263)
Diffstat (limited to 'contrib/llvm/tools/clang/lib/Serialization')
11 files changed, 2762 insertions, 847 deletions
diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTCommon.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTCommon.cpp index ecd249c..684ec24 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTCommon.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTCommon.cpp @@ -147,9 +147,6 @@ serialization::TypeIdxFromBuiltin(const BuiltinType *BT) { case BuiltinType::OCLQueue: ID = PREDEF_TYPE_QUEUE_ID; break; - case BuiltinType::OCLNDRange: - ID = PREDEF_TYPE_NDRANGE_ID; - break; case BuiltinType::OCLReserveID: ID = PREDEF_TYPE_RESERVE_ID_ID; break; @@ -254,6 +251,7 @@ bool serialization::isRedeclarableDeclKind(unsigned Kind) { case Decl::VarTemplateSpecialization: case Decl::VarTemplatePartialSpecialization: case Decl::Function: + case Decl::CXXDeductionGuide: case Decl::CXXMethod: case Decl::CXXConstructor: case Decl::CXXDestructor: diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTReader.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTReader.cpp index 53224e2..50be74f 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTReader.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTReader.cpp @@ -26,6 +26,7 @@ #include "clang/AST/Expr.h" #include "clang/AST/ExprCXX.h" #include "clang/AST/NestedNameSpecifier.h" +#include "clang/AST/ODRHash.h" #include "clang/AST/RawCommentList.h" #include "clang/AST/Type.h" #include "clang/AST/TypeLocVisitor.h" @@ -36,6 +37,7 @@ #include "clang/Basic/FileManager.h" #include "clang/Basic/FileSystemOptions.h" #include "clang/Basic/LangOptions.h" +#include "clang/Basic/MemoryBufferCache.h" #include "clang/Basic/ObjCRuntime.h" #include "clang/Basic/OperatorKinds.h" #include "clang/Basic/Sanitizers.h" @@ -72,6 +74,7 @@ #include "llvm/Bitcode/BitstreamReader.h" #include "llvm/Support/Compression.h" #include "llvm/Support/Compiler.h" +#include "llvm/Support/Error.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/FileSystem.h" #include "llvm/Support/MemoryBuffer.h" @@ -289,6 +292,33 @@ static bool checkLanguageOptions(const LangOptions &LangOpts, return true; } + // Sanitizer feature mismatches are treated as compatible differences. If + // compatible differences aren't allowed, we still only want to check for + // mismatches of non-modular sanitizers (the only ones which can affect AST + // generation). + if (!AllowCompatibleDifferences) { + SanitizerMask ModularSanitizers = getPPTransparentSanitizers(); + SanitizerSet ExistingSanitizers = ExistingLangOpts.Sanitize; + SanitizerSet ImportedSanitizers = LangOpts.Sanitize; + ExistingSanitizers.clear(ModularSanitizers); + ImportedSanitizers.clear(ModularSanitizers); + if (ExistingSanitizers.Mask != ImportedSanitizers.Mask) { + const std::string Flag = "-fsanitize="; + if (Diags) { +#define SANITIZER(NAME, ID) \ + { \ + bool InExistingModule = ExistingSanitizers.has(SanitizerKind::ID); \ + bool InImportedModule = ImportedSanitizers.has(SanitizerKind::ID); \ + if (InExistingModule != InImportedModule) \ + Diags->Report(diag::err_pch_targetopt_feature_mismatch) \ + << InExistingModule << (Flag + NAME); \ + } +#include "clang/Basic/Sanitizers.def" + } + return true; + } + } + return false; } @@ -461,19 +491,9 @@ static bool checkDiagnosticMappings(DiagnosticsEngine &StoredDiags, return checkDiagnosticGroupMappings(StoredDiags, Diags, Complain); } -bool PCHValidator::ReadDiagnosticOptions( - IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts, bool Complain) { - DiagnosticsEngine &ExistingDiags = PP.getDiagnostics(); - IntrusiveRefCntPtr<DiagnosticIDs> DiagIDs(ExistingDiags.getDiagnosticIDs()); - IntrusiveRefCntPtr<DiagnosticsEngine> Diags( - new DiagnosticsEngine(DiagIDs, DiagOpts.get())); - // This should never fail, because we would have processed these options - // before writing them to an ASTFile. - ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false); - - ModuleManager &ModuleMgr = Reader.getModuleManager(); - assert(ModuleMgr.size() >= 1 && "what ASTFile is this then"); - +/// Return the top import module if it is implicit, nullptr otherwise. +static Module *getTopImportImplicitModule(ModuleManager &ModuleMgr, + Preprocessor &PP) { // If the original import came from a file explicitly generated by the user, // don't check the diagnostic mappings. // FIXME: currently this is approximated by checking whether this is not a @@ -481,21 +501,41 @@ bool PCHValidator::ReadDiagnosticOptions( // Note: ModuleMgr.rbegin() may not be the current module, but it must be in // the transitive closure of its imports, since unrelated modules cannot be // imported until after this module finishes validation. - ModuleFile *TopImport = *ModuleMgr.rbegin(); + ModuleFile *TopImport = &*ModuleMgr.rbegin(); while (!TopImport->ImportedBy.empty()) TopImport = TopImport->ImportedBy[0]; if (TopImport->Kind != MK_ImplicitModule) - return false; + return nullptr; StringRef ModuleName = TopImport->ModuleName; assert(!ModuleName.empty() && "diagnostic options read before module name"); Module *M = PP.getHeaderSearchInfo().lookupModule(ModuleName); assert(M && "missing module"); + return M; +} + +bool PCHValidator::ReadDiagnosticOptions( + IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts, bool Complain) { + DiagnosticsEngine &ExistingDiags = PP.getDiagnostics(); + IntrusiveRefCntPtr<DiagnosticIDs> DiagIDs(ExistingDiags.getDiagnosticIDs()); + IntrusiveRefCntPtr<DiagnosticsEngine> Diags( + new DiagnosticsEngine(DiagIDs, DiagOpts.get())); + // This should never fail, because we would have processed these options + // before writing them to an ASTFile. + ProcessWarningOptions(*Diags, *DiagOpts, /*Report*/false); + + ModuleManager &ModuleMgr = Reader.getModuleManager(); + assert(ModuleMgr.size() >= 1 && "what ASTFile is this then"); + + Module *TopM = getTopImportImplicitModule(ModuleMgr, PP); + if (!TopM) + return false; // FIXME: if the diagnostics are incompatible, save a DiagnosticOptions that // contains the union of their flags. - return checkDiagnosticMappings(*Diags, ExistingDiags, M->IsSystem, Complain); + return checkDiagnosticMappings(*Diags, ExistingDiags, TopM->IsSystem, + Complain); } /// \brief Collect the macro definitions provided by the given preprocessor @@ -816,7 +856,7 @@ static bool isInterestingIdentifier(ASTReader &Reader, IdentifierInfo &II, II.isPoisoned() || (IsModule ? II.hasRevertedBuiltin() : II.getObjCOrBuiltinID()) || II.hasRevertedTokenIDToIdentifier() || - (!(IsModule && Reader.getContext().getLangOpts().CPlusPlus) && + (!(IsModule && Reader.getPreprocessor().getLangOpts().CPlusPlus) && II.getFETokenInfo<void>()); } @@ -943,6 +983,10 @@ DeclarationNameKey::DeclarationNameKey(DeclarationName Name) case DeclarationName::CXXLiteralOperatorName: Data = (uint64_t)Name.getCXXLiteralIdentifier(); break; + case DeclarationName::CXXDeductionGuideName: + Data = (uint64_t)Name.getCXXDeductionGuideTemplate() + ->getDeclName().getAsIdentifierInfo(); + break; case DeclarationName::CXXConstructorName: case DeclarationName::CXXDestructorName: case DeclarationName::CXXConversionFunctionName: @@ -959,6 +1003,7 @@ unsigned DeclarationNameKey::getHash() const { switch (Kind) { case DeclarationName::Identifier: case DeclarationName::CXXLiteralOperatorName: + case DeclarationName::CXXDeductionGuideName: ID.AddString(((IdentifierInfo*)Data)->getName()); break; case DeclarationName::ObjCZeroArgSelector: @@ -1002,6 +1047,8 @@ ASTDeclContextNameLookupTrait::ReadKey(const unsigned char *d, unsigned) { uint64_t Data; switch (Kind) { case DeclarationName::Identifier: + case DeclarationName::CXXLiteralOperatorName: + case DeclarationName::CXXDeductionGuideName: Data = (uint64_t)Reader.getLocalIdentifier( F, endian::readNext<uint32_t, little, unaligned>(d)); break; @@ -1016,10 +1063,6 @@ ASTDeclContextNameLookupTrait::ReadKey(const unsigned char *d, unsigned) { case DeclarationName::CXXOperatorName: Data = *d++; // OverloadedOperatorKind break; - case DeclarationName::CXXLiteralOperatorName: - Data = (uint64_t)Reader.getLocalIdentifier( - F, endian::readNext<uint32_t, little, unaligned>(d)); - break; case DeclarationName::CXXConstructorName: case DeclarationName::CXXDestructorName: case DeclarationName::CXXConversionFunctionName: @@ -1103,9 +1146,9 @@ bool ASTReader::ReadVisibleDeclContextStorage(ModuleFile &M, return false; } -void ASTReader::Error(StringRef Msg) { +void ASTReader::Error(StringRef Msg) const { Error(diag::err_fe_pch_malformed, Msg); - if (Context.getLangOpts().Modules && !Diags.isDiagnosticInFlight() && + if (PP.getLangOpts().Modules && !Diags.isDiagnosticInFlight() && !PP.getHeaderSearchInfo().getModuleCachePath().empty()) { Diag(diag::note_module_cache_path) << PP.getHeaderSearchInfo().getModuleCachePath(); @@ -1113,7 +1156,7 @@ void ASTReader::Error(StringRef Msg) { } void ASTReader::Error(unsigned DiagID, - StringRef Arg1, StringRef Arg2) { + StringRef Arg1, StringRef Arg2) const { if (Diags.isDiagnosticInFlight()) Diags.SetDelayedDiagnostic(DiagID, Arg1, Arg2); else @@ -1278,10 +1321,15 @@ bool ASTReader::ReadSLocEntry(int ID) { unsigned RecCode = SLocEntryCursor.readRecord(Code, Record, &Blob); if (RecCode == SM_SLOC_BUFFER_BLOB_COMPRESSED) { + if (!llvm::zlib::isAvailable()) { + Error("zlib is not available"); + return nullptr; + } SmallString<0> Uncompressed; - if (llvm::zlib::uncompress(Blob, Uncompressed, Record[0]) != - llvm::zlib::StatusOK) { - Error("could not decompress embedded file contents"); + if (llvm::Error E = + llvm::zlib::uncompress(Blob, Uncompressed, Record[0])) { + Error("could not decompress embedded file contents: " + + llvm::toString(std::move(E))); return nullptr; } return llvm::MemoryBuffer::getMemBufferCopy(Uncompressed, Name); @@ -1343,15 +1391,14 @@ bool ASTReader::ReadSLocEntry(int ID) { const DeclID *FirstDecl = F->FileSortedDecls + Record[6]; unsigned NumFileDecls = Record[7]; - if (NumFileDecls) { + if (NumFileDecls && ContextObj) { assert(F->FileSortedDecls && "FILE_SORTED_DECLS not encountered yet ?"); FileDeclIDs[FID] = FileDeclsInfo(F, llvm::makeArrayRef(FirstDecl, NumFileDecls)); } const SrcMgr::ContentCache *ContentCache - = SourceMgr.getOrCreateContentCache(File, - /*isSystemFile=*/FileCharacter != SrcMgr::C_User); + = SourceMgr.getOrCreateContentCache(File, isSystem(FileCharacter)); if (OverriddenBuffer && !ContentCache->BufferOverridden && ContentCache->ContentsEntry == ContentCache->OrigEntry && !ContentCache->getRawBuffer()) { @@ -1473,7 +1520,7 @@ MacroInfo *ASTReader::ReadMacroRecord(ModuleFile &F, uint64_t Offset) { Stream.JumpToBit(Offset); RecordData Record; - SmallVector<IdentifierInfo*, 16> MacroArgs; + SmallVector<IdentifierInfo*, 16> MacroParams; MacroInfo *Macro = nullptr; while (true) { @@ -1513,9 +1560,8 @@ MacroInfo *ASTReader::ReadMacroRecord(ModuleFile &F, uint64_t Offset) { return Macro; unsigned NextIndex = 1; // Skip identifier ID. - SubmoduleID SubModID = getGlobalSubmoduleID(F, Record[NextIndex++]); SourceLocation Loc = ReadSourceLocation(F, Record, NextIndex); - MacroInfo *MI = PP.AllocateDeserializedMacroInfo(Loc, SubModID); + MacroInfo *MI = PP.AllocateMacroInfo(Loc); MI->setDefinitionEndLoc(ReadSourceLocation(F, Record, NextIndex)); MI->setIsUsed(Record[NextIndex++]); MI->setUsedForHeaderGuard(Record[NextIndex++]); @@ -1525,17 +1571,17 @@ MacroInfo *ASTReader::ReadMacroRecord(ModuleFile &F, uint64_t Offset) { bool isC99VarArgs = Record[NextIndex++]; bool isGNUVarArgs = Record[NextIndex++]; bool hasCommaPasting = Record[NextIndex++]; - MacroArgs.clear(); + MacroParams.clear(); unsigned NumArgs = Record[NextIndex++]; for (unsigned i = 0; i != NumArgs; ++i) - MacroArgs.push_back(getLocalIdentifier(F, Record[NextIndex++])); + MacroParams.push_back(getLocalIdentifier(F, Record[NextIndex++])); // Install function-like macro info. MI->setIsFunctionLike(); if (isC99VarArgs) MI->setIsC99Varargs(); if (isGNUVarArgs) MI->setIsGNUVarargs(); if (hasCommaPasting) MI->setHasCommaPasting(); - MI->setArgumentList(MacroArgs, PP.getPreprocessorAllocator()); + MI->setParameterList(MacroParams, PP.getPreprocessorAllocator()); } // Remember that we saw this macro last so that we add the tokens that @@ -1575,7 +1621,11 @@ MacroInfo *ASTReader::ReadMacroRecord(ModuleFile &F, uint64_t Offset) { } PreprocessedEntityID -ASTReader::getGlobalPreprocessedEntityID(ModuleFile &M, unsigned LocalID) const { +ASTReader::getGlobalPreprocessedEntityID(ModuleFile &M, + unsigned LocalID) const { + if (!M.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(M); + ContinuousRangeMap<uint32_t, int, 2>::const_iterator I = M.PreprocessedEntityRemap.find(LocalID - NUM_PREDEF_PP_ENTITY_IDS); assert(I != M.PreprocessedEntityRemap.end() @@ -1646,9 +1696,9 @@ HeaderFileInfoTrait::ReadData(internal_key_ref key, const unsigned char *d, HeaderFileInfo HFI; unsigned Flags = *d++; // FIXME: Refactor with mergeHeaderFileInfo in HeaderSearch.cpp. - HFI.isImport |= (Flags >> 4) & 0x01; - HFI.isPragmaOnce |= (Flags >> 3) & 0x01; - HFI.DirInfo = (Flags >> 1) & 0x03; + HFI.isImport |= (Flags >> 5) & 0x01; + HFI.isPragmaOnce |= (Flags >> 4) & 0x01; + HFI.DirInfo = (Flags >> 1) & 0x07; HFI.IndexHeaderMapHeader = Flags & 0x01; // FIXME: Find a better way to handle this. Maybe just store a // "has been included" flag? @@ -1707,15 +1757,15 @@ void ASTReader::ReadDefinedMacros() { // Note that we are loading defined macros. Deserializing Macros(this); - for (auto &I : llvm::reverse(ModuleMgr)) { - BitstreamCursor &MacroCursor = I->MacroCursor; + for (ModuleFile &I : llvm::reverse(ModuleMgr)) { + BitstreamCursor &MacroCursor = I.MacroCursor; // If there was no preprocessor block, skip this file. if (MacroCursor.getBitcodeBytes().empty()) continue; BitstreamCursor Cursor = MacroCursor; - Cursor.JumpToBit(I->MacroStartOffset); + Cursor.JumpToBit(I.MacroStartOffset); RecordData Record; while (true) { @@ -1737,7 +1787,7 @@ void ASTReader::ReadDefinedMacros() { case PP_MACRO_OBJECT_LIKE: case PP_MACRO_FUNCTION_LIKE: { - IdentifierInfo *II = getLocalIdentifier(*I, Record[0]); + IdentifierInfo *II = getLocalIdentifier(I, Record[0]); if (II->isOutOfDate()) updateOutOfDateIdentifier(*II); break; @@ -1978,6 +2028,7 @@ ASTReader::readInputFileInfo(ModuleFile &F, unsigned ID) { R.StoredTime = static_cast<time_t>(Record[2]); R.Overridden = static_cast<bool>(Record[3]); R.Transient = static_cast<bool>(Record[4]); + R.TopLevelModuleMap = static_cast<bool>(Record[5]); R.Filename = Blob; ResolveImportedPath(F, R.Filename); return R; @@ -2149,7 +2200,7 @@ static bool isDiagnosedResult(ASTReader::ASTReadResult ARR, unsigned Caps) { ASTReader::ASTReadResult ASTReader::ReadOptionsBlock( BitstreamCursor &Stream, unsigned ClientLoadCapabilities, bool AllowCompatibleConfigurationMismatch, ASTReaderListener &Listener, - std::string &SuggestedPredefines, bool ValidateDiagnosticOptions) { + std::string &SuggestedPredefines) { if (Stream.EnterSubBlock(OPTIONS_BLOCK_ID)) return Failure; @@ -2191,15 +2242,6 @@ ASTReader::ASTReadResult ASTReader::ReadOptionsBlock( break; } - case DIAGNOSTIC_OPTIONS: { - bool Complain = (ClientLoadCapabilities & ARR_OutOfDate) == 0; - if (ValidateDiagnosticOptions && - !AllowCompatibleConfigurationMismatch && - ParseDiagnosticOptions(Record, Complain, Listener)) - return OutOfDate; - break; - } - case FILE_SYSTEM_OPTIONS: { bool Complain = (ClientLoadCapabilities & ARR_ConfigurationMismatch) == 0; if (!AllowCompatibleConfigurationMismatch && @@ -2240,6 +2282,23 @@ ASTReader::ReadControlBlock(ModuleFile &F, return Failure; } + // Lambda to read the unhashed control block the first time it's called. + // + // For PCM files, the unhashed control block cannot be read until after the + // MODULE_NAME record. However, PCH files have no MODULE_NAME, and yet still + // need to look ahead before reading the IMPORTS record. For consistency, + // this block is always read somehow (see BitstreamEntry::EndBlock). + bool HasReadUnhashedControlBlock = false; + auto readUnhashedControlBlockOnce = [&]() { + if (!HasReadUnhashedControlBlock) { + HasReadUnhashedControlBlock = true; + if (ASTReadResult Result = + readUnhashedControlBlock(F, ImportedBy, ClientLoadCapabilities)) + return Result; + } + return Success; + }; + // Read all of the records and blocks in the control block. RecordData Record; unsigned NumInputs = 0; @@ -2252,6 +2311,11 @@ ASTReader::ReadControlBlock(ModuleFile &F, Error("malformed block record in AST file"); return Failure; case llvm::BitstreamEntry::EndBlock: { + // Validate the module before returning. This call catches an AST with + // no module name and no imports. + if (ASTReadResult Result = readUnhashedControlBlockOnce()) + return Result; + // Validate input files. const HeaderSearchOptions &HSOpts = PP.getHeaderSearchInfo().getHeaderSearchOpts(); @@ -2323,13 +2387,10 @@ ASTReader::ReadControlBlock(ModuleFile &F, // FIXME: Allow this for files explicitly specified with -include-pch. bool AllowCompatibleConfigurationMismatch = F.Kind == MK_ExplicitModule || F.Kind == MK_PrebuiltModule; - const HeaderSearchOptions &HSOpts = - PP.getHeaderSearchInfo().getHeaderSearchOpts(); Result = ReadOptionsBlock(Stream, ClientLoadCapabilities, AllowCompatibleConfigurationMismatch, - *Listener, SuggestedPredefines, - HSOpts.ModulesValidateDiagnosticOptions); + *Listener, SuggestedPredefines); if (Result == Failure) { Error("malformed block record in AST file"); return Result; @@ -2403,12 +2464,13 @@ ASTReader::ReadControlBlock(ModuleFile &F, break; } - case SIGNATURE: - assert((!F.Signature || F.Signature == Record[0]) && "signature changed"); - F.Signature = Record[0]; - break; - case IMPORTS: { + // Validate the AST before processing any imports (otherwise, untangling + // them can be error-prone and expensive). A module will have a name and + // will already have been validated, but this catches the PCH case. + if (ASTReadResult Result = readUnhashedControlBlockOnce()) + return Result; + // Load each of the imported PCH files. unsigned Idx = 0, N = Record.size(); while (Idx < N) { @@ -2421,7 +2483,10 @@ ASTReader::ReadControlBlock(ModuleFile &F, ReadUntranslatedSourceLocation(Record[Idx++]); off_t StoredSize = (off_t)Record[Idx++]; time_t StoredModTime = (time_t)Record[Idx++]; - ASTFileSignature StoredSignature = Record[Idx++]; + ASTFileSignature StoredSignature = { + {{(uint32_t)Record[Idx++], (uint32_t)Record[Idx++], + (uint32_t)Record[Idx++], (uint32_t)Record[Idx++], + (uint32_t)Record[Idx++]}}}; auto ImportedFile = ReadPath(F, Record, Idx); // If our client can't cope with us being out of date, we can't cope with @@ -2473,6 +2538,12 @@ ASTReader::ReadControlBlock(ModuleFile &F, F.ModuleName = Blob; if (Listener) Listener->ReadModuleName(F.ModuleName); + + // Validate the AST as soon as we have a name so we can exit early on + // failure. + if (ASTReadResult Result = readUnhashedControlBlockOnce()) + return Result; + break; case MODULE_DIRECTORY: { @@ -2513,6 +2584,7 @@ ASTReader::ReadControlBlock(ModuleFile &F, F.InputFileOffsets = (const llvm::support::unaligned_uint64_t *)Blob.data(); F.InputFilesLoaded.resize(NumInputs); + F.NumUserInputFiles = NumUserInputs; break; } } @@ -2542,10 +2614,11 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { // contains any declarations lexically within it (which it always does!). // This usually has no cost, since we very rarely need the lookup map for // the translation unit outside C++. - DeclContext *DC = Context.getTranslationUnitDecl(); - if (DC->hasExternalLexicalStorage() && - !getContext().getLangOpts().CPlusPlus) - DC->setMustBuildLookupTable(); + if (ASTContext *Ctx = ContextObj) { + DeclContext *DC = Ctx->getTranslationUnitDecl(); + if (DC->hasExternalLexicalStorage() && !Ctx->getLangOpts().CPlusPlus) + DC->setMustBuildLookupTable(); + } return Success; } @@ -2601,7 +2674,8 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { break; case SUBMODULE_BLOCK_ID: - if (ASTReadResult Result = ReadSubmoduleBlock(F, ClientLoadCapabilities)) + if (ASTReadResult Result = + ReadSubmoduleBlock(F, ClientLoadCapabilities)) return Result; break; @@ -2633,7 +2707,33 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { // Read and process a record. Record.clear(); StringRef Blob; - switch ((ASTRecordTypes)Stream.readRecord(Entry.ID, Record, &Blob)) { + auto RecordType = + (ASTRecordTypes)Stream.readRecord(Entry.ID, Record, &Blob); + + // If we're not loading an AST context, we don't care about most records. + if (!ContextObj) { + switch (RecordType) { + case IDENTIFIER_TABLE: + case IDENTIFIER_OFFSET: + case INTERESTING_IDENTIFIERS: + case STATISTICS: + case PP_CONDITIONAL_STACK: + case PP_COUNTER_VALUE: + case SOURCE_LOCATION_OFFSETS: + case MODULE_OFFSET_MAP: + case SOURCE_MANAGER_LINE_TABLE: + case SOURCE_LOCATION_PRELOADS: + case PPD_ENTITIES_OFFSETS: + case HEADER_SEARCH_TABLE: + case IMPORTED_MODULES: + case MACRO_OFFSET: + break; + default: + continue; + } + } + + switch (RecordType) { default: // Default behavior: ignore. break; @@ -2692,7 +2792,7 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { } case TU_UPDATE_LEXICAL: { - DeclContext *TU = Context.getTranslationUnitDecl(); + DeclContext *TU = ContextObj->getTranslationUnitDecl(); LexicalContents Contents( reinterpret_cast<const llvm::support::unaligned_uint32_t *>( Blob.data()), @@ -2710,7 +2810,8 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { // If we've already loaded the decl, perform the updates when we finish // loading this block. if (Decl *D = GetExistingDecl(ID)) - PendingUpdateRecords.push_back(std::make_pair(ID, D)); + PendingUpdateRecords.push_back( + PendingUpdateRecord(ID, D, /*JustLoaded=*/false)); break; } @@ -2766,6 +2867,14 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { EagerlyDeserializedDecls.push_back(getGlobalDeclID(F, Record[I])); break; + case MODULAR_CODEGEN_DECLS: + // FIXME: Skip reading this record if our ASTConsumer doesn't care about + // them (ie: if we're not codegenerating this module). + if (F.Kind == MK_MainFile) + for (unsigned I = 0, N = Record.size(); I != N; ++I) + EagerlyDeserializedDecls.push_back(getGlobalDeclID(F, Record[I])); + break; + case SPECIAL_TYPES: if (SpecialTypes.empty()) { for (unsigned I = 0, N = Record.size(); I != N; ++I) @@ -2870,6 +2979,21 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { } break; + case PP_CONDITIONAL_STACK: + if (!Record.empty()) { + SmallVector<PPConditionalInfo, 4> ConditionalStack; + for (unsigned Idx = 0, N = Record.size() - 1; Idx < N; /* in loop */) { + auto Loc = ReadSourceLocation(F, Record, Idx); + bool WasSkipping = Record[Idx++]; + bool FoundNonSkip = Record[Idx++]; + bool FoundElse = Record[Idx++]; + ConditionalStack.push_back( + {Loc, WasSkipping, FoundNonSkip, FoundElse}); + } + PP.setReplayablePreambleConditionalStack(ConditionalStack); + } + break; + case PP_COUNTER_VALUE: if (!Record.empty() && Listener) Listener->ReadCounter(F, Record[0]); @@ -2916,80 +3040,9 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { break; } - case MODULE_OFFSET_MAP: { - // Additional remapping information. - const unsigned char *Data = (const unsigned char*)Blob.data(); - const unsigned char *DataEnd = Data + Blob.size(); - - // If we see this entry before SOURCE_LOCATION_OFFSETS, add placeholders. - if (F.SLocRemap.find(0) == F.SLocRemap.end()) { - F.SLocRemap.insert(std::make_pair(0U, 0)); - F.SLocRemap.insert(std::make_pair(2U, 1)); - } - - // Continuous range maps we may be updating in our module. - typedef ContinuousRangeMap<uint32_t, int, 2>::Builder - RemapBuilder; - RemapBuilder SLocRemap(F.SLocRemap); - RemapBuilder IdentifierRemap(F.IdentifierRemap); - RemapBuilder MacroRemap(F.MacroRemap); - RemapBuilder PreprocessedEntityRemap(F.PreprocessedEntityRemap); - RemapBuilder SubmoduleRemap(F.SubmoduleRemap); - RemapBuilder SelectorRemap(F.SelectorRemap); - RemapBuilder DeclRemap(F.DeclRemap); - RemapBuilder TypeRemap(F.TypeRemap); - - while (Data < DataEnd) { - using namespace llvm::support; - uint16_t Len = endian::readNext<uint16_t, little, unaligned>(Data); - StringRef Name = StringRef((const char*)Data, Len); - Data += Len; - ModuleFile *OM = ModuleMgr.lookup(Name); - if (!OM) { - Error("SourceLocation remap refers to unknown module"); - return Failure; - } - - uint32_t SLocOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t IdentifierIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t MacroIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t PreprocessedEntityIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t SubmoduleIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t SelectorIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t DeclIDOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - uint32_t TypeIndexOffset = - endian::readNext<uint32_t, little, unaligned>(Data); - - uint32_t None = std::numeric_limits<uint32_t>::max(); - - auto mapOffset = [&](uint32_t Offset, uint32_t BaseOffset, - RemapBuilder &Remap) { - if (Offset != None) - Remap.insert(std::make_pair(Offset, - static_cast<int>(BaseOffset - Offset))); - }; - mapOffset(SLocOffset, OM->SLocEntryBaseOffset, SLocRemap); - mapOffset(IdentifierIDOffset, OM->BaseIdentifierID, IdentifierRemap); - mapOffset(MacroIDOffset, OM->BaseMacroID, MacroRemap); - mapOffset(PreprocessedEntityIDOffset, OM->BasePreprocessedEntityID, - PreprocessedEntityRemap); - mapOffset(SubmoduleIDOffset, OM->BaseSubmoduleID, SubmoduleRemap); - mapOffset(SelectorIDOffset, OM->BaseSelectorID, SelectorRemap); - mapOffset(DeclIDOffset, OM->BaseDeclID, DeclRemap); - mapOffset(TypeIndexOffset, OM->BaseTypeIndex, TypeRemap); - - // Global -> local mappings. - F.GlobalToLocalDeclIDs[OM] = DeclIDOffset; - } + case MODULE_OFFSET_MAP: + F.ModuleOffsetMap = Blob; break; - } case SOURCE_MANAGER_LINE_TABLE: if (ParseLineTable(F, Record)) @@ -3103,7 +3156,8 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { // If we've already loaded the decl, perform the updates when we finish // loading this block. if (Decl *D = GetExistingDecl(ID)) - PendingUpdateRecords.push_back(std::make_pair(ID, D)); + PendingUpdateRecords.push_back( + PendingUpdateRecord(ID, D, /*JustLoaded=*/false)); } break; } @@ -3123,14 +3177,6 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { F.ObjCCategories.swap(Record); break; - case DIAG_PRAGMA_MAPPINGS: - if (F.PragmaDiagMappings.empty()) - F.PragmaDiagMappings.swap(Record); - else - F.PragmaDiagMappings.insert(F.PragmaDiagMappings.end(), - Record.begin(), Record.end()); - break; - case CUDA_SPECIAL_DECL_REFS: // Later tables overwrite earlier ones. // FIXME: Modules will have trouble with this. @@ -3245,8 +3291,11 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { for (unsigned I = 0, N = Record.size(); I != N; /**/) { unsigned GlobalID = getGlobalSubmoduleID(F, Record[I++]); SourceLocation Loc = ReadSourceLocation(F, Record, I); - if (GlobalID) + if (GlobalID) { ImportedModules.push_back(ImportedSubmodule(GlobalID, Loc)); + if (DeserializationListener) + DeserializationListener->ModuleImportRead(GlobalID, Loc); + } } } break; @@ -3319,7 +3368,110 @@ ASTReader::ReadASTBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { } ForceCUDAHostDeviceDepth = Record[0]; break; + + case PACK_PRAGMA_OPTIONS: { + if (Record.size() < 3) { + Error("invalid pragma pack record"); + return Failure; + } + PragmaPackCurrentValue = Record[0]; + PragmaPackCurrentLocation = ReadSourceLocation(F, Record[1]); + unsigned NumStackEntries = Record[2]; + unsigned Idx = 3; + // Reset the stack when importing a new module. + PragmaPackStack.clear(); + for (unsigned I = 0; I < NumStackEntries; ++I) { + PragmaPackStackEntry Entry; + Entry.Value = Record[Idx++]; + Entry.Location = ReadSourceLocation(F, Record[Idx++]); + PragmaPackStrings.push_back(ReadString(Record, Idx)); + Entry.SlotLabel = PragmaPackStrings.back(); + PragmaPackStack.push_back(Entry); + } + break; + } + } + } +} + +void ASTReader::ReadModuleOffsetMap(ModuleFile &F) const { + assert(!F.ModuleOffsetMap.empty() && "no module offset map to read"); + + // Additional remapping information. + const unsigned char *Data = (const unsigned char*)F.ModuleOffsetMap.data(); + const unsigned char *DataEnd = Data + F.ModuleOffsetMap.size(); + F.ModuleOffsetMap = StringRef(); + + // If we see this entry before SOURCE_LOCATION_OFFSETS, add placeholders. + if (F.SLocRemap.find(0) == F.SLocRemap.end()) { + F.SLocRemap.insert(std::make_pair(0U, 0)); + F.SLocRemap.insert(std::make_pair(2U, 1)); + } + + // Continuous range maps we may be updating in our module. + typedef ContinuousRangeMap<uint32_t, int, 2>::Builder + RemapBuilder; + RemapBuilder SLocRemap(F.SLocRemap); + RemapBuilder IdentifierRemap(F.IdentifierRemap); + RemapBuilder MacroRemap(F.MacroRemap); + RemapBuilder PreprocessedEntityRemap(F.PreprocessedEntityRemap); + RemapBuilder SubmoduleRemap(F.SubmoduleRemap); + RemapBuilder SelectorRemap(F.SelectorRemap); + RemapBuilder DeclRemap(F.DeclRemap); + RemapBuilder TypeRemap(F.TypeRemap); + + while (Data < DataEnd) { + // FIXME: Looking up dependency modules by filename is horrible. + using namespace llvm::support; + uint16_t Len = endian::readNext<uint16_t, little, unaligned>(Data); + StringRef Name = StringRef((const char*)Data, Len); + Data += Len; + ModuleFile *OM = ModuleMgr.lookup(Name); + if (!OM) { + std::string Msg = + "SourceLocation remap refers to unknown module, cannot find "; + Msg.append(Name); + Error(Msg); + return; } + + uint32_t SLocOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t IdentifierIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t MacroIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t PreprocessedEntityIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t SubmoduleIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t SelectorIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t DeclIDOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + uint32_t TypeIndexOffset = + endian::readNext<uint32_t, little, unaligned>(Data); + + uint32_t None = std::numeric_limits<uint32_t>::max(); + + auto mapOffset = [&](uint32_t Offset, uint32_t BaseOffset, + RemapBuilder &Remap) { + if (Offset != None) + Remap.insert(std::make_pair(Offset, + static_cast<int>(BaseOffset - Offset))); + }; + mapOffset(SLocOffset, OM->SLocEntryBaseOffset, SLocRemap); + mapOffset(IdentifierIDOffset, OM->BaseIdentifierID, IdentifierRemap); + mapOffset(MacroIDOffset, OM->BaseMacroID, MacroRemap); + mapOffset(PreprocessedEntityIDOffset, OM->BasePreprocessedEntityID, + PreprocessedEntityRemap); + mapOffset(SubmoduleIDOffset, OM->BaseSubmoduleID, SubmoduleRemap); + mapOffset(SelectorIDOffset, OM->BaseSelectorID, SelectorRemap); + mapOffset(DeclIDOffset, OM->BaseDeclID, DeclRemap); + mapOffset(TypeIndexOffset, OM->BaseTypeIndex, TypeRemap); + + // Global -> local mappings. + F.GlobalToLocalDeclIDs[OM] = DeclIDOffset; } } @@ -3330,20 +3482,13 @@ ASTReader::ReadModuleMapFileBlock(RecordData &Record, ModuleFile &F, unsigned Idx = 0; F.ModuleMapPath = ReadPath(F, Record, Idx); - if (F.Kind == MK_ExplicitModule || F.Kind == MK_PrebuiltModule) { - // For an explicitly-loaded module, we don't care whether the original - // module map file exists or matches. - return Success; - } - // Try to resolve ModuleName in the current header search context and // verify that it is found in the same module map file as we saved. If the // top-level AST file is a main file, skip this check because there is no // usable header search context. assert(!F.ModuleName.empty() && "MODULE_NAME should come before MODULE_MAP_FILE"); - if (F.Kind == MK_ImplicitModule && - (*ModuleMgr.begin())->Kind != MK_MainFile) { + if (F.Kind == MK_ImplicitModule && ModuleMgr.begin()->Kind != MK_MainFile) { // An implicitly-loaded module file should have its module listed in some // module map file that we've already loaded. Module *M = PP.getHeaderSearchInfo().lookupModule(F.ModuleName); @@ -3456,8 +3601,8 @@ static void moveMethodToBackOfGlobalList(Sema &S, ObjCMethodDecl *Method) { void ASTReader::makeNamesVisible(const HiddenNames &Names, Module *Owner) { assert(Owner->NameVisibility != Module::Hidden && "nothing to make visible?"); for (Decl *D : Names) { - bool wasHidden = D->Hidden; - D->Hidden = false; + bool wasHidden = D->isHidden(); + D->setVisibleDespiteOwningModule(); if (wasHidden && SemaObj) { if (ObjCMethodDecl *Method = dyn_cast<ObjCMethodDecl>(D)) { @@ -3524,7 +3669,7 @@ void ASTReader::mergeDefinitionVisibility(NamedDecl *Def, if (Def->isHidden()) { // If MergedDef is visible or becomes visible, make the definition visible. if (!MergedDef->isHidden()) - Def->Hidden = false; + Def->setVisibleDespiteOwningModule(); else if (getContext().getLangOpts().ModulesLocalVisibility) { getContext().mergeDefinitionIntoModule( Def, MergedDef->getImportedOwningModule(), @@ -3543,7 +3688,7 @@ bool ASTReader::loadGlobalIndex() { return false; if (TriedLoadingGlobalIndex || !UseGlobalIndex || - !Context.getLangOpts().Modules) + !PP.getLangOpts().Modules) return true; // Try to load the global index. @@ -3561,7 +3706,7 @@ bool ASTReader::loadGlobalIndex() { } bool ASTReader::isGlobalIndexUnavailable() const { - return Context.getLangOpts().Modules && UseGlobalIndex && + return PP.getLangOpts().Modules && UseGlobalIndex && !hasGlobalIndex() && TriedLoadingGlobalIndex; } @@ -3573,6 +3718,8 @@ static void updateModuleTimestamp(ModuleFile &MF) { if (EC) return; OS << "Timestamp file\n"; + OS.close(); + OS.clear_error(); // Avoid triggering a fatal error. } /// \brief Given a cursor at the start of an AST file, scan ahead and drop the @@ -3617,14 +3764,16 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, Deserializing AnASTFile(this); // Bump the generation number. - unsigned PreviousGeneration = incrementGeneration(Context); + unsigned PreviousGeneration = 0; + if (ContextObj) + PreviousGeneration = incrementGeneration(*ContextObj); unsigned NumModules = ModuleMgr.size(); SmallVector<ImportedModule, 4> Loaded; - switch(ASTReadResult ReadResult = ReadASTCore(FileName, Type, ImportLoc, - /*ImportedBy=*/nullptr, Loaded, - 0, 0, 0, - ClientLoadCapabilities)) { + switch (ASTReadResult ReadResult = + ReadASTCore(FileName, Type, ImportLoc, + /*ImportedBy=*/nullptr, Loaded, 0, 0, + ASTFileSignature(), ClientLoadCapabilities)) { case Failure: case Missing: case OutOfDate: @@ -3635,11 +3784,10 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, for (const ImportedModule &IM : Loaded) LoadedSet.insert(IM.Mod); - ModuleMgr.removeModules(ModuleMgr.begin() + NumModules, ModuleMgr.end(), - LoadedSet, - Context.getLangOpts().Modules - ? &PP.getHeaderSearchInfo().getModuleMap() - : nullptr); + ModuleMgr.removeModules(ModuleMgr.begin() + NumModules, LoadedSet, + PP.getLangOpts().Modules + ? &PP.getHeaderSearchInfo().getModuleMap() + : nullptr); // If we find that any modules are unusable, the global index is going // to be out-of-date. Just remove it. @@ -3685,6 +3833,13 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, SourceMgr.getLoadedSLocEntryByID(Index); } + // Map the original source file ID into the ID space of the current + // compilation. + if (F.OriginalSourceFileID.isValid()) { + F.OriginalSourceFileID = FileID::get( + F.SLocEntryBaseID + F.OriginalSourceFileID.getOpaqueValue() - 1); + } + // Preload all the pending interesting identifiers by marking them out of // date. for (auto Offset : F.PreloadIdentifierOffsets) { @@ -3726,7 +3881,7 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, F.ImportLoc = TranslateSourceLocation(*M->ImportedBy, M->ImportLoc); } - if (!Context.getLangOpts().CPlusPlus || + if (!PP.getLangOpts().CPlusPlus || (Type != MK_ImplicitModule && Type != MK_ExplicitModule && Type != MK_PrebuiltModule)) { // Mark all of the identifiers in the identifier table as being out of date, @@ -3783,7 +3938,8 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, // Might be unnecessary as use declarations are only used to build the // module itself. - InitializeContext(); + if (ContextObj) + InitializeContext(); if (SemaObj) UpdateSema(); @@ -3793,10 +3949,6 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, ModuleFile &PrimaryModule = ModuleMgr.getPrimaryModule(); if (PrimaryModule.OriginalSourceFileID.isValid()) { - PrimaryModule.OriginalSourceFileID - = FileID::get(PrimaryModule.SLocEntryBaseID - + PrimaryModule.OriginalSourceFileID.getOpaqueValue() - 1); - // If this AST file is a precompiled preamble, then set the // preamble file ID of the source manager to the file source file // from which the preamble was built. @@ -3809,10 +3961,12 @@ ASTReader::ASTReadResult ASTReader::ReadAST(StringRef FileName, // For any Objective-C class definitions we have already loaded, make sure // that we load any additional categories. - for (unsigned I = 0, N = ObjCClassesLoaded.size(); I != N; ++I) { - loadObjCCategories(ObjCClassesLoaded[I]->getGlobalID(), - ObjCClassesLoaded[I], - PreviousGeneration); + if (ContextObj) { + for (unsigned I = 0, N = ObjCClassesLoaded.size(); I != N; ++I) { + loadObjCCategories(ObjCClassesLoaded[I]->getGlobalID(), + ObjCClassesLoaded[I], + PreviousGeneration); + } } if (PP.getHeaderSearchInfo() @@ -3986,6 +4140,12 @@ ASTReader::ReadASTCore(StringRef FileName, Loaded.push_back(ImportedModule(M, ImportedBy, ImportLoc)); return Success; + case UNHASHED_CONTROL_BLOCK_ID: + // This block is handled using look-ahead during ReadControlBlock. We + // shouldn't get here! + Error("malformed block record in AST file"); + return Failure; + default: if (Stream.SkipBlock()) { Error("malformed block record in AST file"); @@ -3998,6 +4158,122 @@ ASTReader::ReadASTCore(StringRef FileName, return Success; } +ASTReader::ASTReadResult +ASTReader::readUnhashedControlBlock(ModuleFile &F, bool WasImportedBy, + unsigned ClientLoadCapabilities) { + const HeaderSearchOptions &HSOpts = + PP.getHeaderSearchInfo().getHeaderSearchOpts(); + bool AllowCompatibleConfigurationMismatch = + F.Kind == MK_ExplicitModule || F.Kind == MK_PrebuiltModule; + + ASTReadResult Result = readUnhashedControlBlockImpl( + &F, F.Data, ClientLoadCapabilities, AllowCompatibleConfigurationMismatch, + Listener.get(), + WasImportedBy ? false : HSOpts.ModulesValidateDiagnosticOptions); + + // If F was directly imported by another module, it's implicitly validated by + // the importing module. + if (DisableValidation || WasImportedBy || + (AllowConfigurationMismatch && Result == ConfigurationMismatch)) + return Success; + + if (Result == Failure) { + Error("malformed block record in AST file"); + return Failure; + } + + if (Result == OutOfDate && F.Kind == MK_ImplicitModule) { + // If this module has already been finalized in the PCMCache, we're stuck + // with it; we can only load a single version of each module. + // + // This can happen when a module is imported in two contexts: in one, as a + // user module; in another, as a system module (due to an import from + // another module marked with the [system] flag). It usually indicates a + // bug in the module map: this module should also be marked with [system]. + // + // If -Wno-system-headers (the default), and the first import is as a + // system module, then validation will fail during the as-user import, + // since -Werror flags won't have been validated. However, it's reasonable + // to treat this consistently as a system module. + // + // If -Wsystem-headers, the PCM on disk was built with + // -Wno-system-headers, and the first import is as a user module, then + // validation will fail during the as-system import since the PCM on disk + // doesn't guarantee that -Werror was respected. However, the -Werror + // flags were checked during the initial as-user import. + if (PCMCache.isBufferFinal(F.FileName)) { + Diag(diag::warn_module_system_bit_conflict) << F.FileName; + return Success; + } + } + + return Result; +} + +ASTReader::ASTReadResult ASTReader::readUnhashedControlBlockImpl( + ModuleFile *F, llvm::StringRef StreamData, unsigned ClientLoadCapabilities, + bool AllowCompatibleConfigurationMismatch, ASTReaderListener *Listener, + bool ValidateDiagnosticOptions) { + // Initialize a stream. + BitstreamCursor Stream(StreamData); + + // Sniff for the signature. + if (!startsWithASTFileMagic(Stream)) + return Failure; + + // Scan for the UNHASHED_CONTROL_BLOCK_ID block. + if (SkipCursorToBlock(Stream, UNHASHED_CONTROL_BLOCK_ID)) + return Failure; + + // Read all of the records in the options block. + RecordData Record; + ASTReadResult Result = Success; + while (1) { + llvm::BitstreamEntry Entry = Stream.advance(); + + switch (Entry.Kind) { + case llvm::BitstreamEntry::Error: + case llvm::BitstreamEntry::SubBlock: + return Failure; + + case llvm::BitstreamEntry::EndBlock: + return Result; + + case llvm::BitstreamEntry::Record: + // The interesting case. + break; + } + + // Read and process a record. + Record.clear(); + switch ( + (UnhashedControlBlockRecordTypes)Stream.readRecord(Entry.ID, Record)) { + case SIGNATURE: { + if (F) + std::copy(Record.begin(), Record.end(), F->Signature.data()); + break; + } + case DIAGNOSTIC_OPTIONS: { + bool Complain = (ClientLoadCapabilities & ARR_OutOfDate) == 0; + if (Listener && ValidateDiagnosticOptions && + !AllowCompatibleConfigurationMismatch && + ParseDiagnosticOptions(Record, Complain, *Listener)) + Result = OutOfDate; // Don't return early. Read the signature. + break; + } + case DIAG_PRAGMA_MAPPINGS: + if (!F) + break; + if (F->PragmaDiagMappings.empty()) + F->PragmaDiagMappings.swap(Record); + else + F->PragmaDiagMappings.insert(F->PragmaDiagMappings.end(), + Record.begin(), Record.end()); + break; + } + } +} + /// Parse a record and blob containing module file extension metadata. static bool parseModuleFileExtensionMetadata( const SmallVectorImpl<uint64_t> &Record, @@ -4070,6 +4346,9 @@ ASTReader::ASTReadResult ASTReader::ReadExtensionBlock(ModuleFile &F) { } void ASTReader::InitializeContext() { + assert(ContextObj && "no context to initialize"); + ASTContext &Context = *ContextObj; + // If there's a listener, notify them that we "read" the translation unit. if (DeserializationListener) DeserializationListener->DeclRead(PREDEF_DECL_TRANSLATION_UNIT_ID, @@ -4214,23 +4493,24 @@ void ASTReader::finalizeForWriting() { static ASTFileSignature readASTFileSignature(StringRef PCH) { BitstreamCursor Stream(PCH); if (!startsWithASTFileMagic(Stream)) - return 0; + return ASTFileSignature(); - // Scan for the CONTROL_BLOCK_ID block. - if (SkipCursorToBlock(Stream, CONTROL_BLOCK_ID)) - return 0; + // Scan for the UNHASHED_CONTROL_BLOCK_ID block. + if (SkipCursorToBlock(Stream, UNHASHED_CONTROL_BLOCK_ID)) + return ASTFileSignature(); - // Scan for SIGNATURE inside the control block. + // Scan for SIGNATURE inside the diagnostic options block. ASTReader::RecordData Record; while (true) { llvm::BitstreamEntry Entry = Stream.advanceSkippingSubblocks(); if (Entry.Kind != llvm::BitstreamEntry::Record) - return 0; + return ASTFileSignature(); Record.clear(); StringRef Blob; if (SIGNATURE == Stream.readRecord(Entry.ID, Record, &Blob)) - return Record[0]; + return {{{(uint32_t)Record[0], (uint32_t)Record[1], (uint32_t)Record[2], + (uint32_t)Record[3], (uint32_t)Record[4]}}}; } } @@ -4349,7 +4629,8 @@ bool ASTReader::readASTFileControlBlock( } // Initialize the stream - BitstreamCursor Stream(PCHContainerRdr.ExtractPCH(**Buffer)); + StringRef Bytes = PCHContainerRdr.ExtractPCH(**Buffer); + BitstreamCursor Stream(Bytes); // Sniff for the signature. if (!startsWithASTFileMagic(Stream)) @@ -4377,8 +4658,7 @@ bool ASTReader::readASTFileControlBlock( std::string IgnoredSuggestedPredefines; if (ReadOptionsBlock(Stream, ARR_ConfigurationMismatch | ARR_OutOfDate, /*AllowCompatibleConfigurationMismatch*/ false, - Listener, IgnoredSuggestedPredefines, - ValidateDiagnosticOptions) != Success) + Listener, IgnoredSuggestedPredefines) != Success) return true; break; } @@ -4499,6 +4779,7 @@ bool ASTReader::readASTFileControlBlock( // Look for module file extension blocks, if requested. if (FindModuleFileExtensions) { + BitstreamCursor SavedStream = Stream; while (!SkipCursorToBlock(Stream, EXTENSION_BLOCK_ID)) { bool DoneWithExtensionBlock = false; while (!DoneWithExtensionBlock) { @@ -4537,16 +4818,25 @@ bool ASTReader::readASTFileControlBlock( } } } + Stream = SavedStream; } + // Scan for the UNHASHED_CONTROL_BLOCK_ID block. + if (readUnhashedControlBlockImpl( + nullptr, Bytes, ARR_ConfigurationMismatch | ARR_OutOfDate, + /*AllowCompatibleConfigurationMismatch*/ false, &Listener, + ValidateDiagnosticOptions) != Success) + return true; + return false; } -bool ASTReader::isAcceptableASTFile( - StringRef Filename, FileManager &FileMgr, - const PCHContainerReader &PCHContainerRdr, const LangOptions &LangOpts, - const TargetOptions &TargetOpts, const PreprocessorOptions &PPOpts, - std::string ExistingModuleCachePath) { +bool ASTReader::isAcceptableASTFile(StringRef Filename, FileManager &FileMgr, + const PCHContainerReader &PCHContainerRdr, + const LangOptions &LangOpts, + const TargetOptions &TargetOpts, + const PreprocessorOptions &PPOpts, + StringRef ExistingModuleCachePath) { SimplePCHValidator validator(LangOpts, TargetOpts, PPOpts, ExistingModuleCachePath, FileMgr); return !readASTFileControlBlock(Filename, FileMgr, PCHContainerRdr, @@ -4566,6 +4856,7 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { ModuleMap &ModMap = PP.getHeaderSearchInfo().getModuleMap(); bool First = true; Module *CurrentModule = nullptr; + Module::ModuleKind ModuleKind = Module::ModuleMapModule; RecordData Record; while (true) { llvm::BitstreamEntry Entry = F.Stream.advanceSkippingSubblocks(); @@ -4628,8 +4919,9 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { // Retrieve this (sub)module from the module map, creating it if // necessary. - CurrentModule = ModMap.findOrCreateModule(Name, ParentModule, IsFramework, - IsExplicit).first; + CurrentModule = + ModMap.findOrCreateModule(Name, ParentModule, IsFramework, IsExplicit) + .first; // FIXME: set the definition loc for CurrentModule, or call // ModMap.setInferredModuleAllowedBy() @@ -4655,8 +4947,10 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { } CurrentModule->setASTFile(F.File); + CurrentModule->PresumedModuleMapFile = F.ModuleMapPath; } + CurrentModule->Kind = ModuleKind; CurrentModule->Signature = F.Signature; CurrentModule->IsFromModuleFile = true; CurrentModule->IsSystem = IsSystem || CurrentModule->IsSystem; @@ -4696,13 +4990,9 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { if (!CurrentModule->getUmbrellaHeader()) ModMap.setUmbrellaHeader(CurrentModule, Umbrella, Blob); else if (CurrentModule->getUmbrellaHeader().Entry != Umbrella) { - // This can be a spurious difference caused by changing the VFS to - // point to a different copy of the file, and it is too late to - // to rebuild safely. - // FIXME: If we wrote the virtual paths instead of the 'real' paths, - // after input file validation only real problems would remain and we - // could just error. For now, assume it's okay. - break; + if ((ClientLoadCapabilities & ARR_OutOfDate) == 0) + Error("mismatched umbrella headers in submodule"); + return OutOfDate; } } break; @@ -4759,6 +5049,7 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { SubmodulesLoaded.resize(SubmodulesLoaded.size() + F.LocalNumSubmodules); } + ModuleKind = (Module::ModuleKind)Record[2]; break; } @@ -4792,8 +5083,8 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { break; } case SUBMODULE_REQUIRES: { - CurrentModule->addRequirement(Blob, Record[0], Context.getLangOpts(), - Context.getTargetInfo()); + CurrentModule->addRequirement(Blob, Record[0], PP.getLangOpts(), + PP.getTargetInfo()); break; } @@ -4819,10 +5110,12 @@ ASTReader::ReadSubmoduleBlock(ModuleFile &F, unsigned ClientLoadCapabilities) { } case SUBMODULE_INITIALIZERS: + if (!ContextObj) + break; SmallVector<uint32_t, 16> Inits; for (auto &ID : Record) Inits.push_back(getGlobalDeclID(F, ID)); - Context.addLazyModuleInitializers(CurrentModule, Inits); + ContextObj->addLazyModuleInitializers(CurrentModule, Inits); break; } } @@ -4950,6 +5243,8 @@ bool ASTReader::ParseHeaderSearchOptions(const RecordData &Record, HSOpts.ModuleCachePath = ReadString(Record, Idx); HSOpts.ModuleUserBuildPath = ReadString(Record, Idx); HSOpts.DisableModuleHash = Record[Idx++]; + HSOpts.ImplicitModuleMaps = Record[Idx++]; + HSOpts.ModuleMapFileHomeIsCwd = Record[Idx++]; HSOpts.UseBuiltinIncludes = Record[Idx++]; HSOpts.UseStandardSystemIncludes = Record[Idx++]; HSOpts.UseStandardCXXIncludes = Record[Idx++]; @@ -5288,48 +5583,141 @@ HeaderFileInfo ASTReader::GetHeaderFileInfo(const FileEntry *FE) { } void ASTReader::ReadPragmaDiagnosticMappings(DiagnosticsEngine &Diag) { - // FIXME: Make it work properly with modules. - SmallVector<DiagnosticsEngine::DiagState *, 32> DiagStates; - for (ModuleIterator I = ModuleMgr.begin(), E = ModuleMgr.end(); I != E; ++I) { - ModuleFile &F = *(*I); + using DiagState = DiagnosticsEngine::DiagState; + SmallVector<DiagState *, 32> DiagStates; + + for (ModuleFile &F : ModuleMgr) { unsigned Idx = 0; + auto &Record = F.PragmaDiagMappings; + if (Record.empty()) + continue; + DiagStates.clear(); - assert(!Diag.DiagStates.empty()); - DiagStates.push_back(&Diag.DiagStates.front()); // the command-line one. - while (Idx < F.PragmaDiagMappings.size()) { - SourceLocation Loc = ReadSourceLocation(F, F.PragmaDiagMappings[Idx++]); - unsigned DiagStateID = F.PragmaDiagMappings[Idx++]; - if (DiagStateID != 0) { - Diag.DiagStatePoints.push_back( - DiagnosticsEngine::DiagStatePoint(DiagStates[DiagStateID-1], - FullSourceLoc(Loc, SourceMgr))); - continue; - } - assert(DiagStateID == 0); + auto ReadDiagState = + [&](const DiagState &BasedOn, SourceLocation Loc, + bool IncludeNonPragmaStates) -> DiagnosticsEngine::DiagState * { + unsigned BackrefID = Record[Idx++]; + if (BackrefID != 0) + return DiagStates[BackrefID - 1]; + // A new DiagState was created here. - Diag.DiagStates.push_back(*Diag.GetCurDiagState()); - DiagnosticsEngine::DiagState *NewState = &Diag.DiagStates.back(); + Diag.DiagStates.push_back(BasedOn); + DiagState *NewState = &Diag.DiagStates.back(); DiagStates.push_back(NewState); - Diag.DiagStatePoints.push_back( - DiagnosticsEngine::DiagStatePoint(NewState, - FullSourceLoc(Loc, SourceMgr))); - while (true) { - assert(Idx < F.PragmaDiagMappings.size() && - "Invalid data, didn't find '-1' marking end of diag/map pairs"); - if (Idx >= F.PragmaDiagMappings.size()) { - break; // Something is messed up but at least avoid infinite loop in - // release build. - } - unsigned DiagID = F.PragmaDiagMappings[Idx++]; - if (DiagID == (unsigned)-1) { - break; // no more diag/map pairs for this location. + unsigned Size = Record[Idx++]; + assert(Idx + Size * 2 <= Record.size() && + "Invalid data, not enough diag/map pairs"); + while (Size--) { + unsigned DiagID = Record[Idx++]; + DiagnosticMapping NewMapping = + DiagnosticMapping::deserialize(Record[Idx++]); + if (!NewMapping.isPragma() && !IncludeNonPragmaStates) + continue; + + DiagnosticMapping &Mapping = NewState->getOrAddMapping(DiagID); + + // If this mapping was specified as a warning but the severity was + // upgraded due to diagnostic settings, simulate the current diagnostic + // settings (and use a warning). + if (NewMapping.wasUpgradedFromWarning() && !Mapping.isErrorOrFatal()) { + NewMapping.setSeverity(diag::Severity::Warning); + NewMapping.setUpgradedFromWarning(false); } - diag::Severity Map = (diag::Severity)F.PragmaDiagMappings[Idx++]; - DiagnosticMapping Mapping = Diag.makeUserMapping(Map, Loc); - Diag.GetCurDiagState()->setMapping(DiagID, Mapping); + + Mapping = NewMapping; + } + return NewState; + }; + + // Read the first state. + DiagState *FirstState; + if (F.Kind == MK_ImplicitModule) { + // Implicitly-built modules are reused with different diagnostic + // settings. Use the initial diagnostic state from Diag to simulate this + // compilation's diagnostic settings. + FirstState = Diag.DiagStatesByLoc.FirstDiagState; + DiagStates.push_back(FirstState); + + // Skip the initial diagnostic state from the serialized module. + assert(Record[1] == 0 && + "Invalid data, unexpected backref in initial state"); + Idx = 3 + Record[2] * 2; + assert(Idx < Record.size() && + "Invalid data, not enough state change pairs in initial state"); + } else if (F.isModule()) { + // For an explicit module, preserve the flags from the module build + // command line (-w, -Weverything, -Werror, ...) along with any explicit + // -Wblah flags. + unsigned Flags = Record[Idx++]; + DiagState Initial; + Initial.SuppressSystemWarnings = Flags & 1; Flags >>= 1; + Initial.ErrorsAsFatal = Flags & 1; Flags >>= 1; + Initial.WarningsAsErrors = Flags & 1; Flags >>= 1; + Initial.EnableAllWarnings = Flags & 1; Flags >>= 1; + Initial.IgnoreAllWarnings = Flags & 1; Flags >>= 1; + Initial.ExtBehavior = (diag::Severity)Flags; + FirstState = ReadDiagState(Initial, SourceLocation(), true); + + // Set up the root buffer of the module to start with the initial + // diagnostic state of the module itself, to cover files that contain no + // explicit transitions (for which we did not serialize anything). + Diag.DiagStatesByLoc.Files[F.OriginalSourceFileID] + .StateTransitions.push_back({FirstState, 0}); + } else { + // For prefix ASTs, start with whatever the user configured on the + // command line. + Idx++; // Skip flags. + FirstState = ReadDiagState(*Diag.DiagStatesByLoc.CurDiagState, + SourceLocation(), false); + } + + // Read the state transitions. + unsigned NumLocations = Record[Idx++]; + while (NumLocations--) { + assert(Idx < Record.size() && + "Invalid data, missing pragma diagnostic states"); + SourceLocation Loc = ReadSourceLocation(F, Record[Idx++]); + auto IDAndOffset = SourceMgr.getDecomposedLoc(Loc); + assert(IDAndOffset.second == 0 && "not a start location for a FileID"); + unsigned Transitions = Record[Idx++]; + + // Note that we don't need to set up Parent/ParentOffset here, because + // we won't be changing the diagnostic state within imported FileIDs + // (other than perhaps appending to the main source file, which has no + // parent). + auto &F = Diag.DiagStatesByLoc.Files[IDAndOffset.first]; + F.StateTransitions.reserve(F.StateTransitions.size() + Transitions); + for (unsigned I = 0; I != Transitions; ++I) { + unsigned Offset = Record[Idx++]; + auto *State = + ReadDiagState(*FirstState, Loc.getLocWithOffset(Offset), false); + F.StateTransitions.push_back({State, Offset}); } } + + // Read the final state. + assert(Idx < Record.size() && + "Invalid data, missing final pragma diagnostic state"); + SourceLocation CurStateLoc = + ReadSourceLocation(F, F.PragmaDiagMappings[Idx++]); + auto *CurState = ReadDiagState(*FirstState, CurStateLoc, false); + + if (!F.isModule()) { + Diag.DiagStatesByLoc.CurDiagState = CurState; + Diag.DiagStatesByLoc.CurDiagStateLoc = CurStateLoc; + + // Preserve the property that the imaginary root file describes the + // current state. + auto &T = Diag.DiagStatesByLoc.Files[FileID()].StateTransitions; + if (T.empty()) + T.push_back({CurState, 0}); + else + T[0].State = CurState; + } + + // Don't try to read these mappings again. + Record.clear(); } } @@ -5348,6 +5736,8 @@ ASTReader::RecordLocation ASTReader::TypeCursorForIndex(unsigned Index) { /// location. It is a helper routine for GetType, which deals with reading type /// IDs. QualType ASTReader::readTypeRecord(unsigned Index) { + assert(ContextObj && "reading type with no AST context"); + ASTContext &Context = *ContextObj; RecordLocation Loc = TypeCursorForIndex(Index); BitstreamCursor &DeclsCursor = Loc.F->DeclsCursor; @@ -5508,13 +5898,13 @@ QualType ASTReader::readTypeRecord(unsigned Index) { } case TYPE_FUNCTION_NO_PROTO: { - if (Record.size() != 6) { + if (Record.size() != 7) { Error("incorrect encoding of no-proto function type"); return QualType(); } QualType ResultType = readType(*Loc.F, Record, Idx); FunctionType::ExtInfo Info(Record[1], Record[2], Record[3], - (CallingConv)Record[4], Record[5]); + (CallingConv)Record[4], Record[5], Record[6]); return Context.getFunctionNoProtoType(ResultType, Info); } @@ -5526,9 +5916,10 @@ QualType ASTReader::readTypeRecord(unsigned Index) { /*hasregparm*/ Record[2], /*regparm*/ Record[3], static_cast<CallingConv>(Record[4]), - /*produces*/ Record[5]); + /*produces*/ Record[5], + /*nocallersavedregs*/ Record[6]); - unsigned Idx = 6; + unsigned Idx = 7; EPI.Variadic = Record[Idx++]; EPI.HasTrailingReturn = Record[Idx++]; @@ -5606,6 +5997,14 @@ QualType ASTReader::readTypeRecord(unsigned Index) { return Context.getAutoType(Deduced, Keyword, IsDependent); } + case TYPE_DEDUCED_TEMPLATE_SPECIALIZATION: { + TemplateName Name = ReadTemplateName(*Loc.F, Record, Idx); + QualType Deduced = readType(*Loc.F, Record, Idx); + bool IsDependent = Deduced.isNull() ? Record[Idx++] : false; + return Context.getDeducedTemplateSpecializationType(Name, Deduced, + IsDependent); + } + case TYPE_RECORD: { if (Record.size() != 2) { Error("incorrect encoding of record type"); @@ -5840,6 +6239,17 @@ QualType ASTReader::readTypeRecord(unsigned Index) { return Context.getPipeType(ElementType, ReadOnly); } + case TYPE_DEPENDENT_SIZED_EXT_VECTOR: { + unsigned Idx = 0; + + // DependentSizedExtVectorType + QualType ElementType = readType(*Loc.F, Record, Idx); + Expr *SizeExpr = ReadExpr(*Loc.F); + SourceLocation AttrLoc = ReadSourceLocation(*Loc.F, Record, Idx); + + return Context.getDependentSizedExtVectorType(ElementType, SizeExpr, + AttrLoc); + } } llvm_unreachable("Invalid TypeCode!"); } @@ -6037,6 +6447,11 @@ void TypeLocReader::VisitAutoTypeLoc(AutoTypeLoc TL) { TL.setNameLoc(ReadSourceLocation()); } +void TypeLocReader::VisitDeducedTemplateSpecializationTypeLoc( + DeducedTemplateSpecializationTypeLoc TL) { + TL.setTemplateNameLoc(ReadSourceLocation()); +} + void TypeLocReader::VisitRecordTypeLoc(RecordTypeLoc TL) { TL.setNameLoc(ReadSourceLocation()); } @@ -6179,6 +6594,9 @@ ASTReader::GetTypeSourceInfo(ModuleFile &F, const ASTReader::RecordData &Record, } QualType ASTReader::GetType(TypeID ID) { + assert(ContextObj && "reading type with no AST context"); + ASTContext &Context = *ContextObj; + unsigned FastQuals = ID & Qualifiers::FastMask; unsigned Index = ID >> Qualifiers::FastWidth; @@ -6304,9 +6722,6 @@ QualType ASTReader::GetType(TypeID ID) { case PREDEF_TYPE_QUEUE_ID: T = Context.OCLQueueTy; break; - case PREDEF_TYPE_NDRANGE_ID: - T = Context.OCLNDRangeTy; - break; case PREDEF_TYPE_RESERVE_ID_ID: T = Context.OCLReserveIDTy; break; @@ -6363,6 +6778,9 @@ ASTReader::getGlobalTypeID(ModuleFile &F, unsigned LocalID) const { if (LocalIndex < NUM_PREDEF_TYPE_IDS) return LocalID; + if (!F.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(F); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = F.TypeRemap.find(LocalIndex - NUM_PREDEF_TYPE_IDS); assert(I != F.TypeRemap.end() && "Invalid index into type index remap"); @@ -6437,12 +6855,6 @@ Decl *ASTReader::GetExternalDecl(uint32_t ID) { return GetDecl(ID); } -template<typename TemplateSpecializationDecl> -static void completeRedeclChainForTemplateSpecialization(Decl *D) { - if (auto *TSD = dyn_cast<TemplateSpecializationDecl>(D)) - TSD->getSpecializedTemplate()->LoadLazySpecializations(); -} - void ASTReader::CompleteRedeclChain(const Decl *D) { if (NumCurrentElementsDeserializing) { // We arrange to not care about the complete redeclaration chain while we're @@ -6516,6 +6928,9 @@ ASTReader::GetExternalCXXCtorInitializers(uint64_t Offset) { } CXXBaseSpecifier *ASTReader::GetExternalCXXBaseSpecifiers(uint64_t Offset) { + assert(ContextObj && "reading base specifiers with no AST context"); + ASTContext &Context = *ContextObj; + RecordLocation Loc = getLocalBitOffset(Offset); BitstreamCursor &Cursor = Loc.F->DeclsCursor; SavedStreamPosition SavedPosition(Cursor); @@ -6543,6 +6958,9 @@ ASTReader::getGlobalDeclID(ModuleFile &F, LocalDeclID LocalID) const { if (LocalID < NUM_PREDEF_DECL_IDS) return LocalID; + if (!F.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(F); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = F.DeclRemap.find(LocalID - NUM_PREDEF_DECL_IDS); assert(I != F.DeclRemap.end() && "Invalid index into decl index remap"); @@ -6644,8 +7062,9 @@ static Decl *getPredefinedDecl(ASTContext &Context, PredefinedDeclIDs ID) { } Decl *ASTReader::GetExistingDecl(DeclID ID) { + assert(ContextObj && "reading decl with no AST context"); if (ID < NUM_PREDEF_DECL_IDS) { - Decl *D = getPredefinedDecl(Context, (PredefinedDeclIDs)ID); + Decl *D = getPredefinedDecl(*ContextObj, (PredefinedDeclIDs)ID); if (D) { // Track that we have merged the declaration with ID \p ID into the // pre-existing predefined declaration \p D. @@ -6728,6 +7147,9 @@ Stmt *ASTReader::GetExternalDeclStmt(uint64_t Offset) { // Offset here is a global offset across the entire chain. RecordLocation Loc = getLocalBitOffset(Offset); Loc.F->DeclsCursor.JumpToBit(Loc.Offset); + assert(NumCurrentElementsDeserializing == 0 && + "should not be called while already deserializing"); + Deserializing D(this); return ReadStmtFromStream(*Loc.F); } @@ -6920,31 +7342,6 @@ static void PassObjCImplDeclToConsumer(ObjCImplDecl *ImplD, Consumer->HandleInterestingDecl(DeclGroupRef(ImplD)); } -void ASTReader::PassInterestingDeclsToConsumer() { - assert(Consumer); - - if (PassingDeclsToConsumer) - return; - - // Guard variable to avoid recursively redoing the process of passing - // decls to consumer. - SaveAndRestore<bool> GuardPassingDeclsToConsumer(PassingDeclsToConsumer, - true); - - // Ensure that we've loaded all potentially-interesting declarations - // that need to be eagerly loaded. - for (auto ID : EagerlyDeserializedDecls) - GetDecl(ID); - EagerlyDeserializedDecls.clear(); - - while (!InterestingDecls.empty()) { - Decl *D = InterestingDecls.front(); - InterestingDecls.pop_front(); - - PassInterestingDeclToConsumer(D); - } -} - void ASTReader::PassInterestingDeclToConsumer(Decl *D) { if (ObjCImplDecl *ImplD = dyn_cast<ObjCImplDecl>(D)) PassObjCImplDeclToConsumer(ImplD, Consumer); @@ -7062,7 +7459,7 @@ void ASTReader::PrintStats() { } template<typename Key, typename ModuleFile, unsigned InitialCapacity> -static void +LLVM_DUMP_METHOD static void dumpModuleIDMap(StringRef Name, const ContinuousRangeMap<Key, ModuleFile *, InitialCapacity> &Map) { @@ -7092,18 +7489,15 @@ LLVM_DUMP_METHOD void ASTReader::dump() { GlobalPreprocessedEntityMap); llvm::errs() << "\n*** PCH/Modules Loaded:"; - for (ModuleManager::ModuleConstIterator M = ModuleMgr.begin(), - MEnd = ModuleMgr.end(); - M != MEnd; ++M) - (*M)->dump(); + for (ModuleFile &M : ModuleMgr) + M.dump(); } /// Return the amount of memory used by memory buffers, breaking down /// by heap-backed versus mmap'ed memory. void ASTReader::getMemoryBufferSizes(MemoryBufferSizes &sizes) const { - for (ModuleConstIterator I = ModuleMgr.begin(), - E = ModuleMgr.end(); I != E; ++I) { - if (llvm::MemoryBuffer *buf = (*I)->Buffer.get()) { + for (ModuleFile &I : ModuleMgr) { + if (llvm::MemoryBuffer *buf = I.Buffer) { size_t bytes = buf->getBufferSize(); switch (buf->getBufferKind()) { case llvm::MemoryBuffer::MemoryBuffer_Malloc: @@ -7132,7 +7526,7 @@ void ASTReader::InitializeSema(Sema &S) { // FIXME: What happens if these are changed by a module import? if (!FPPragmaOptions.empty()) { assert(FPPragmaOptions.size() == 1 && "Wrong number of FP_PRAGMA_OPTIONS"); - SemaObj->FPFeatures.fp_contract = FPPragmaOptions[0]; + SemaObj->FPFeatures = FPOptions(FPPragmaOptions[0]); } SemaObj->OpenCLFeatures.copy(OpenCLExtensions); @@ -7173,6 +7567,34 @@ void ASTReader::UpdateSema() { PointersToMembersPragmaLocation); } SemaObj->ForceCUDAHostDeviceDepth = ForceCUDAHostDeviceDepth; + + if (PragmaPackCurrentValue) { + // The bottom of the stack might have a default value. It must be adjusted + // to the current value to ensure that the packing state is preserved after + // popping entries that were included/imported from a PCH/module. + bool DropFirst = false; + if (!PragmaPackStack.empty() && + PragmaPackStack.front().Location.isInvalid()) { + assert(PragmaPackStack.front().Value == SemaObj->PackStack.DefaultValue && + "Expected a default alignment value"); + SemaObj->PackStack.Stack.emplace_back( + PragmaPackStack.front().SlotLabel, SemaObj->PackStack.CurrentValue, + SemaObj->PackStack.CurrentPragmaLocation); + DropFirst = true; + } + for (const auto &Entry : + llvm::makeArrayRef(PragmaPackStack).drop_front(DropFirst ? 1 : 0)) + SemaObj->PackStack.Stack.emplace_back(Entry.SlotLabel, Entry.Value, + Entry.Location); + if (PragmaPackCurrentLocation.isInvalid()) { + assert(*PragmaPackCurrentValue == SemaObj->PackStack.DefaultValue && + "Expected a default alignment value"); + // Keep the current values. + } else { + SemaObj->PackStack.CurrentValue = *PragmaPackCurrentValue; + SemaObj->PackStack.CurrentPragmaLocation = PragmaPackCurrentLocation; + } + } } IdentifierInfo *ASTReader::get(StringRef Name) { @@ -7187,7 +7609,7 @@ IdentifierInfo *ASTReader::get(StringRef Name) { // all interesting declarations, and don't need to use the scope for name // lookups). Perform the lookup in PCH files, though, since we don't build // a complete initial identifier table if we're carrying on from a PCH. - if (Context.getLangOpts().CPlusPlus) { + if (PP.getLangOpts().CPlusPlus) { for (auto F : ModuleMgr.pch_modules()) if (Visitor(*F)) break; @@ -7720,6 +8142,9 @@ IdentifierID ASTReader::getGlobalIdentifierID(ModuleFile &M, unsigned LocalID) { if (LocalID < NUM_PREDEF_IDENT_IDS) return LocalID; + if (!M.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(M); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = M.IdentifierRemap.find(LocalID - NUM_PREDEF_IDENT_IDS); assert(I != M.IdentifierRemap.end() @@ -7758,6 +8183,9 @@ MacroID ASTReader::getGlobalMacroID(ModuleFile &M, unsigned LocalID) { if (LocalID < NUM_PREDEF_MACRO_IDS) return LocalID; + if (!M.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(M); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = M.MacroRemap.find(LocalID - NUM_PREDEF_MACRO_IDS); assert(I != M.MacroRemap.end() && "Invalid index into macro index remap"); @@ -7770,6 +8198,9 @@ ASTReader::getGlobalSubmoduleID(ModuleFile &M, unsigned LocalID) { if (LocalID < NUM_PREDEF_SUBMODULE_IDS) return LocalID; + if (!M.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(M); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = M.SubmoduleRemap.find(LocalID - NUM_PREDEF_SUBMODULE_IDS); assert(I != M.SubmoduleRemap.end() @@ -7832,8 +8263,9 @@ ASTReader::getSourceDescriptor(unsigned ID) { return ExternalASTSource::ASTSourceDescriptor(*M); // If there is only a single PCH, return it instead. - // Chained PCH are not suported. - if (ModuleMgr.size() == 1) { + // Chained PCH are not supported. + const auto &PCHChain = ModuleMgr.pch_modules(); + if (std::distance(std::begin(PCHChain), std::end(PCHChain))) { ModuleFile &MF = ModuleMgr.getPrimaryModule(); StringRef ModuleName = llvm::sys::path::filename(MF.OriginalSourceFileName); StringRef FileName = llvm::sys::path::filename(MF.FileName); @@ -7843,6 +8275,13 @@ ASTReader::getSourceDescriptor(unsigned ID) { return None; } +ExternalASTSource::ExtKind ASTReader::hasExternalDefinitions(const Decl *FD) { + auto I = BodySource.find(FD); + if (I == BodySource.end()) + return EK_ReplyHazy; + return I->second ? EK_Never : EK_Always; +} + Selector ASTReader::getLocalSelector(ModuleFile &M, unsigned LocalID) { return DecodeSelector(getGlobalSelectorID(M, LocalID)); } @@ -7886,6 +8325,9 @@ ASTReader::getGlobalSelectorID(ModuleFile &M, unsigned LocalID) const { if (LocalID < NUM_PREDEF_SELECTOR_IDS) return LocalID; + if (!M.ModuleOffsetMap.empty()) + ReadModuleOffsetMap(M); + ContinuousRangeMap<uint32_t, int, 2>::iterator I = M.SelectorRemap.find(LocalID - NUM_PREDEF_SELECTOR_IDS); assert(I != M.SelectorRemap.end() @@ -7897,6 +8339,7 @@ ASTReader::getGlobalSelectorID(ModuleFile &M, unsigned LocalID) const { DeclarationName ASTReader::ReadDeclarationName(ModuleFile &F, const RecordData &Record, unsigned &Idx) { + ASTContext &Context = getContext(); DeclarationName::NameKind Kind = (DeclarationName::NameKind)Record[Idx++]; switch (Kind) { case DeclarationName::Identifier: @@ -7915,6 +8358,10 @@ ASTReader::ReadDeclarationName(ModuleFile &F, return Context.DeclarationNames.getCXXDestructorName( Context.getCanonicalType(readType(F, Record, Idx))); + case DeclarationName::CXXDeductionGuideName: + return Context.DeclarationNames.getCXXDeductionGuideName( + ReadDeclAs<TemplateDecl>(F, Record, Idx)); + case DeclarationName::CXXConversionFunctionName: return Context.DeclarationNames.getCXXConversionFunctionName( Context.getCanonicalType(readType(F, Record, Idx))); @@ -7962,6 +8409,7 @@ void ASTReader::ReadDeclarationNameLoc(ModuleFile &F, case DeclarationName::ObjCOneArgSelector: case DeclarationName::ObjCMultiArgSelector: case DeclarationName::CXXUsingDirective: + case DeclarationName::CXXDeductionGuideName: break; } } @@ -7982,7 +8430,8 @@ void ASTReader::ReadQualifierInfo(ModuleFile &F, QualifierInfo &Info, unsigned NumTPLists = Record[Idx++]; Info.NumTemplParamLists = NumTPLists; if (NumTPLists) { - Info.TemplParamLists = new (Context) TemplateParameterList*[NumTPLists]; + Info.TemplParamLists = + new (getContext()) TemplateParameterList *[NumTPLists]; for (unsigned i = 0; i != NumTPLists; ++i) Info.TemplParamLists[i] = ReadTemplateParameterList(F, Record, Idx); } @@ -7991,6 +8440,7 @@ void ASTReader::ReadQualifierInfo(ModuleFile &F, QualifierInfo &Info, TemplateName ASTReader::ReadTemplateName(ModuleFile &F, const RecordData &Record, unsigned &Idx) { + ASTContext &Context = getContext(); TemplateName::NameKind Kind = (TemplateName::NameKind)Record[Idx++]; switch (Kind) { case TemplateName::Template: @@ -8051,6 +8501,7 @@ TemplateArgument ASTReader::ReadTemplateArgument(ModuleFile &F, const RecordData &Record, unsigned &Idx, bool Canonicalize) { + ASTContext &Context = getContext(); if (Canonicalize) { // The caller wants a canonical template argument. Sometimes the AST only // wants template arguments in canonical form (particularly as the template @@ -8114,9 +8565,8 @@ ASTReader::ReadTemplateParameterList(ModuleFile &F, Params.push_back(ReadDeclAs<NamedDecl>(F, Record, Idx)); // TODO: Concepts - TemplateParameterList* TemplateParams = - TemplateParameterList::Create(Context, TemplateLoc, LAngleLoc, - Params, RAngleLoc, nullptr); + TemplateParameterList *TemplateParams = TemplateParameterList::Create( + getContext(), TemplateLoc, LAngleLoc, Params, RAngleLoc, nullptr); return TemplateParams; } @@ -8135,11 +8585,11 @@ ReadTemplateArgumentList(SmallVectorImpl<TemplateArgument> &TemplArgs, void ASTReader::ReadUnresolvedSet(ModuleFile &F, LazyASTUnresolvedSet &Set, const RecordData &Record, unsigned &Idx) { unsigned NumDecls = Record[Idx++]; - Set.reserve(Context, NumDecls); + Set.reserve(getContext(), NumDecls); while (NumDecls--) { DeclID ID = ReadDeclID(F, Record, Idx); AccessSpecifier AS = (AccessSpecifier)Record[Idx++]; - Set.addLazyDecl(Context, ID, AS); + Set.addLazyDecl(getContext(), ID, AS); } } @@ -8162,6 +8612,7 @@ ASTReader::ReadCXXBaseSpecifier(ModuleFile &F, CXXCtorInitializer ** ASTReader::ReadCXXCtorInitializers(ModuleFile &F, const RecordData &Record, unsigned &Idx) { + ASTContext &Context = getContext(); unsigned NumInitializers = Record[Idx++]; assert(NumInitializers && "wrote ctor initializers but have no inits"); auto **CtorInitializers = new (Context) CXXCtorInitializer*[NumInitializers]; @@ -8227,6 +8678,7 @@ ASTReader::ReadCXXCtorInitializers(ModuleFile &F, const RecordData &Record, NestedNameSpecifier * ASTReader::ReadNestedNameSpecifier(ModuleFile &F, const RecordData &Record, unsigned &Idx) { + ASTContext &Context = getContext(); unsigned N = Record[Idx++]; NestedNameSpecifier *NNS = nullptr, *Prev = nullptr; for (unsigned I = 0; I != N; ++I) { @@ -8282,6 +8734,7 @@ ASTReader::ReadNestedNameSpecifier(ModuleFile &F, NestedNameSpecifierLoc ASTReader::ReadNestedNameSpecifierLoc(ModuleFile &F, const RecordData &Record, unsigned &Idx) { + ASTContext &Context = getContext(); unsigned N = Record[Idx++]; NestedNameSpecifierLocBuilder Builder; for (unsigned I = 0; I != N; ++I) { @@ -8403,14 +8856,14 @@ CXXTemporary *ASTReader::ReadCXXTemporary(ModuleFile &F, const RecordData &Record, unsigned &Idx) { CXXDestructorDecl *Decl = ReadDeclAs<CXXDestructorDecl>(F, Record, Idx); - return CXXTemporary::Create(Context, Decl); + return CXXTemporary::Create(getContext(), Decl); } -DiagnosticBuilder ASTReader::Diag(unsigned DiagID) { +DiagnosticBuilder ASTReader::Diag(unsigned DiagID) const { return Diag(CurrentImportLoc, DiagID); } -DiagnosticBuilder ASTReader::Diag(SourceLocation Loc, unsigned DiagID) { +DiagnosticBuilder ASTReader::Diag(SourceLocation Loc, unsigned DiagID) const { return Diags.Report(Loc, DiagID); } @@ -8439,6 +8892,7 @@ void ASTReader::ClearSwitchCaseIDs() { } void ASTReader::ReadComments() { + ASTContext &Context = getContext(); std::vector<RawComment *> Comments; for (SmallVectorImpl<std::pair<BitstreamCursor, serialization::ModuleFile *> >::iterator @@ -8493,6 +8947,34 @@ void ASTReader::ReadComments() { } } +void ASTReader::visitInputFiles(serialization::ModuleFile &MF, + bool IncludeSystem, bool Complain, + llvm::function_ref<void(const serialization::InputFile &IF, + bool isSystem)> Visitor) { + unsigned NumUserInputs = MF.NumUserInputFiles; + unsigned NumInputs = MF.InputFilesLoaded.size(); + assert(NumUserInputs <= NumInputs); + unsigned N = IncludeSystem ? NumInputs : NumUserInputs; + for (unsigned I = 0; I < N; ++I) { + bool IsSystem = I >= NumUserInputs; + InputFile IF = getInputFile(MF, I+1, Complain); + Visitor(IF, IsSystem); + } +} + +void ASTReader::visitTopLevelModuleMaps( + serialization::ModuleFile &MF, + llvm::function_ref<void(const FileEntry *FE)> Visitor) { + unsigned NumInputs = MF.InputFilesLoaded.size(); + for (unsigned I = 0; I < NumInputs; ++I) { + InputFileInfo IFI = readInputFileInfo(MF, I + 1); + if (IFI.TopLevelModuleMap) + // FIXME: This unnecessarily re-reads the InputFileInfo. + if (auto *FE = getInputFile(MF, I + 1).getFile()) + Visitor(FE); + } +} + std::string ASTReader::getOwningModuleNameForDiagnostic(const Decl *D) { // If we know the owning module, use it. if (Module *M = D->getImportedOwningModule()) @@ -8583,7 +9065,7 @@ void ASTReader::finishPendingActions() { while (!PendingUpdateRecords.empty()) { auto Update = PendingUpdateRecords.pop_back_val(); ReadingKindTracker ReadingKind(Read_Decl, *this); - loadDeclUpdateRecords(Update.first, Update.second); + loadDeclUpdateRecords(Update); } } @@ -8651,9 +9133,9 @@ void ASTReader::finishPendingActions() { // FIXME: Check for =delete/=default? // FIXME: Complain about ODR violations here? const FunctionDecl *Defn = nullptr; - if (!getContext().getLangOpts().Modules || !FD->hasBody(Defn)) + if (!getContext().getLangOpts().Modules || !FD->hasBody(Defn)) { FD->setLazyBody(PB->second); - else + } else mergeDefinitionVisibility(const_cast<FunctionDecl*>(Defn), FD); continue; } @@ -8789,24 +9271,796 @@ void ASTReader::diagnoseOdrViolations() { continue; bool Diagnosed = false; - for (auto *RD : Merge.second) { + CXXRecordDecl *FirstRecord = Merge.first; + std::string FirstModule = getOwningModuleNameForDiagnostic(FirstRecord); + for (CXXRecordDecl *SecondRecord : Merge.second) { // Multiple different declarations got merged together; tell the user // where they came from. - if (Merge.first != RD) { - // FIXME: Walk the definition, figure out what's different, - // and diagnose that. - if (!Diagnosed) { - std::string Module = getOwningModuleNameForDiagnostic(Merge.first); - Diag(Merge.first->getLocation(), - diag::err_module_odr_violation_different_definitions) - << Merge.first << Module.empty() << Module; - Diagnosed = true; + if (FirstRecord == SecondRecord) + continue; + + std::string SecondModule = getOwningModuleNameForDiagnostic(SecondRecord); + using DeclHashes = llvm::SmallVector<std::pair<Decl *, unsigned>, 4>; + DeclHashes FirstHashes; + DeclHashes SecondHashes; + ODRHash Hash; + + auto PopulateHashes = [&Hash, FirstRecord](DeclHashes &Hashes, + CXXRecordDecl *Record) { + for (auto *D : Record->decls()) { + // Due to decl merging, the first CXXRecordDecl is the parent of + // Decls in both records. + if (!ODRHash::isWhitelistedDecl(D, FirstRecord)) + continue; + Hash.clear(); + Hash.AddSubDecl(D); + Hashes.emplace_back(D, Hash.CalculateHash()); + } + }; + PopulateHashes(FirstHashes, FirstRecord); + PopulateHashes(SecondHashes, SecondRecord); + + // Used with err_module_odr_violation_mismatch_decl and + // note_module_odr_violation_mismatch_decl + // This list should be the same Decl's as in ODRHash::isWhiteListedDecl + enum { + EndOfClass, + PublicSpecifer, + PrivateSpecifer, + ProtectedSpecifer, + StaticAssert, + Field, + CXXMethod, + TypeAlias, + TypeDef, + Var, + Friend, + Other + } FirstDiffType = Other, + SecondDiffType = Other; + + auto DifferenceSelector = [](Decl *D) { + assert(D && "valid Decl required"); + switch (D->getKind()) { + default: + return Other; + case Decl::AccessSpec: + switch (D->getAccess()) { + case AS_public: + return PublicSpecifer; + case AS_private: + return PrivateSpecifer; + case AS_protected: + return ProtectedSpecifer; + case AS_none: + break; + } + llvm_unreachable("Invalid access specifier"); + case Decl::StaticAssert: + return StaticAssert; + case Decl::Field: + return Field; + case Decl::CXXMethod: + case Decl::CXXConstructor: + case Decl::CXXDestructor: + return CXXMethod; + case Decl::TypeAlias: + return TypeAlias; + case Decl::Typedef: + return TypeDef; + case Decl::Var: + return Var; + case Decl::Friend: + return Friend; } + }; + + Decl *FirstDecl = nullptr; + Decl *SecondDecl = nullptr; + auto FirstIt = FirstHashes.begin(); + auto SecondIt = SecondHashes.begin(); + + // If there is a diagnoseable difference, FirstDiffType and + // SecondDiffType will not be Other and FirstDecl and SecondDecl will be + // filled in if not EndOfClass. + while (FirstIt != FirstHashes.end() || SecondIt != SecondHashes.end()) { + if (FirstIt != FirstHashes.end() && SecondIt != SecondHashes.end() && + FirstIt->second == SecondIt->second) { + ++FirstIt; + ++SecondIt; + continue; + } + + FirstDecl = FirstIt == FirstHashes.end() ? nullptr : FirstIt->first; + SecondDecl = SecondIt == SecondHashes.end() ? nullptr : SecondIt->first; - Diag(RD->getLocation(), + FirstDiffType = FirstDecl ? DifferenceSelector(FirstDecl) : EndOfClass; + SecondDiffType = + SecondDecl ? DifferenceSelector(SecondDecl) : EndOfClass; + + break; + } + + if (FirstDiffType == Other || SecondDiffType == Other) { + // Reaching this point means an unexpected Decl was encountered + // or no difference was detected. This causes a generic error + // message to be emitted. + Diag(FirstRecord->getLocation(), + diag::err_module_odr_violation_different_definitions) + << FirstRecord << FirstModule.empty() << FirstModule; + + if (FirstDecl) { + Diag(FirstDecl->getLocation(), diag::note_first_module_difference) + << FirstRecord << FirstDecl->getSourceRange(); + } + + Diag(SecondRecord->getLocation(), diag::note_module_odr_violation_different_definitions) - << getOwningModuleNameForDiagnostic(RD); + << SecondModule; + + if (SecondDecl) { + Diag(SecondDecl->getLocation(), diag::note_second_module_difference) + << SecondDecl->getSourceRange(); + } + + Diagnosed = true; + break; + } + + if (FirstDiffType != SecondDiffType) { + SourceLocation FirstLoc; + SourceRange FirstRange; + if (FirstDiffType == EndOfClass) { + FirstLoc = FirstRecord->getBraceRange().getEnd(); + } else { + FirstLoc = FirstIt->first->getLocation(); + FirstRange = FirstIt->first->getSourceRange(); + } + Diag(FirstLoc, diag::err_module_odr_violation_mismatch_decl) + << FirstRecord << FirstModule.empty() << FirstModule << FirstRange + << FirstDiffType; + + SourceLocation SecondLoc; + SourceRange SecondRange; + if (SecondDiffType == EndOfClass) { + SecondLoc = SecondRecord->getBraceRange().getEnd(); + } else { + SecondLoc = SecondDecl->getLocation(); + SecondRange = SecondDecl->getSourceRange(); + } + Diag(SecondLoc, diag::note_module_odr_violation_mismatch_decl) + << SecondModule << SecondRange << SecondDiffType; + Diagnosed = true; + break; + } + + assert(FirstDiffType == SecondDiffType); + + // Used with err_module_odr_violation_mismatch_decl_diff and + // note_module_odr_violation_mismatch_decl_diff + enum ODRDeclDifference{ + StaticAssertCondition, + StaticAssertMessage, + StaticAssertOnlyMessage, + FieldName, + FieldTypeName, + FieldSingleBitField, + FieldDifferentWidthBitField, + FieldSingleMutable, + FieldSingleInitializer, + FieldDifferentInitializers, + MethodName, + MethodDeleted, + MethodVirtual, + MethodStatic, + MethodVolatile, + MethodConst, + MethodInline, + MethodNumberParameters, + MethodParameterType, + MethodParameterName, + MethodParameterSingleDefaultArgument, + MethodParameterDifferentDefaultArgument, + TypedefName, + TypedefType, + VarName, + VarType, + VarSingleInitializer, + VarDifferentInitializer, + VarConstexpr, + FriendTypeFunction, + FriendType, + FriendFunction, + }; + + // These lambdas have the common portions of the ODR diagnostics. This + // has the same return as Diag(), so addition parameters can be passed + // in with operator<< + auto ODRDiagError = [FirstRecord, &FirstModule, this]( + SourceLocation Loc, SourceRange Range, ODRDeclDifference DiffType) { + return Diag(Loc, diag::err_module_odr_violation_mismatch_decl_diff) + << FirstRecord << FirstModule.empty() << FirstModule << Range + << DiffType; + }; + auto ODRDiagNote = [&SecondModule, this]( + SourceLocation Loc, SourceRange Range, ODRDeclDifference DiffType) { + return Diag(Loc, diag::note_module_odr_violation_mismatch_decl_diff) + << SecondModule << Range << DiffType; + }; + + auto ComputeODRHash = [&Hash](const Stmt* S) { + assert(S); + Hash.clear(); + Hash.AddStmt(S); + return Hash.CalculateHash(); + }; + + auto ComputeQualTypeODRHash = [&Hash](QualType Ty) { + Hash.clear(); + Hash.AddQualType(Ty); + return Hash.CalculateHash(); + }; + + switch (FirstDiffType) { + case Other: + case EndOfClass: + case PublicSpecifer: + case PrivateSpecifer: + case ProtectedSpecifer: + llvm_unreachable("Invalid diff type"); + + case StaticAssert: { + StaticAssertDecl *FirstSA = cast<StaticAssertDecl>(FirstDecl); + StaticAssertDecl *SecondSA = cast<StaticAssertDecl>(SecondDecl); + + Expr *FirstExpr = FirstSA->getAssertExpr(); + Expr *SecondExpr = SecondSA->getAssertExpr(); + unsigned FirstODRHash = ComputeODRHash(FirstExpr); + unsigned SecondODRHash = ComputeODRHash(SecondExpr); + if (FirstODRHash != SecondODRHash) { + ODRDiagError(FirstExpr->getLocStart(), FirstExpr->getSourceRange(), + StaticAssertCondition); + ODRDiagNote(SecondExpr->getLocStart(), + SecondExpr->getSourceRange(), StaticAssertCondition); + Diagnosed = true; + break; + } + + StringLiteral *FirstStr = FirstSA->getMessage(); + StringLiteral *SecondStr = SecondSA->getMessage(); + assert((FirstStr || SecondStr) && "Both messages cannot be empty"); + if ((FirstStr && !SecondStr) || (!FirstStr && SecondStr)) { + SourceLocation FirstLoc, SecondLoc; + SourceRange FirstRange, SecondRange; + if (FirstStr) { + FirstLoc = FirstStr->getLocStart(); + FirstRange = FirstStr->getSourceRange(); + } else { + FirstLoc = FirstSA->getLocStart(); + FirstRange = FirstSA->getSourceRange(); + } + if (SecondStr) { + SecondLoc = SecondStr->getLocStart(); + SecondRange = SecondStr->getSourceRange(); + } else { + SecondLoc = SecondSA->getLocStart(); + SecondRange = SecondSA->getSourceRange(); + } + ODRDiagError(FirstLoc, FirstRange, StaticAssertOnlyMessage) + << (FirstStr == nullptr); + ODRDiagNote(SecondLoc, SecondRange, StaticAssertOnlyMessage) + << (SecondStr == nullptr); + Diagnosed = true; + break; + } + + if (FirstStr && SecondStr && + FirstStr->getString() != SecondStr->getString()) { + ODRDiagError(FirstStr->getLocStart(), FirstStr->getSourceRange(), + StaticAssertMessage); + ODRDiagNote(SecondStr->getLocStart(), SecondStr->getSourceRange(), + StaticAssertMessage); + Diagnosed = true; + break; + } + break; + } + case Field: { + FieldDecl *FirstField = cast<FieldDecl>(FirstDecl); + FieldDecl *SecondField = cast<FieldDecl>(SecondDecl); + IdentifierInfo *FirstII = FirstField->getIdentifier(); + IdentifierInfo *SecondII = SecondField->getIdentifier(); + if (FirstII->getName() != SecondII->getName()) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldName) + << FirstII; + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldName) + << SecondII; + + Diagnosed = true; + break; + } + + assert(getContext().hasSameType(FirstField->getType(), + SecondField->getType())); + + QualType FirstType = FirstField->getType(); + QualType SecondType = SecondField->getType(); + if (ComputeQualTypeODRHash(FirstType) != + ComputeQualTypeODRHash(SecondType)) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldTypeName) + << FirstII << FirstType; + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldTypeName) + << SecondII << SecondType; + + Diagnosed = true; + break; + } + + const bool IsFirstBitField = FirstField->isBitField(); + const bool IsSecondBitField = SecondField->isBitField(); + if (IsFirstBitField != IsSecondBitField) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldSingleBitField) + << FirstII << IsFirstBitField; + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldSingleBitField) + << SecondII << IsSecondBitField; + Diagnosed = true; + break; + } + + if (IsFirstBitField && IsSecondBitField) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldDifferentWidthBitField) + << FirstII << FirstField->getBitWidth()->getSourceRange(); + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldDifferentWidthBitField) + << SecondII << SecondField->getBitWidth()->getSourceRange(); + Diagnosed = true; + break; + } + + const bool IsFirstMutable = FirstField->isMutable(); + const bool IsSecondMutable = SecondField->isMutable(); + if (IsFirstMutable != IsSecondMutable) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldSingleMutable) + << FirstII << IsFirstMutable; + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldSingleMutable) + << SecondII << IsSecondMutable; + Diagnosed = true; + break; + } + + const Expr *FirstInitializer = FirstField->getInClassInitializer(); + const Expr *SecondInitializer = SecondField->getInClassInitializer(); + if ((!FirstInitializer && SecondInitializer) || + (FirstInitializer && !SecondInitializer)) { + ODRDiagError(FirstField->getLocation(), FirstField->getSourceRange(), + FieldSingleInitializer) + << FirstII << (FirstInitializer != nullptr); + ODRDiagNote(SecondField->getLocation(), SecondField->getSourceRange(), + FieldSingleInitializer) + << SecondII << (SecondInitializer != nullptr); + Diagnosed = true; + break; + } + + if (FirstInitializer && SecondInitializer) { + unsigned FirstInitHash = ComputeODRHash(FirstInitializer); + unsigned SecondInitHash = ComputeODRHash(SecondInitializer); + if (FirstInitHash != SecondInitHash) { + ODRDiagError(FirstField->getLocation(), + FirstField->getSourceRange(), + FieldDifferentInitializers) + << FirstII << FirstInitializer->getSourceRange(); + ODRDiagNote(SecondField->getLocation(), + SecondField->getSourceRange(), + FieldDifferentInitializers) + << SecondII << SecondInitializer->getSourceRange(); + Diagnosed = true; + break; + } + } + + break; } + case CXXMethod: { + enum { + DiagMethod, + DiagConstructor, + DiagDestructor, + } FirstMethodType, + SecondMethodType; + auto GetMethodTypeForDiagnostics = [](const CXXMethodDecl* D) { + if (isa<CXXConstructorDecl>(D)) return DiagConstructor; + if (isa<CXXDestructorDecl>(D)) return DiagDestructor; + return DiagMethod; + }; + const CXXMethodDecl *FirstMethod = cast<CXXMethodDecl>(FirstDecl); + const CXXMethodDecl *SecondMethod = cast<CXXMethodDecl>(SecondDecl); + FirstMethodType = GetMethodTypeForDiagnostics(FirstMethod); + SecondMethodType = GetMethodTypeForDiagnostics(SecondMethod); + auto FirstName = FirstMethod->getDeclName(); + auto SecondName = SecondMethod->getDeclName(); + if (FirstMethodType != SecondMethodType || FirstName != SecondName) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodName) + << FirstMethodType << FirstName; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodName) + << SecondMethodType << SecondName; + + Diagnosed = true; + break; + } + + const bool FirstDeleted = FirstMethod->isDeleted(); + const bool SecondDeleted = SecondMethod->isDeleted(); + if (FirstDeleted != SecondDeleted) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodDeleted) + << FirstMethodType << FirstName << FirstDeleted; + + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodDeleted) + << SecondMethodType << SecondName << SecondDeleted; + Diagnosed = true; + break; + } + + const bool FirstVirtual = FirstMethod->isVirtualAsWritten(); + const bool SecondVirtual = SecondMethod->isVirtualAsWritten(); + const bool FirstPure = FirstMethod->isPure(); + const bool SecondPure = SecondMethod->isPure(); + if ((FirstVirtual || SecondVirtual) && + (FirstVirtual != SecondVirtual || FirstPure != SecondPure)) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodVirtual) + << FirstMethodType << FirstName << FirstPure << FirstVirtual; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodVirtual) + << SecondMethodType << SecondName << SecondPure << SecondVirtual; + Diagnosed = true; + break; + } + + // CXXMethodDecl::isStatic uses the canonical Decl. With Decl merging, + // FirstDecl is the canonical Decl of SecondDecl, so the storage + // class needs to be checked instead. + const auto FirstStorage = FirstMethod->getStorageClass(); + const auto SecondStorage = SecondMethod->getStorageClass(); + const bool FirstStatic = FirstStorage == SC_Static; + const bool SecondStatic = SecondStorage == SC_Static; + if (FirstStatic != SecondStatic) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodStatic) + << FirstMethodType << FirstName << FirstStatic; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodStatic) + << SecondMethodType << SecondName << SecondStatic; + Diagnosed = true; + break; + } + + const bool FirstVolatile = FirstMethod->isVolatile(); + const bool SecondVolatile = SecondMethod->isVolatile(); + if (FirstVolatile != SecondVolatile) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodVolatile) + << FirstMethodType << FirstName << FirstVolatile; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodVolatile) + << SecondMethodType << SecondName << SecondVolatile; + Diagnosed = true; + break; + } + + const bool FirstConst = FirstMethod->isConst(); + const bool SecondConst = SecondMethod->isConst(); + if (FirstConst != SecondConst) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodConst) + << FirstMethodType << FirstName << FirstConst; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodConst) + << SecondMethodType << SecondName << SecondConst; + Diagnosed = true; + break; + } + + const bool FirstInline = FirstMethod->isInlineSpecified(); + const bool SecondInline = SecondMethod->isInlineSpecified(); + if (FirstInline != SecondInline) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodInline) + << FirstMethodType << FirstName << FirstInline; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodInline) + << SecondMethodType << SecondName << SecondInline; + Diagnosed = true; + break; + } + + const unsigned FirstNumParameters = FirstMethod->param_size(); + const unsigned SecondNumParameters = SecondMethod->param_size(); + if (FirstNumParameters != SecondNumParameters) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodNumberParameters) + << FirstMethodType << FirstName << FirstNumParameters; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodNumberParameters) + << SecondMethodType << SecondName << SecondNumParameters; + Diagnosed = true; + break; + } + + // Need this status boolean to know when break out of the switch. + bool ParameterMismatch = false; + for (unsigned I = 0; I < FirstNumParameters; ++I) { + const ParmVarDecl *FirstParam = FirstMethod->getParamDecl(I); + const ParmVarDecl *SecondParam = SecondMethod->getParamDecl(I); + + QualType FirstParamType = FirstParam->getType(); + QualType SecondParamType = SecondParam->getType(); + if (FirstParamType != SecondParamType && + ComputeQualTypeODRHash(FirstParamType) != + ComputeQualTypeODRHash(SecondParamType)) { + if (const DecayedType *ParamDecayedType = + FirstParamType->getAs<DecayedType>()) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodParameterType) + << FirstMethodType << FirstName << (I + 1) << FirstParamType + << true << ParamDecayedType->getOriginalType(); + } else { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodParameterType) + << FirstMethodType << FirstName << (I + 1) << FirstParamType + << false; + } + + if (const DecayedType *ParamDecayedType = + SecondParamType->getAs<DecayedType>()) { + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodParameterType) + << SecondMethodType << SecondName << (I + 1) + << SecondParamType << true + << ParamDecayedType->getOriginalType(); + } else { + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodParameterType) + << SecondMethodType << SecondName << (I + 1) + << SecondParamType << false; + } + ParameterMismatch = true; + break; + } + + DeclarationName FirstParamName = FirstParam->getDeclName(); + DeclarationName SecondParamName = SecondParam->getDeclName(); + if (FirstParamName != SecondParamName) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), MethodParameterName) + << FirstMethodType << FirstName << (I + 1) << FirstParamName; + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), MethodParameterName) + << SecondMethodType << SecondName << (I + 1) << SecondParamName; + ParameterMismatch = true; + break; + } + + const Expr *FirstInit = FirstParam->getInit(); + const Expr *SecondInit = SecondParam->getInit(); + if ((FirstInit == nullptr) != (SecondInit == nullptr)) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), + MethodParameterSingleDefaultArgument) + << FirstMethodType << FirstName << (I + 1) + << (FirstInit == nullptr) + << (FirstInit ? FirstInit->getSourceRange() : SourceRange()); + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), + MethodParameterSingleDefaultArgument) + << SecondMethodType << SecondName << (I + 1) + << (SecondInit == nullptr) + << (SecondInit ? SecondInit->getSourceRange() : SourceRange()); + ParameterMismatch = true; + break; + } + + if (FirstInit && SecondInit && + ComputeODRHash(FirstInit) != ComputeODRHash(SecondInit)) { + ODRDiagError(FirstMethod->getLocation(), + FirstMethod->getSourceRange(), + MethodParameterDifferentDefaultArgument) + << FirstMethodType << FirstName << (I + 1) + << FirstInit->getSourceRange(); + ODRDiagNote(SecondMethod->getLocation(), + SecondMethod->getSourceRange(), + MethodParameterDifferentDefaultArgument) + << SecondMethodType << SecondName << (I + 1) + << SecondInit->getSourceRange(); + ParameterMismatch = true; + break; + + } + } + + if (ParameterMismatch) { + Diagnosed = true; + break; + } + + break; + } + case TypeAlias: + case TypeDef: { + TypedefNameDecl *FirstTD = cast<TypedefNameDecl>(FirstDecl); + TypedefNameDecl *SecondTD = cast<TypedefNameDecl>(SecondDecl); + auto FirstName = FirstTD->getDeclName(); + auto SecondName = SecondTD->getDeclName(); + if (FirstName != SecondName) { + ODRDiagError(FirstTD->getLocation(), FirstTD->getSourceRange(), + TypedefName) + << (FirstDiffType == TypeAlias) << FirstName; + ODRDiagNote(SecondTD->getLocation(), SecondTD->getSourceRange(), + TypedefName) + << (FirstDiffType == TypeAlias) << SecondName; + Diagnosed = true; + break; + } + + QualType FirstType = FirstTD->getUnderlyingType(); + QualType SecondType = SecondTD->getUnderlyingType(); + if (ComputeQualTypeODRHash(FirstType) != + ComputeQualTypeODRHash(SecondType)) { + ODRDiagError(FirstTD->getLocation(), FirstTD->getSourceRange(), + TypedefType) + << (FirstDiffType == TypeAlias) << FirstName << FirstType; + ODRDiagNote(SecondTD->getLocation(), SecondTD->getSourceRange(), + TypedefType) + << (FirstDiffType == TypeAlias) << SecondName << SecondType; + Diagnosed = true; + break; + } + break; + } + case Var: { + VarDecl *FirstVD = cast<VarDecl>(FirstDecl); + VarDecl *SecondVD = cast<VarDecl>(SecondDecl); + auto FirstName = FirstVD->getDeclName(); + auto SecondName = SecondVD->getDeclName(); + if (FirstName != SecondName) { + ODRDiagError(FirstVD->getLocation(), FirstVD->getSourceRange(), + VarName) + << FirstName; + ODRDiagNote(SecondVD->getLocation(), SecondVD->getSourceRange(), + VarName) + << SecondName; + Diagnosed = true; + break; + } + + QualType FirstType = FirstVD->getType(); + QualType SecondType = SecondVD->getType(); + if (ComputeQualTypeODRHash(FirstType) != + ComputeQualTypeODRHash(SecondType)) { + ODRDiagError(FirstVD->getLocation(), FirstVD->getSourceRange(), + VarType) + << FirstName << FirstType; + ODRDiagNote(SecondVD->getLocation(), SecondVD->getSourceRange(), + VarType) + << SecondName << SecondType; + Diagnosed = true; + break; + } + + const Expr *FirstInit = FirstVD->getInit(); + const Expr *SecondInit = SecondVD->getInit(); + if ((FirstInit == nullptr) != (SecondInit == nullptr)) { + ODRDiagError(FirstVD->getLocation(), FirstVD->getSourceRange(), + VarSingleInitializer) + << FirstName << (FirstInit == nullptr) + << (FirstInit ? FirstInit->getSourceRange(): SourceRange()); + ODRDiagNote(SecondVD->getLocation(), SecondVD->getSourceRange(), + VarSingleInitializer) + << SecondName << (SecondInit == nullptr) + << (SecondInit ? SecondInit->getSourceRange() : SourceRange()); + Diagnosed = true; + break; + } + + if (FirstInit && SecondInit && + ComputeODRHash(FirstInit) != ComputeODRHash(SecondInit)) { + ODRDiagError(FirstVD->getLocation(), FirstVD->getSourceRange(), + VarDifferentInitializer) + << FirstName << FirstInit->getSourceRange(); + ODRDiagNote(SecondVD->getLocation(), SecondVD->getSourceRange(), + VarDifferentInitializer) + << SecondName << SecondInit->getSourceRange(); + Diagnosed = true; + break; + } + + const bool FirstIsConstexpr = FirstVD->isConstexpr(); + const bool SecondIsConstexpr = SecondVD->isConstexpr(); + if (FirstIsConstexpr != SecondIsConstexpr) { + ODRDiagError(FirstVD->getLocation(), FirstVD->getSourceRange(), + VarConstexpr) + << FirstName << FirstIsConstexpr; + ODRDiagNote(SecondVD->getLocation(), SecondVD->getSourceRange(), + VarConstexpr) + << SecondName << SecondIsConstexpr; + Diagnosed = true; + break; + } + break; + } + case Friend: { + FriendDecl *FirstFriend = cast<FriendDecl>(FirstDecl); + FriendDecl *SecondFriend = cast<FriendDecl>(SecondDecl); + + NamedDecl *FirstND = FirstFriend->getFriendDecl(); + NamedDecl *SecondND = SecondFriend->getFriendDecl(); + + TypeSourceInfo *FirstTSI = FirstFriend->getFriendType(); + TypeSourceInfo *SecondTSI = SecondFriend->getFriendType(); + + if (FirstND && SecondND) { + ODRDiagError(FirstFriend->getFriendLoc(), + FirstFriend->getSourceRange(), FriendFunction) + << FirstND; + ODRDiagNote(SecondFriend->getFriendLoc(), + SecondFriend->getSourceRange(), FriendFunction) + << SecondND; + + Diagnosed = true; + break; + } + + if (FirstTSI && SecondTSI) { + QualType FirstFriendType = FirstTSI->getType(); + QualType SecondFriendType = SecondTSI->getType(); + assert(ComputeQualTypeODRHash(FirstFriendType) != + ComputeQualTypeODRHash(SecondFriendType)); + ODRDiagError(FirstFriend->getFriendLoc(), + FirstFriend->getSourceRange(), FriendType) + << FirstFriendType; + ODRDiagNote(SecondFriend->getFriendLoc(), + SecondFriend->getSourceRange(), FriendType) + << SecondFriendType; + Diagnosed = true; + break; + } + + ODRDiagError(FirstFriend->getFriendLoc(), FirstFriend->getSourceRange(), + FriendTypeFunction) + << (FirstTSI == nullptr); + ODRDiagNote(SecondFriend->getFriendLoc(), + SecondFriend->getSourceRange(), FriendTypeFunction) + << (SecondTSI == nullptr); + + Diagnosed = true; + break; + } + } + + if (Diagnosed == true) + continue; + + Diag(FirstDecl->getLocation(), + diag::err_module_odr_violation_mismatch_decl_unknown) + << FirstRecord << FirstModule.empty() << FirstModule << FirstDiffType + << FirstDecl->getSourceRange(); + Diag(SecondDecl->getLocation(), + diag::note_module_odr_violation_mismatch_decl_unknown) + << SecondModule << FirstDiffType << SecondDecl->getSourceRange(); + Diagnosed = true; } if (!Diagnosed) { @@ -8847,10 +10101,10 @@ void ASTReader::FinishedDeserializing() { ProcessingUpdatesRAIIObj ProcessingUpdates(*this); auto *FPT = Update.second->getType()->castAs<FunctionProtoType>(); auto ESI = FPT->getExtProtoInfo().ExceptionSpec; - if (auto *Listener = Context.getASTMutationListener()) + if (auto *Listener = getContext().getASTMutationListener()) Listener->ResolvedExceptionSpec(cast<FunctionDecl>(Update.second)); for (auto *Redecl : Update.second->redecls()) - Context.adjustExceptionSpec(cast<FunctionDecl>(Redecl), ESI); + getContext().adjustExceptionSpec(cast<FunctionDecl>(Redecl), ESI); } } @@ -8892,7 +10146,7 @@ void ASTReader::pushExternalDeclIntoScope(NamedDecl *D, DeclarationName Name) { } } -ASTReader::ASTReader(Preprocessor &PP, ASTContext &Context, +ASTReader::ASTReader(Preprocessor &PP, ASTContext *Context, const PCHContainerReader &PCHContainerRdr, ArrayRef<std::shared_ptr<ModuleFileExtension>> Extensions, StringRef isysroot, bool DisableValidation, @@ -8905,8 +10159,10 @@ ASTReader::ASTReader(Preprocessor &PP, ASTContext &Context, : cast<ASTReaderListener>(new PCHValidator(PP, *this))), SourceMgr(PP.getSourceManager()), FileMgr(PP.getFileManager()), PCHContainerRdr(PCHContainerRdr), Diags(PP.getDiagnostics()), PP(PP), - Context(Context), ModuleMgr(PP.getFileManager(), PCHContainerRdr), - DummyIdResolver(PP), ReadTimer(std::move(ReadTimer)), isysroot(isysroot), + ContextObj(Context), + ModuleMgr(PP.getFileManager(), PP.getPCMCache(), PCHContainerRdr), + PCMCache(PP.getPCMCache()), DummyIdResolver(PP), + ReadTimer(std::move(ReadTimer)), isysroot(isysroot), DisableValidation(DisableValidation), AllowASTWithCompilerErrors(AllowASTWithCompilerErrors), AllowConfigurationMismatch(AllowConfigurationMismatch), diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp index 707a924..0853415 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTReaderDecl.cpp @@ -119,12 +119,16 @@ namespace clang { } void ReadCXXRecordDefinition(CXXRecordDecl *D, bool Update); - void ReadCXXDefinitionData(struct CXXRecordDecl::DefinitionData &Data); + void ReadCXXDefinitionData(struct CXXRecordDecl::DefinitionData &Data, + const CXXRecordDecl *D); void MergeDefinitionData(CXXRecordDecl *D, struct CXXRecordDecl::DefinitionData &&NewDD); void ReadObjCDefinitionData(struct ObjCInterfaceDecl::DefinitionData &Data); void MergeDefinitionData(ObjCInterfaceDecl *D, struct ObjCInterfaceDecl::DefinitionData &&NewDD); + void ReadObjCDefinitionData(struct ObjCProtocolDecl::DefinitionData &Data); + void MergeDefinitionData(ObjCProtocolDecl *D, + struct ObjCProtocolDecl::DefinitionData &&NewDD); static NamedDecl *getAnonymousDeclForMerging(ASTReader &Reader, DeclContext *DC, @@ -215,6 +219,30 @@ namespace clang { TypedefNameForLinkage(nullptr), HasPendingBody(false), IsDeclMarkedUsed(false) {} + template <typename T> static + void AddLazySpecializations(T *D, + SmallVectorImpl<serialization::DeclID>& IDs) { + if (IDs.empty()) + return; + + // FIXME: We should avoid this pattern of getting the ASTContext. + ASTContext &C = D->getASTContext(); + + auto *&LazySpecializations = D->getCommonPtr()->LazySpecializations; + + if (auto &Old = LazySpecializations) { + IDs.insert(IDs.end(), Old + 1, Old + 1 + Old[0]); + std::sort(IDs.begin(), IDs.end()); + IDs.erase(std::unique(IDs.begin(), IDs.end()), IDs.end()); + } + + auto *Result = new (C) serialization::DeclID[1 + IDs.size()]; + *Result = IDs.size(); + std::copy(IDs.begin(), IDs.end(), Result + 1); + + LazySpecializations = Result; + } + template <typename DeclT> static Decl *getMostRecentDeclImpl(Redeclarable<DeclT> *D); static Decl *getMostRecentDeclImpl(...); @@ -240,9 +268,10 @@ namespace clang { /// \brief Determine whether this declaration has a pending body. bool hasPendingBody() const { return HasPendingBody; } + void ReadFunctionDefinition(FunctionDecl *FD); void Visit(Decl *D); - void UpdateDecl(Decl *D); + void UpdateDecl(Decl *D, llvm::SmallVectorImpl<serialization::DeclID>&); static void setNextObjCCategory(ObjCCategoryDecl *Cat, ObjCCategoryDecl *Next) { @@ -292,6 +321,7 @@ namespace clang { void VisitUnresolvedUsingValueDecl(UnresolvedUsingValueDecl *D); void VisitDeclaratorDecl(DeclaratorDecl *DD); void VisitFunctionDecl(FunctionDecl *FD); + void VisitCXXDeductionGuideDecl(CXXDeductionGuideDecl *GD); void VisitCXXMethodDecl(CXXMethodDecl *D); void VisitCXXConstructorDecl(CXXConstructorDecl *D); void VisitCXXDestructorDecl(CXXDestructorDecl *D); @@ -421,6 +451,19 @@ uint64_t ASTDeclReader::GetCurrentCursorOffset() { return Loc.F->DeclsCursor.GetCurrentBitNo() + Loc.F->GlobalBitOffset; } +void ASTDeclReader::ReadFunctionDefinition(FunctionDecl *FD) { + if (Record.readInt()) + Reader.BodySource[FD] = Loc.F->Kind == ModuleKind::MK_MainFile; + if (auto *CD = dyn_cast<CXXConstructorDecl>(FD)) { + CD->NumCtorInitializers = Record.readInt(); + if (CD->NumCtorInitializers) + CD->CtorInitializers = ReadGlobalOffset(); + } + // Store the offset of the body so we can lazily load it later. + Reader.PendingBodies[FD] = GetCurrentCursorOffset(); + HasPendingBody = true; +} + void ASTDeclReader::Visit(Decl *D) { DeclVisitor<ASTDeclReader, void>::Visit(D); @@ -457,15 +500,8 @@ void ASTDeclReader::Visit(Decl *D) { // We only read it if FD doesn't already have a body (e.g., from another // module). // FIXME: Can we diagnose ODR violations somehow? - if (Record.readInt()) { - if (auto *CD = dyn_cast<CXXConstructorDecl>(FD)) { - CD->NumCtorInitializers = Record.readInt(); - if (CD->NumCtorInitializers) - CD->CtorInitializers = ReadGlobalOffset(); - } - Reader.PendingBodies[FD] = GetCurrentCursorOffset(); - HasPendingBody = true; - } + if (Record.readInt()) + ReadFunctionDefinition(FD); } } @@ -513,32 +549,32 @@ void ASTDeclReader::VisitDecl(Decl *D) { D->setTopLevelDeclInObjCContainer(Record.readInt()); D->setAccess((AccessSpecifier)Record.readInt()); D->FromASTFile = true; - D->setModulePrivate(Record.readInt()); - D->Hidden = D->isModulePrivate(); + bool ModulePrivate = Record.readInt(); // Determine whether this declaration is part of a (sub)module. If so, it // may not yet be visible. if (unsigned SubmoduleID = readSubmoduleID()) { // Store the owning submodule ID in the declaration. + D->setModuleOwnershipKind( + ModulePrivate ? Decl::ModuleOwnershipKind::ModulePrivate + : Decl::ModuleOwnershipKind::VisibleWhenImported); D->setOwningModuleID(SubmoduleID); - if (D->Hidden) { - // Module-private declarations are never visible, so there is no work to do. + if (ModulePrivate) { + // Module-private declarations are never visible, so there is no work to + // do. } else if (Reader.getContext().getLangOpts().ModulesLocalVisibility) { // If local visibility is being tracked, this declaration will become - // hidden and visible as the owning module does. Inform Sema that this - // declaration might not be visible. - D->Hidden = true; + // hidden and visible as the owning module does. } else if (Module *Owner = Reader.getSubmodule(SubmoduleID)) { - if (Owner->NameVisibility != Module::AllVisible) { - // The owning module is not visible. Mark this declaration as hidden. - D->Hidden = true; - - // Note that this declaration was hidden because its owning module is - // not yet visible. + // Mark the declaration as visible when its owning module becomes visible. + if (Owner->NameVisibility == Module::AllVisible) + D->setVisibleDespiteOwningModule(); + else Reader.HiddenNamesMap[Owner].push_back(D); - } } + } else if (ModulePrivate) { + D->setModuleOwnershipKind(Decl::ModuleOwnershipKind::ModulePrivate); } } @@ -592,6 +628,11 @@ ASTDeclReader::VisitTypedefNameDecl(TypedefNameDecl *TD) { TD->setModedTypeSourceInfo(TInfo, modedT); } else TD->setTypeSourceInfo(TInfo); + // Read and discard the declaration for which this is a typedef name for + // linkage, if it exists. We cannot rely on our type to pull in this decl, + // because it might have been merged with a type from another module and + // thus might not refer to our version of the declaration. + ReadDecl(); return Redecl; } @@ -738,6 +779,7 @@ void ASTDeclReader::VisitFunctionDecl(FunctionDecl *FD) { FD->SClass = (StorageClass)Record.readInt(); FD->IsInline = Record.readInt(); FD->IsInlineSpecified = Record.readInt(); + FD->IsExplicitSpecified = Record.readInt(); FD->IsVirtualAsWritten = Record.readInt(); FD->IsPure = Record.readInt(); FD->HasInheritedPrototype = Record.readInt(); @@ -1032,18 +1074,8 @@ void ASTDeclReader::VisitObjCIvarDecl(ObjCIvarDecl *IVD) { IVD->setSynthesize(synth); } -void ASTDeclReader::VisitObjCProtocolDecl(ObjCProtocolDecl *PD) { - RedeclarableResult Redecl = VisitRedeclarable(PD); - VisitObjCContainerDecl(PD); - mergeRedeclarable(PD, Redecl); - - if (Record.readInt()) { - // Read the definition. - PD->allocateDefinitionData(); - - // Set the definition data of the canonical declaration, so other - // redeclarations will see it. - PD->getCanonicalDecl()->Data = PD->Data; +void ASTDeclReader::ReadObjCDefinitionData( + struct ObjCProtocolDecl::DefinitionData &Data) { unsigned NumProtoRefs = Record.readInt(); SmallVector<ObjCProtocolDecl *, 16> ProtoRefs; @@ -1054,9 +1086,37 @@ void ASTDeclReader::VisitObjCProtocolDecl(ObjCProtocolDecl *PD) { ProtoLocs.reserve(NumProtoRefs); for (unsigned I = 0; I != NumProtoRefs; ++I) ProtoLocs.push_back(ReadSourceLocation()); - PD->setProtocolList(ProtoRefs.data(), NumProtoRefs, ProtoLocs.data(), - Reader.getContext()); + Data.ReferencedProtocols.set(ProtoRefs.data(), NumProtoRefs, + ProtoLocs.data(), Reader.getContext()); +} + +void ASTDeclReader::MergeDefinitionData(ObjCProtocolDecl *D, + struct ObjCProtocolDecl::DefinitionData &&NewDD) { + // FIXME: odr checking? +} + +void ASTDeclReader::VisitObjCProtocolDecl(ObjCProtocolDecl *PD) { + RedeclarableResult Redecl = VisitRedeclarable(PD); + VisitObjCContainerDecl(PD); + mergeRedeclarable(PD, Redecl); + + if (Record.readInt()) { + // Read the definition. + PD->allocateDefinitionData(); + + ReadObjCDefinitionData(PD->data()); + ObjCProtocolDecl *Canon = PD->getCanonicalDecl(); + if (Canon->Data.getPointer()) { + // If we already have a definition, keep the definition invariant and + // merge the data. + MergeDefinitionData(Canon, std::move(PD->data())); + PD->Data = Canon->Data; + } else { + // Set the definition data of the canonical declaration, so other + // redeclarations will see it. + PD->getCanonicalDecl()->Data = PD->Data; + } // Note that we have deserialized a definition. Reader.PendingDefinitions.insert(PD); } else { @@ -1112,8 +1172,12 @@ void ASTDeclReader::VisitObjCPropertyDecl(ObjCPropertyDecl *D) { (ObjCPropertyDecl::PropertyAttributeKind)Record.readInt()); D->setPropertyImplementation( (ObjCPropertyDecl::PropertyControl)Record.readInt()); - D->setGetterName(Record.readDeclarationName().getObjCSelector()); - D->setSetterName(Record.readDeclarationName().getObjCSelector()); + DeclarationName GetterName = Record.readDeclarationName(); + SourceLocation GetterLoc = ReadSourceLocation(); + D->setGetterName(GetterName.getObjCSelector(), GetterLoc); + DeclarationName SetterName = Record.readDeclarationName(); + SourceLocation SetterLoc = ReadSourceLocation(); + D->setSetterName(SetterName.getObjCSelector(), SetterLoc); D->setGetterMethodDecl(ReadDeclAs<ObjCMethodDecl>()); D->setSetterMethodDecl(ReadDeclAs<ObjCMethodDecl>()); D->setPropertyIvarDecl(ReadDeclAs<ObjCIvarDecl>()); @@ -1126,7 +1190,6 @@ void ASTDeclReader::VisitObjCImplDecl(ObjCImplDecl *D) { void ASTDeclReader::VisitObjCCategoryImplDecl(ObjCCategoryImplDecl *D) { VisitObjCImplDecl(D); - D->setIdentifier(Record.getIdentifierInfo()); D->CategoryNameLoc = ReadSourceLocation(); } @@ -1211,6 +1274,7 @@ ASTDeclReader::RedeclarableResult ASTDeclReader::VisitVarDeclImpl(VarDecl *VD) { VD->NonParmVarDeclBits.IsConstexpr = Record.readInt(); VD->NonParmVarDeclBits.IsInitCapture = Record.readInt(); VD->NonParmVarDeclBits.PreviousDeclInSameBlockScope = Record.readInt(); + VD->NonParmVarDeclBits.ImplicitParamKind = Record.readInt(); } Linkage VarLinkage = Linkage(Record.readInt()); VD->setCachedLinkage(VarLinkage); @@ -1473,7 +1537,7 @@ void ASTDeclReader::VisitUnresolvedUsingTypenameDecl( } void ASTDeclReader::ReadCXXDefinitionData( - struct CXXRecordDecl::DefinitionData &Data) { + struct CXXRecordDecl::DefinitionData &Data, const CXXRecordDecl *D) { // Note: the caller has deserialized the IsLambda bit already. Data.UserDeclaredConstructor = Record.readInt(); Data.UserDeclaredSpecialMembers = Record.readInt(); @@ -1495,9 +1559,11 @@ void ASTDeclReader::ReadCXXDefinitionData( Data.HasUninitializedFields = Record.readInt(); Data.HasInheritedConstructor = Record.readInt(); Data.HasInheritedAssignment = Record.readInt(); + Data.NeedOverloadResolutionForCopyConstructor = Record.readInt(); Data.NeedOverloadResolutionForMoveConstructor = Record.readInt(); Data.NeedOverloadResolutionForMoveAssignment = Record.readInt(); Data.NeedOverloadResolutionForDestructor = Record.readInt(); + Data.DefaultedCopyConstructorIsDeleted = Record.readInt(); Data.DefaultedMoveConstructorIsDeleted = Record.readInt(); Data.DefaultedMoveAssignmentIsDeleted = Record.readInt(); Data.DefaultedDestructorIsDeleted = Record.readInt(); @@ -1506,16 +1572,26 @@ void ASTDeclReader::ReadCXXDefinitionData( Data.HasIrrelevantDestructor = Record.readInt(); Data.HasConstexprNonCopyMoveConstructor = Record.readInt(); Data.HasDefaultedDefaultConstructor = Record.readInt(); + Data.CanPassInRegisters = Record.readInt(); Data.DefaultedDefaultConstructorIsConstexpr = Record.readInt(); Data.HasConstexprDefaultConstructor = Record.readInt(); Data.HasNonLiteralTypeFieldsOrBases = Record.readInt(); Data.ComputedVisibleConversions = Record.readInt(); Data.UserProvidedDefaultConstructor = Record.readInt(); Data.DeclaredSpecialMembers = Record.readInt(); - Data.ImplicitCopyConstructorHasConstParam = Record.readInt(); + Data.ImplicitCopyConstructorCanHaveConstParamForVBase = Record.readInt(); + Data.ImplicitCopyConstructorCanHaveConstParamForNonVBase = Record.readInt(); Data.ImplicitCopyAssignmentHasConstParam = Record.readInt(); Data.HasDeclaredCopyConstructorWithConstParam = Record.readInt(); Data.HasDeclaredCopyAssignmentWithConstParam = Record.readInt(); + Data.ODRHash = Record.readInt(); + Data.HasODRHash = true; + + if (Record.readInt()) { + Reader.BodySource[D] = Loc.F->Kind == ModuleKind::MK_MainFile + ? ExternalASTSource::EK_Never + : ExternalASTSource::EK_Always; + } Data.NumBases = Record.readInt(); if (Data.NumBases) @@ -1540,8 +1616,8 @@ void ASTDeclReader::ReadCXXDefinitionData( Lambda.NumExplicitCaptures = Record.readInt(); Lambda.ManglingNumber = Record.readInt(); Lambda.ContextDecl = ReadDeclID(); - Lambda.Captures - = (Capture*)Reader.Context.Allocate(sizeof(Capture)*Lambda.NumCaptures); + Lambda.Captures = (Capture *)Reader.getContext().Allocate( + sizeof(Capture) * Lambda.NumCaptures); Capture *ToCapture = Lambda.Captures; Lambda.MethodTyInfo = GetTypeSourceInfo(); for (unsigned I = 0, N = Lambda.NumCaptures; I != N; ++I) { @@ -1624,9 +1700,11 @@ void ASTDeclReader::MergeDefinitionData( MATCH_FIELD(HasUninitializedFields) MATCH_FIELD(HasInheritedConstructor) MATCH_FIELD(HasInheritedAssignment) + MATCH_FIELD(NeedOverloadResolutionForCopyConstructor) MATCH_FIELD(NeedOverloadResolutionForMoveConstructor) MATCH_FIELD(NeedOverloadResolutionForMoveAssignment) MATCH_FIELD(NeedOverloadResolutionForDestructor) + MATCH_FIELD(DefaultedCopyConstructorIsDeleted) MATCH_FIELD(DefaultedMoveConstructorIsDeleted) MATCH_FIELD(DefaultedMoveAssignmentIsDeleted) MATCH_FIELD(DefaultedDestructorIsDeleted) @@ -1635,13 +1713,15 @@ void ASTDeclReader::MergeDefinitionData( MATCH_FIELD(HasIrrelevantDestructor) OR_FIELD(HasConstexprNonCopyMoveConstructor) OR_FIELD(HasDefaultedDefaultConstructor) + MATCH_FIELD(CanPassInRegisters) MATCH_FIELD(DefaultedDefaultConstructorIsConstexpr) OR_FIELD(HasConstexprDefaultConstructor) MATCH_FIELD(HasNonLiteralTypeFieldsOrBases) // ComputedVisibleConversions is handled below. MATCH_FIELD(UserProvidedDefaultConstructor) OR_FIELD(DeclaredSpecialMembers) - MATCH_FIELD(ImplicitCopyConstructorHasConstParam) + MATCH_FIELD(ImplicitCopyConstructorCanHaveConstParamForVBase) + MATCH_FIELD(ImplicitCopyConstructorCanHaveConstParamForNonVBase) MATCH_FIELD(ImplicitCopyAssignmentHasConstParam) OR_FIELD(HasDeclaredCopyConstructorWithConstParam) OR_FIELD(HasDeclaredCopyAssignmentWithConstParam) @@ -1669,6 +1749,10 @@ void ASTDeclReader::MergeDefinitionData( // when they occur within the body of a function template specialization). } + if (D->getODRHash() != MergeDD.ODRHash) { + DetectedOdrViolation = true; + } + if (DetectedOdrViolation) Reader.PendingOdrMergeFailures[DD.Definition].push_back(MergeDD.Definition); } @@ -1686,7 +1770,7 @@ void ASTDeclReader::ReadCXXRecordDefinition(CXXRecordDecl *D, bool Update) { else DD = new (C) struct CXXRecordDecl::DefinitionData(D); - ReadCXXDefinitionData(*DD); + ReadCXXDefinitionData(*DD, D); // We might already have a definition for this record. This can happen either // because we're reading an update record, or because we've already done some @@ -1775,6 +1859,10 @@ ASTDeclReader::VisitCXXRecordDeclImpl(CXXRecordDecl *D) { return Redecl; } +void ASTDeclReader::VisitCXXDeductionGuideDecl(CXXDeductionGuideDecl *D) { + VisitFunctionDecl(D); +} + void ASTDeclReader::VisitCXXMethodDecl(CXXMethodDecl *D) { VisitFunctionDecl(D); @@ -1804,8 +1892,6 @@ void ASTDeclReader::VisitCXXConstructorDecl(CXXConstructorDecl *D) { } VisitCXXMethodDecl(D); - - D->IsExplicitSpecified = Record.readInt(); } void ASTDeclReader::VisitCXXDestructorDecl(CXXDestructorDecl *D) { @@ -1821,7 +1907,6 @@ void ASTDeclReader::VisitCXXDestructorDecl(CXXDestructorDecl *D) { void ASTDeclReader::VisitCXXConversionDecl(CXXConversionDecl *D) { VisitCXXMethodDecl(D); - D->IsExplicitSpecified = Record.readInt(); } void ASTDeclReader::VisitImportDecl(ImportDecl *D) { @@ -1831,7 +1916,7 @@ void ASTDeclReader::VisitImportDecl(ImportDecl *D) { SourceLocation *StoredLocs = D->getTrailingObjects<SourceLocation>(); for (unsigned I = 0, N = Record.back(); I != N; ++I) StoredLocs[I] = ReadSourceLocation(); - (void)Record.readInt(); // The number of stored source locations. + Record.skipInts(1); // The number of stored source locations. } void ASTDeclReader::VisitAccessSpecDecl(AccessSpecDecl *D) { @@ -1873,6 +1958,7 @@ DeclID ASTDeclReader::VisitTemplateDecl(TemplateDecl *D) { DeclID PatternID = ReadDeclID(); NamedDecl *TemplatedDecl = cast_or_null<NamedDecl>(Reader.GetDecl(PatternID)); TemplateParameterList *TemplateParams = Record.readTemplateParameterList(); + // FIXME handle associated constraints D->init(TemplatedDecl, TemplateParams); return PatternID; @@ -1917,21 +2003,6 @@ ASTDeclReader::VisitRedeclarableTemplateDecl(RedeclarableTemplateDecl *D) { return Redecl; } -static DeclID *newDeclIDList(ASTContext &Context, DeclID *Old, - SmallVectorImpl<DeclID> &IDs) { - assert(!IDs.empty() && "no IDs to add to list"); - if (Old) { - IDs.insert(IDs.end(), Old + 1, Old + 1 + Old[0]); - std::sort(IDs.begin(), IDs.end()); - IDs.erase(std::unique(IDs.begin(), IDs.end()), IDs.end()); - } - - auto *Result = new (Context) DeclID[1 + IDs.size()]; - *Result = IDs.size(); - std::copy(IDs.begin(), IDs.end(), Result + 1); - return Result; -} - void ASTDeclReader::VisitClassTemplateDecl(ClassTemplateDecl *D) { RedeclarableResult Redecl = VisitRedeclarableTemplateDecl(D); @@ -1940,19 +2011,14 @@ void ASTDeclReader::VisitClassTemplateDecl(ClassTemplateDecl *D) { // the specializations. SmallVector<serialization::DeclID, 32> SpecIDs; ReadDeclIDList(SpecIDs); - - if (!SpecIDs.empty()) { - auto *CommonPtr = D->getCommonPtr(); - CommonPtr->LazySpecializations = newDeclIDList( - Reader.getContext(), CommonPtr->LazySpecializations, SpecIDs); - } + ASTDeclReader::AddLazySpecializations(D, SpecIDs); } if (D->getTemplatedDecl()->TemplateOrInstantiation) { // We were loaded before our templated declaration was. We've not set up // its corresponding type yet (see VisitCXXRecordDeclImpl), so reconstruct // it now. - Reader.Context.getInjectedClassNameType( + Reader.getContext().getInjectedClassNameType( D->getTemplatedDecl(), D->getInjectedClassNameSpecialization()); } } @@ -1972,12 +2038,7 @@ void ASTDeclReader::VisitVarTemplateDecl(VarTemplateDecl *D) { // the specializations. SmallVector<serialization::DeclID, 32> SpecIDs; ReadDeclIDList(SpecIDs); - - if (!SpecIDs.empty()) { - auto *CommonPtr = D->getCommonPtr(); - CommonPtr->LazySpecializations = newDeclIDList( - Reader.getContext(), CommonPtr->LazySpecializations, SpecIDs); - } + ASTDeclReader::AddLazySpecializations(D, SpecIDs); } } @@ -2083,12 +2144,7 @@ void ASTDeclReader::VisitFunctionTemplateDecl(FunctionTemplateDecl *D) { // This FunctionTemplateDecl owns a CommonPtr; read it. SmallVector<serialization::DeclID, 32> SpecIDs; ReadDeclIDList(SpecIDs); - - if (!SpecIDs.empty()) { - auto *CommonPtr = D->getCommonPtr(); - CommonPtr->LazySpecializations = newDeclIDList( - Reader.getContext(), CommonPtr->LazySpecializations, SpecIDs); - } + ASTDeclReader::AddLazySpecializations(D, SpecIDs); } } @@ -2439,8 +2495,8 @@ void ASTDeclReader::mergeMergeable(Mergeable<T> *D) { if (FindExistingResult ExistingRes = findExisting(static_cast<T*>(D))) if (T *Existing = ExistingRes) - Reader.Context.setPrimaryMergedDecl(static_cast<T*>(D), - Existing->getCanonicalDecl()); + Reader.getContext().setPrimaryMergedDecl(static_cast<T *>(D), + Existing->getCanonicalDecl()); } void ASTDeclReader::VisitOMPThreadPrivateDecl(OMPThreadPrivateDecl *D) { @@ -2471,12 +2527,12 @@ void ASTDeclReader::VisitOMPCapturedExprDecl(OMPCapturedExprDecl *D) { //===----------------------------------------------------------------------===// /// \brief Reads attributes from the current stream position. -void ASTReader::ReadAttributes(ModuleFile &F, AttrVec &Attrs, - const RecordData &Record, unsigned &Idx) { - for (unsigned i = 0, e = Record[Idx++]; i != e; ++i) { +void ASTReader::ReadAttributes(ASTRecordReader &Record, AttrVec &Attrs) { + for (unsigned i = 0, e = Record.readInt(); i != e; ++i) { Attr *New = nullptr; - attr::Kind Kind = (attr::Kind)Record[Idx++]; - SourceRange Range = ReadSourceRange(F, Record, Idx); + attr::Kind Kind = (attr::Kind)Record.readInt(); + SourceRange Range = Record.readSourceRange(); + ASTContext &Context = getContext(); #include "clang/Serialization/AttrPCHRead.inc" @@ -2531,7 +2587,11 @@ static bool isConsumerInterestedIn(ASTContext &Ctx, Decl *D, bool HasBody) { Var->isThisDeclarationADefinition() == VarDecl::Definition; if (FunctionDecl *Func = dyn_cast<FunctionDecl>(D)) return Func->doesThisDeclarationHaveABody() || HasBody; - + + if (auto *ES = D->getASTContext().getExternalSource()) + if (ES->hasExternalDefinitions(D) == ExternalASTSource::EK_Never) + return true; + return false; } @@ -2648,6 +2708,45 @@ static bool isSameTemplateParameterList(const TemplateParameterList *X, return true; } +/// Determine whether the attributes we can overload on are identical for A and +/// B. Will ignore any overloadable attrs represented in the type of A and B. +static bool hasSameOverloadableAttrs(const FunctionDecl *A, + const FunctionDecl *B) { + // Note that pass_object_size attributes are represented in the function's + // ExtParameterInfo, so we don't need to check them here. + + SmallVector<const EnableIfAttr *, 4> AEnableIfs; + // Since this is an equality check, we can ignore that enable_if attrs show up + // in reverse order. + for (const auto *EIA : A->specific_attrs<EnableIfAttr>()) + AEnableIfs.push_back(EIA); + + SmallVector<const EnableIfAttr *, 4> BEnableIfs; + for (const auto *EIA : B->specific_attrs<EnableIfAttr>()) + BEnableIfs.push_back(EIA); + + // Two very common cases: either we have 0 enable_if attrs, or we have an + // unequal number of enable_if attrs. + if (AEnableIfs.empty() && BEnableIfs.empty()) + return true; + + if (AEnableIfs.size() != BEnableIfs.size()) + return false; + + llvm::FoldingSetNodeID Cand1ID, Cand2ID; + for (unsigned I = 0, E = AEnableIfs.size(); I != E; ++I) { + Cand1ID.clear(); + Cand2ID.clear(); + + AEnableIfs[I]->getCond()->Profile(Cand1ID, A->getASTContext(), true); + BEnableIfs[I]->getCond()->Profile(Cand2ID, B->getASTContext(), true); + if (Cand1ID != Cand2ID) + return false; + } + + return true; +} + /// \brief Determine whether the two declarations refer to the same entity. static bool isSameEntity(NamedDecl *X, NamedDecl *Y) { assert(X->getDeclName() == Y->getDeclName() && "Declaration name mismatch!"); @@ -2703,8 +2802,24 @@ static bool isSameEntity(NamedDecl *X, NamedDecl *Y) { CtorY->getInheritedConstructor().getConstructor())) return false; } - return (FuncX->getLinkageInternal() == FuncY->getLinkageInternal()) && - FuncX->getASTContext().hasSameType(FuncX->getType(), FuncY->getType()); + ASTContext &C = FuncX->getASTContext(); + if (!C.hasSameType(FuncX->getType(), FuncY->getType())) { + // We can get functions with different types on the redecl chain in C++17 + // if they have differing exception specifications and at least one of + // the excpetion specs is unresolved. + // FIXME: Do we need to check for C++14 deduced return types here too? + auto *XFPT = FuncX->getType()->getAs<FunctionProtoType>(); + auto *YFPT = FuncY->getType()->getAs<FunctionProtoType>(); + if (C.getLangOpts().CPlusPlus1z && XFPT && YFPT && + (isUnresolvedExceptionSpec(XFPT->getExceptionSpecType()) || + isUnresolvedExceptionSpec(YFPT->getExceptionSpecType())) && + C.hasSameFunctionTypeIgnoringExceptionSpec(FuncX->getType(), + FuncY->getType())) + return true; + return false; + } + return FuncX->getLinkageInternal() == FuncY->getLinkageInternal() && + hasSameOverloadableAttrs(FuncX, FuncY); } // Variables with the same type and linkage match. @@ -2816,7 +2931,7 @@ DeclContext *ASTDeclReader::getPrimaryContextForMerging(ASTReader &Reader, // commit to DC being the canonical definition now, and will fix this when // we load the update record. if (!DD) { - DD = new (Reader.Context) struct CXXRecordDecl::DefinitionData(RD); + DD = new (Reader.getContext()) struct CXXRecordDecl::DefinitionData(RD); RD->IsCompleteDefinition = true; RD->DefinitionData = DD; RD->getCanonicalDecl()->DefinitionData = DD; @@ -3261,6 +3376,7 @@ Decl *ASTReader::ReadDeclRecord(DeclID ID) { ASTDeclReader Reader(*this, Record, Loc, ID, DeclLoc); unsigned Code = DeclsCursor.ReadCode(); + ASTContext &Context = getContext(); Decl *D = nullptr; switch ((DeclCode)Record.readRecord(DeclsCursor, Code)) { case DECL_CONTEXT_LEXICAL: @@ -3323,6 +3439,9 @@ Decl *ASTReader::ReadDeclRecord(DeclID ID) { case DECL_CXX_RECORD: D = CXXRecordDecl::CreateDeserialized(Context, ID); break; + case DECL_CXX_DEDUCTION_GUIDE: + D = CXXDeductionGuideDecl::CreateDeserialized(Context, ID); + break; case DECL_CXX_METHOD: D = CXXMethodDecl::CreateDeserialized(Context, ID); break; @@ -3517,7 +3636,8 @@ Decl *ASTReader::ReadDeclRecord(DeclID ID) { assert(Record.getIdx() == Record.size()); // Load any relevant update records. - PendingUpdateRecords.push_back(std::make_pair(ID, D)); + PendingUpdateRecords.push_back( + PendingUpdateRecord(ID, D, /*JustLoaded=*/true)); // Load the categories after recursive loading is finished. if (ObjCInterfaceDecl *Class = dyn_cast<ObjCInterfaceDecl>(D)) @@ -3531,23 +3651,58 @@ Decl *ASTReader::ReadDeclRecord(DeclID ID) { // AST consumer might need to know about, queue it. // We don't pass it to the consumer immediately because we may be in recursive // loading, and some declarations may still be initializing. - if (isConsumerInterestedIn(Context, D, Reader.hasPendingBody())) - InterestingDecls.push_back(D); + PotentiallyInterestingDecls.push_back( + InterestingDecl(D, Reader.hasPendingBody())); return D; } -void ASTReader::loadDeclUpdateRecords(serialization::DeclID ID, Decl *D) { +void ASTReader::PassInterestingDeclsToConsumer() { + assert(Consumer); + + if (PassingDeclsToConsumer) + return; + + // Guard variable to avoid recursively redoing the process of passing + // decls to consumer. + SaveAndRestore<bool> GuardPassingDeclsToConsumer(PassingDeclsToConsumer, + true); + + // Ensure that we've loaded all potentially-interesting declarations + // that need to be eagerly loaded. + for (auto ID : EagerlyDeserializedDecls) + GetDecl(ID); + EagerlyDeserializedDecls.clear(); + + while (!PotentiallyInterestingDecls.empty()) { + InterestingDecl D = PotentiallyInterestingDecls.front(); + PotentiallyInterestingDecls.pop_front(); + if (isConsumerInterestedIn(getContext(), D.getDecl(), D.hasPendingBody())) + PassInterestingDeclToConsumer(D.getDecl()); + } +} + +void ASTReader::loadDeclUpdateRecords(PendingUpdateRecord &Record) { // The declaration may have been modified by files later in the chain. // If this is the case, read the record containing the updates from each file // and pass it to ASTDeclReader to make the modifications. + serialization::GlobalDeclID ID = Record.ID; + Decl *D = Record.D; ProcessingUpdatesRAIIObj ProcessingUpdates(*this); DeclUpdateOffsetsMap::iterator UpdI = DeclUpdateOffsets.find(ID); + + llvm::SmallVector<serialization::DeclID, 8> PendingLazySpecializationIDs; + if (UpdI != DeclUpdateOffsets.end()) { auto UpdateOffsets = std::move(UpdI->second); DeclUpdateOffsets.erase(UpdI); - bool WasInteresting = isConsumerInterestedIn(Context, D, false); + // Check if this decl was interesting to the consumer. If we just loaded + // the declaration, then we know it was interesting and we skip the call + // to isConsumerInterestedIn because it is unsafe to call in the + // current ASTReader state. + bool WasInteresting = + Record.JustLoaded || isConsumerInterestedIn(getContext(), D, false); for (auto &FileAndOffset : UpdateOffsets) { ModuleFile *F = FileAndOffset.first; uint64_t Offset = FileAndOffset.second; @@ -3562,17 +3717,29 @@ void ASTReader::loadDeclUpdateRecords(serialization::DeclID ID, Decl *D) { ASTDeclReader Reader(*this, Record, RecordLocation(F, Offset), ID, SourceLocation()); - Reader.UpdateDecl(D); + Reader.UpdateDecl(D, PendingLazySpecializationIDs); // We might have made this declaration interesting. If so, remember that // we need to hand it off to the consumer. if (!WasInteresting && - isConsumerInterestedIn(Context, D, Reader.hasPendingBody())) { - InterestingDecls.push_back(D); + isConsumerInterestedIn(getContext(), D, Reader.hasPendingBody())) { + PotentiallyInterestingDecls.push_back( + InterestingDecl(D, Reader.hasPendingBody())); WasInteresting = true; } } } + // Add the lazy specializations to the template. + assert((PendingLazySpecializationIDs.empty() || isa<ClassTemplateDecl>(D) || + isa<FunctionTemplateDecl>(D) || isa<VarTemplateDecl>(D)) && + "Must not have pending specializations"); + if (auto *CTD = dyn_cast<ClassTemplateDecl>(D)) + ASTDeclReader::AddLazySpecializations(CTD, PendingLazySpecializationIDs); + else if (auto *FTD = dyn_cast<FunctionTemplateDecl>(D)) + ASTDeclReader::AddLazySpecializations(FTD, PendingLazySpecializationIDs); + else if (auto *VTD = dyn_cast<VarTemplateDecl>(D)) + ASTDeclReader::AddLazySpecializations(VTD, PendingLazySpecializationIDs); + PendingLazySpecializationIDs.clear(); // Load the pending visible updates for this decl context, if it has any. auto I = PendingVisibleUpdates.find(ID); @@ -3769,7 +3936,8 @@ static void forAllLaterRedecls(DeclT *D, Fn F) { } } -void ASTDeclReader::UpdateDecl(Decl *D) { +void ASTDeclReader::UpdateDecl(Decl *D, + llvm::SmallVectorImpl<serialization::DeclID> &PendingLazySpecializationIDs) { while (Record.getIdx() < Record.size()) { switch ((DeclUpdateKind)Record.readInt()) { case UPD_CXX_ADDED_IMPLICIT_MEMBER: { @@ -3785,8 +3953,8 @@ void ASTDeclReader::UpdateDecl(Decl *D) { } case UPD_CXX_ADDED_TEMPLATE_SPECIALIZATION: - // It will be added to the template's specializations set when loaded. - (void)Record.readDecl(); + // It will be added to the template's lazy specialization set. + PendingLazySpecializationIDs.push_back(ReadDeclID()); break; case UPD_CXX_ADDED_ANONYMOUS_NAMESPACE: { @@ -3804,10 +3972,21 @@ void ASTDeclReader::UpdateDecl(Decl *D) { break; } - case UPD_CXX_INSTANTIATED_STATIC_DATA_MEMBER: - cast<VarDecl>(D)->getMemberSpecializationInfo()->setPointOfInstantiation( + case UPD_CXX_INSTANTIATED_STATIC_DATA_MEMBER: { + VarDecl *VD = cast<VarDecl>(D); + VD->getMemberSpecializationInfo()->setPointOfInstantiation( ReadSourceLocation()); + uint64_t Val = Record.readInt(); + if (Val && !VD->getInit()) { + VD->setInit(Record.readExpr()); + if (Val > 1) { // IsInitKnownICE = 1, IsInitNotICE = 2, IsInitICE = 3 + EvaluatedStmt *Eval = VD->ensureEvaluatedStmt(); + Eval->CheckedICE = true; + Eval->IsICE = Val == 3; + } + } break; + } case UPD_CXX_INSTANTIATED_DEFAULT_ARGUMENT: { auto Param = cast<ParmVarDecl>(D); @@ -3858,14 +4037,7 @@ void ASTDeclReader::UpdateDecl(Decl *D) { }); } FD->setInnerLocStart(ReadSourceLocation()); - if (auto *CD = dyn_cast<CXXConstructorDecl>(FD)) { - CD->NumCtorInitializers = Record.readInt(); - if (CD->NumCtorInitializers) - CD->CtorInitializers = ReadGlobalOffset(); - } - // Store the offset of the body so we can lazily load it later. - Reader.PendingBodies[FD] = GetCurrentCursorOffset(); - HasPendingBody = true; + ReadFunctionDefinition(FD); assert(Record.getIdx() == Record.size() && "lazy body must be last"); break; } @@ -3951,7 +4123,7 @@ void ASTDeclReader::UpdateDecl(Decl *D) { // FIXME: If the exception specification is already present, check that it // matches. if (isUnresolvedExceptionSpec(FPT->getExceptionSpecType())) { - FD->setType(Reader.Context.getFunctionType( + FD->setType(Reader.getContext().getFunctionType( FPT->getReturnType(), FPT->getParamTypes(), FPT->getExtProtoInfo().withExceptionSpec(ESI))); @@ -3969,28 +4141,31 @@ void ASTDeclReader::UpdateDecl(Decl *D) { for (auto *Redecl : merged_redecls(D)) { // FIXME: If the return type is already deduced, check that it matches. FunctionDecl *FD = cast<FunctionDecl>(Redecl); - Reader.Context.adjustDeducedFunctionResultType(FD, DeducedResultType); + Reader.getContext().adjustDeducedFunctionResultType(FD, + DeducedResultType); } break; } case UPD_DECL_MARKED_USED: { // Maintain AST consistency: any later redeclarations are used too. - D->markUsed(Reader.Context); + D->markUsed(Reader.getContext()); break; } case UPD_MANGLING_NUMBER: - Reader.Context.setManglingNumber(cast<NamedDecl>(D), Record.readInt()); + Reader.getContext().setManglingNumber(cast<NamedDecl>(D), + Record.readInt()); break; case UPD_STATIC_LOCAL_NUMBER: - Reader.Context.setStaticLocalNumber(cast<VarDecl>(D), Record.readInt()); + Reader.getContext().setStaticLocalNumber(cast<VarDecl>(D), + Record.readInt()); break; case UPD_DECL_MARKED_OPENMP_THREADPRIVATE: - D->addAttr(OMPThreadPrivateDeclAttr::CreateImplicit( - Reader.Context, ReadSourceRange())); + D->addAttr(OMPThreadPrivateDeclAttr::CreateImplicit(Reader.getContext(), + ReadSourceRange())); break; case UPD_DECL_EXPORTED: { @@ -4010,7 +4185,7 @@ void ASTDeclReader::UpdateDecl(Decl *D) { Reader.HiddenNamesMap[Owner].push_back(Exported); } else { // The declaration is now visible. - Exported->Hidden = false; + Exported->setVisibleDespiteOwningModule(); } break; } diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTReaderStmt.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTReaderStmt.cpp index 686a69b..3f5da02 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTReaderStmt.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTReaderStmt.cpp @@ -367,23 +367,45 @@ void ASTStmtReader::VisitMSAsmStmt(MSAsmStmt *S) { } void ASTStmtReader::VisitCoroutineBodyStmt(CoroutineBodyStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + assert(Record.peekInt() == S->NumParams); + Record.skipInts(1); + auto *StoredStmts = S->getStoredStmts(); + for (unsigned i = 0; + i < CoroutineBodyStmt::SubStmt::FirstParamMove + S->NumParams; ++i) + StoredStmts[i] = Record.readSubStmt(); } void ASTStmtReader::VisitCoreturnStmt(CoreturnStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + S->CoreturnLoc = Record.readSourceLocation(); + for (auto &SubStmt: S->SubStmts) + SubStmt = Record.readSubStmt(); + S->IsImplicit = Record.readInt() != 0; } -void ASTStmtReader::VisitCoawaitExpr(CoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtReader::VisitCoawaitExpr(CoawaitExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); + E->OpaqueValue = cast_or_null<OpaqueValueExpr>(Record.readSubStmt()); + E->setIsImplicit(Record.readInt() != 0); +} + +void ASTStmtReader::VisitCoyieldExpr(CoyieldExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); + E->OpaqueValue = cast_or_null<OpaqueValueExpr>(Record.readSubStmt()); } -void ASTStmtReader::VisitCoyieldExpr(CoyieldExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtReader::VisitDependentCoawaitExpr(DependentCoawaitExpr *E) { + VisitExpr(E); + E->KeywordLoc = ReadSourceLocation(); + for (auto &SubExpr: E->SubExprs) + SubExpr = Record.readSubStmt(); } void ASTStmtReader::VisitCapturedStmt(CapturedStmt *S) { @@ -665,7 +687,7 @@ void ASTStmtReader::VisitBinaryOperator(BinaryOperator *E) { E->setRHS(Record.readSubExpr()); E->setOpcode((BinaryOperator::Opcode)Record.readInt()); E->setOperatorLoc(ReadSourceLocation()); - E->setFPContractable((bool)Record.readInt()); + E->setFPFeatures(FPOptions(Record.readInt())); } void ASTStmtReader::VisitCompoundAssignOperator(CompoundAssignOperator *E) { @@ -1220,7 +1242,7 @@ void ASTStmtReader::VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) { VisitCallExpr(E); E->Operator = (OverloadedOperatorKind)Record.readInt(); E->Range = Record.readSourceRange(); - E->setFPContractable((bool)Record.readInt()); + E->setFPFeatures(FPOptions(Record.readInt())); } void ASTStmtReader::VisitCXXConstructExpr(CXXConstructExpr *E) { @@ -1829,6 +1851,9 @@ OMPClause *OMPClauseReader::readClause() { case OMPC_reduction: C = OMPReductionClause::CreateEmpty(Context, Reader->Record.readInt()); break; + case OMPC_task_reduction: + C = OMPTaskReductionClause::CreateEmpty(Context, Reader->Record.readInt()); + break; case OMPC_linear: C = OMPLinearClause::CreateEmpty(Context, Reader->Record.readInt()); break; @@ -1928,7 +1953,8 @@ OMPClause *OMPClauseReader::readClause() { } void OMPClauseReader::VisitOMPClauseWithPreInit(OMPClauseWithPreInit *C) { - C->setPreInitStmt(Reader->Record.readSubStmt()); + C->setPreInitStmt(Reader->Record.readSubStmt(), + static_cast<OpenMPDirectiveKind>(Reader->Record.readInt())); } void OMPClauseReader::VisitOMPClauseWithPostUpdate(OMPClauseWithPostUpdate *C) { @@ -1937,6 +1963,7 @@ void OMPClauseReader::VisitOMPClauseWithPostUpdate(OMPClauseWithPostUpdate *C) { } void OMPClauseReader::VisitOMPIfClause(OMPIfClause *C) { + VisitOMPClauseWithPreInit(C); C->setNameModifier(static_cast<OpenMPDirectiveKind>(Reader->Record.readInt())); C->setNameModifierLoc(Reader->ReadSourceLocation()); C->setColonLoc(Reader->ReadSourceLocation()); @@ -1950,6 +1977,7 @@ void OMPClauseReader::VisitOMPFinalClause(OMPFinalClause *C) { } void OMPClauseReader::VisitOMPNumThreadsClause(OMPNumThreadsClause *C) { + VisitOMPClauseWithPreInit(C); C->setNumThreads(Reader->Record.readSubExpr()); C->setLParenLoc(Reader->ReadSourceLocation()); } @@ -2130,6 +2158,40 @@ void OMPClauseReader::VisitOMPReductionClause(OMPReductionClause *C) { C->setReductionOps(Vars); } +void OMPClauseReader::VisitOMPTaskReductionClause(OMPTaskReductionClause *C) { + VisitOMPClauseWithPostUpdate(C); + C->setLParenLoc(Reader->ReadSourceLocation()); + C->setColonLoc(Reader->ReadSourceLocation()); + NestedNameSpecifierLoc NNSL = Reader->Record.readNestedNameSpecifierLoc(); + DeclarationNameInfo DNI; + Reader->ReadDeclarationNameInfo(DNI); + C->setQualifierLoc(NNSL); + C->setNameInfo(DNI); + + unsigned NumVars = C->varlist_size(); + SmallVector<Expr *, 16> Vars; + Vars.reserve(NumVars); + for (unsigned I = 0; I != NumVars; ++I) + Vars.push_back(Reader->Record.readSubExpr()); + C->setVarRefs(Vars); + Vars.clear(); + for (unsigned I = 0; I != NumVars; ++I) + Vars.push_back(Reader->Record.readSubExpr()); + C->setPrivates(Vars); + Vars.clear(); + for (unsigned I = 0; I != NumVars; ++I) + Vars.push_back(Reader->Record.readSubExpr()); + C->setLHSExprs(Vars); + Vars.clear(); + for (unsigned I = 0; I != NumVars; ++I) + Vars.push_back(Reader->Record.readSubExpr()); + C->setRHSExprs(Vars); + Vars.clear(); + for (unsigned I = 0; I != NumVars; ++I) + Vars.push_back(Reader->Record.readSubExpr()); + C->setReductionOps(Vars); +} + void OMPClauseReader::VisitOMPLinearClause(OMPLinearClause *C) { VisitOMPClauseWithPostUpdate(C); C->setLParenLoc(Reader->ReadSourceLocation()); @@ -2297,11 +2359,13 @@ void OMPClauseReader::VisitOMPMapClause(OMPMapClause *C) { } void OMPClauseReader::VisitOMPNumTeamsClause(OMPNumTeamsClause *C) { + VisitOMPClauseWithPreInit(C); C->setNumTeams(Reader->Record.readSubExpr()); C->setLParenLoc(Reader->ReadSourceLocation()); } void OMPClauseReader::VisitOMPThreadLimitClause(OMPThreadLimitClause *C) { + VisitOMPClauseWithPreInit(C); C->setThreadLimit(Reader->Record.readSubExpr()); C->setLParenLoc(Reader->ReadSourceLocation()); } @@ -2566,6 +2630,15 @@ void ASTStmtReader::VisitOMPLoopDirective(OMPLoopDirective *D) { if (isOpenMPLoopBoundSharingDirective(D->getDirectiveKind())) { D->setPrevLowerBoundVariable(Record.readSubExpr()); D->setPrevUpperBoundVariable(Record.readSubExpr()); + D->setDistInc(Record.readSubExpr()); + D->setPrevEnsureUpperBound(Record.readSubExpr()); + D->setCombinedLowerBoundVariable(Record.readSubExpr()); + D->setCombinedUpperBoundVariable(Record.readSubExpr()); + D->setCombinedEnsureUpperBound(Record.readSubExpr()); + D->setCombinedInit(Record.readSubExpr()); + D->setCombinedCond(Record.readSubExpr()); + D->setCombinedNextLowerBound(Record.readSubExpr()); + D->setCombinedNextUpperBound(Record.readSubExpr()); } SmallVector<Expr *, 4> Sub; unsigned CollapsedNum = D->getCollapsedNumber(); @@ -2690,6 +2763,8 @@ void ASTStmtReader::VisitOMPTaskwaitDirective(OMPTaskwaitDirective *D) { void ASTStmtReader::VisitOMPTaskgroupDirective(OMPTaskgroupDirective *D) { VisitStmt(D); + // The NumClauses field was read in ReadStmtFromStream. + Record.skipInts(1); VisitOMPExecutableDirective(D); } @@ -2909,7 +2984,7 @@ Stmt *ASTReader::ReadStmtFromStream(ModuleFile &F) { llvm::BitstreamCursor &Cursor = F.DeclsCursor; // Map of offset to previously deserialized stmt. The offset points - /// just after the stmt record. + // just after the stmt record. llvm::DenseMap<uint64_t, Stmt *> StmtEntries; #ifndef NDEBUG @@ -2935,6 +3010,7 @@ Stmt *ASTReader::ReadStmtFromStream(ModuleFile &F) { break; } + ASTContext &Context = getContext(); Stmt *S = nullptr; bool Finished = false; bool IsStmtReference = false; @@ -3459,7 +3535,8 @@ Stmt *ASTReader::ReadStmtFromStream(ModuleFile &F) { break; case STMT_OMP_TASKGROUP_DIRECTIVE: - S = OMPTaskgroupDirective::CreateEmpty(Context, Empty); + S = OMPTaskgroupDirective::CreateEmpty( + Context, Record[ASTStmtReader::NumStmtFields], Empty); break; case STMT_OMP_FLUSH_DIRECTIVE: @@ -3887,6 +3964,29 @@ Stmt *ASTReader::ReadStmtFromStream(ModuleFile &F) { S = LambdaExpr::CreateDeserialized(Context, NumCaptures); break; } + + case STMT_COROUTINE_BODY: { + unsigned NumParams = Record[ASTStmtReader::NumStmtFields]; + S = CoroutineBodyStmt::Create(Context, Empty, NumParams); + break; + } + + case STMT_CORETURN: + S = new (Context) CoreturnStmt(Empty); + break; + + case EXPR_COAWAIT: + S = new (Context) CoawaitExpr(Empty); + break; + + case EXPR_COYIELD: + S = new (Context) CoyieldExpr(Empty); + break; + + case EXPR_DEPENDENT_COAWAIT: + S = new (Context) DependentCoawaitExpr(Empty); + break; + } // We hit a STMT_STOP, so we're done with this expression. diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTWriter.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTWriter.cpp index 886523e..128e53b 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTWriter.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTWriter.cpp @@ -18,8 +18,8 @@ #include "clang/AST/ASTContext.h" #include "clang/AST/ASTUnresolvedSet.h" #include "clang/AST/Decl.h" -#include "clang/AST/DeclContextInternals.h" #include "clang/AST/DeclCXX.h" +#include "clang/AST/DeclContextInternals.h" #include "clang/AST/DeclFriend.h" #include "clang/AST/DeclTemplate.h" #include "clang/AST/Expr.h" @@ -33,8 +33,9 @@ #include "clang/Basic/DiagnosticOptions.h" #include "clang/Basic/FileManager.h" #include "clang/Basic/FileSystemOptions.h" -#include "clang/Basic/LangOptions.h" #include "clang/Basic/LLVM.h" +#include "clang/Basic/LangOptions.h" +#include "clang/Basic/MemoryBufferCache.h" #include "clang/Basic/Module.h" #include "clang/Basic/ObjCRuntime.h" #include "clang/Basic/SourceManager.h" @@ -64,20 +65,22 @@ #include "llvm/ADT/Hashing.h" #include "llvm/ADT/IntrusiveRefCntPtr.h" #include "llvm/ADT/Optional.h" +#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/SmallSet.h" #include "llvm/ADT/SmallString.h" -#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/StringExtras.h" #include "llvm/Bitcode/BitCodes.h" #include "llvm/Bitcode/BitstreamWriter.h" #include "llvm/Support/Casting.h" #include "llvm/Support/Compression.h" #include "llvm/Support/EndianStream.h" +#include "llvm/Support/Error.h" #include "llvm/Support/ErrorHandling.h" #include "llvm/Support/MemoryBuffer.h" #include "llvm/Support/OnDiskHashTable.h" #include "llvm/Support/Path.h" #include "llvm/Support/Process.h" +#include "llvm/Support/SHA1.h" #include "llvm/Support/raw_ostream.h" #include <algorithm> #include <cassert> @@ -252,6 +255,7 @@ void ASTTypeWriter::VisitFunctionType(const FunctionType *T) { // FIXME: need to stabilize encoding of calling convention... Record.push_back(C.getCC()); Record.push_back(C.getProducesResult()); + Record.push_back(C.getNoCallerSavedRegs()); if (C.getHasRegParm() || C.getRegParm() || C.getProducesResult()) AbbrevToUse = 0; @@ -348,6 +352,15 @@ void ASTTypeWriter::VisitAutoType(const AutoType *T) { Code = TYPE_AUTO; } +void ASTTypeWriter::VisitDeducedTemplateSpecializationType( + const DeducedTemplateSpecializationType *T) { + Record.AddTemplateName(T->getTemplateName()); + Record.AddTypeRef(T->getDeducedType()); + if (T->getDeducedType().isNull()) + Record.push_back(T->isDependentType()); + Code = TYPE_DEDUCED_TEMPLATE_SPECIALIZATION; +} + void ASTTypeWriter::VisitTagType(const TagType *T) { Record.push_back(T->isDependentType()); Record.AddDeclRef(T->getDecl()->getCanonicalDecl()); @@ -414,8 +427,10 @@ ASTTypeWriter::VisitDependentSizedArrayType(const DependentSizedArrayType *T) { void ASTTypeWriter::VisitDependentSizedExtVectorType( const DependentSizedExtVectorType *T) { - // FIXME: Serialize this type (C++ only) - llvm_unreachable("Cannot serialize dependent sized extended vector types"); + Record.AddTypeRef(T->getElementType()); + Record.AddStmt(T->getSizeExpr()); + Record.AddSourceLocation(T->getAttributeLoc()); + Code = TYPE_DEPENDENT_SIZED_EXT_VECTOR; } void @@ -682,6 +697,11 @@ void TypeLocWriter::VisitAutoTypeLoc(AutoTypeLoc TL) { Record.AddSourceLocation(TL.getNameLoc()); } +void TypeLocWriter::VisitDeducedTemplateSpecializationTypeLoc( + DeducedTemplateSpecializationTypeLoc TL) { + Record.AddSourceLocation(TL.getTemplateNameLoc()); +} + void TypeLocWriter::VisitRecordTypeLoc(RecordTypeLoc TL) { Record.AddSourceLocation(TL.getNameLoc()); } @@ -820,6 +840,7 @@ void ASTWriter::WriteTypeAbbrevs() { Abv->Add(BitCodeAbbrevOp(0)); // RegParm Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 4)); // CC Abv->Add(BitCodeAbbrevOp(0)); // ProducesResult + Abv->Add(BitCodeAbbrevOp(0)); // NoCallerSavedRegs // FunctionProtoType Abv->Add(BitCodeAbbrevOp(0)); // IsVariadic Abv->Add(BitCodeAbbrevOp(0)); // HasTrailingReturn @@ -1001,7 +1022,6 @@ void ASTWriter::WriteBlockInfoBlock() { // Control Block. BLOCK(CONTROL_BLOCK); RECORD(METADATA); - RECORD(SIGNATURE); RECORD(MODULE_NAME); RECORD(MODULE_DIRECTORY); RECORD(MODULE_MAP_FILE); @@ -1014,7 +1034,6 @@ void ASTWriter::WriteBlockInfoBlock() { BLOCK(OPTIONS_BLOCK); RECORD(LANGUAGE_OPTIONS); RECORD(TARGET_OPTIONS); - RECORD(DIAGNOSTIC_OPTIONS); RECORD(FILE_SYSTEM_OPTIONS); RECORD(HEADER_SEARCH_OPTIONS); RECORD(PREPROCESSOR_OPTIONS); @@ -1029,6 +1048,7 @@ void ASTWriter::WriteBlockInfoBlock() { RECORD(IDENTIFIER_OFFSET); RECORD(IDENTIFIER_TABLE); RECORD(EAGERLY_DESERIALIZED_DECLS); + RECORD(MODULAR_CODEGEN_DECLS); RECORD(SPECIAL_TYPES); RECORD(STATISTICS); RECORD(TENTATIVE_DEFINITIONS); @@ -1049,7 +1069,6 @@ void ASTWriter::WriteBlockInfoBlock() { RECORD(UPDATE_VISIBLE); RECORD(DECL_UPDATE_OFFSETS); RECORD(DECL_UPDATES); - RECORD(DIAG_PRAGMA_MAPPINGS); RECORD(CUDA_SPECIAL_DECL_REFS); RECORD(HEADER_SEARCH_TABLE); RECORD(FP_PRAGMA_OPTIONS); @@ -1074,6 +1093,7 @@ void ASTWriter::WriteBlockInfoBlock() { RECORD(UNUSED_LOCAL_TYPEDEF_NAME_CANDIDATES); RECORD(DELETE_EXPRS_TO_ANALYZE); RECORD(CUDA_PRAGMA_FORCE_HOST_DEVICE_DEPTH); + RECORD(PP_CONDITIONAL_STACK); // SourceManager Block. BLOCK(SOURCE_MANAGER_BLOCK); @@ -1242,6 +1262,11 @@ void ASTWriter::WriteBlockInfoBlock() { BLOCK(EXTENSION_BLOCK); RECORD(EXTENSION_METADATA); + BLOCK(UNHASHED_CONTROL_BLOCK); + RECORD(SIGNATURE); + RECORD(DIAGNOSTIC_OPTIONS); + RECORD(DIAG_PRAGMA_MAPPINGS); + #undef RECORD #undef BLOCK Stream.ExitBlock(); @@ -1304,21 +1329,73 @@ adjustFilenameForRelocatableAST(const char *Filename, StringRef BaseDir) { return Filename + Pos; } -static ASTFileSignature getSignature() { - while (true) { - if (ASTFileSignature S = llvm::sys::Process::GetRandomNumber()) - return S; - // Rely on GetRandomNumber to eventually return non-zero... +ASTFileSignature ASTWriter::createSignature(StringRef Bytes) { + // Calculate the hash till start of UNHASHED_CONTROL_BLOCK. + llvm::SHA1 Hasher; + Hasher.update(ArrayRef<uint8_t>(Bytes.bytes_begin(), Bytes.size())); + auto Hash = Hasher.result(); + + // Convert to an array [5*i32]. + ASTFileSignature Signature; + auto LShift = [&](unsigned char Val, unsigned Shift) { + return (uint32_t)Val << Shift; + }; + for (int I = 0; I != 5; ++I) + Signature[I] = LShift(Hash[I * 4 + 0], 24) | LShift(Hash[I * 4 + 1], 16) | + LShift(Hash[I * 4 + 2], 8) | LShift(Hash[I * 4 + 3], 0); + + return Signature; +} + +ASTFileSignature ASTWriter::writeUnhashedControlBlock(Preprocessor &PP, + ASTContext &Context) { + // Flush first to prepare the PCM hash (signature). + Stream.FlushToWord(); + auto StartOfUnhashedControl = Stream.GetCurrentBitNo() >> 3; + + // Enter the block and prepare to write records. + RecordData Record; + Stream.EnterSubblock(UNHASHED_CONTROL_BLOCK_ID, 5); + + // For implicit modules, write the hash of the PCM as its signature. + ASTFileSignature Signature; + if (WritingModule && + PP.getHeaderSearchInfo().getHeaderSearchOpts().ModulesHashContent) { + Signature = createSignature(StringRef(Buffer.begin(), StartOfUnhashedControl)); + Record.append(Signature.begin(), Signature.end()); + Stream.EmitRecord(SIGNATURE, Record); + Record.clear(); } + + // Diagnostic options. + const auto &Diags = Context.getDiagnostics(); + const DiagnosticOptions &DiagOpts = Diags.getDiagnosticOptions(); +#define DIAGOPT(Name, Bits, Default) Record.push_back(DiagOpts.Name); +#define ENUM_DIAGOPT(Name, Type, Bits, Default) \ + Record.push_back(static_cast<unsigned>(DiagOpts.get##Name())); +#include "clang/Basic/DiagnosticOptions.def" + Record.push_back(DiagOpts.Warnings.size()); + for (unsigned I = 0, N = DiagOpts.Warnings.size(); I != N; ++I) + AddString(DiagOpts.Warnings[I], Record); + Record.push_back(DiagOpts.Remarks.size()); + for (unsigned I = 0, N = DiagOpts.Remarks.size(); I != N; ++I) + AddString(DiagOpts.Remarks[I], Record); + // Note: we don't serialize the log or serialization file names, because they + // are generally transient files and will almost always be overridden. + Stream.EmitRecord(DIAGNOSTIC_OPTIONS, Record); + + // Write out the diagnostic/pragma mappings. + WritePragmaDiagnosticMappings(Diags, /* IsModule = */ WritingModule); + + // Leave the options block. + Stream.ExitBlock(); + return Signature; } /// \brief Write the control block. -uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, - ASTContext &Context, - StringRef isysroot, - const std::string &OutputFile) { - ASTFileSignature Signature = 0; - +void ASTWriter::WriteControlBlock(Preprocessor &PP, ASTContext &Context, + StringRef isysroot, + const std::string &OutputFile) { using namespace llvm; Stream.EnterSubblock(CONTROL_BLOCK_ID, 5); RecordData Record; @@ -1345,17 +1422,8 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, Stream.EmitRecordWithBlob(MetadataAbbrevCode, Record, getClangFullRepositoryVersion()); } - if (WritingModule) { - // For implicit modules we output a signature that we can use to ensure - // duplicate module builds don't collide in the cache as their output order - // is non-deterministic. - // FIXME: Remove this when output is deterministic. - if (Context.getLangOpts().ImplicitModules) { - Signature = getSignature(); - RecordData::value_type Record[] = {Signature}; - Stream.EmitRecord(SIGNATURE, Record); - } + if (WritingModule) { // Module name auto Abbrev = std::make_shared<BitCodeAbbrev>(); Abbrev->Add(BitCodeAbbrevOp(MODULE_NAME)); @@ -1394,13 +1462,14 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, } // Module map file - if (WritingModule) { + if (WritingModule && WritingModule->Kind == Module::ModuleMapModule) { Record.clear(); auto &Map = PP.getHeaderSearchInfo().getModuleMap(); - - // Primary module map file. - AddPath(Map.getModuleMapFileForUniquing(WritingModule)->getName(), Record); + AddPath(WritingModule->PresumedModuleMapFile.empty() + ? Map.getModuleMapFileForUniquing(WritingModule)->getName() + : StringRef(WritingModule->PresumedModuleMapFile), + Record); // Additional module map files. if (auto *AdditionalModMaps = @@ -1420,17 +1489,23 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, serialization::ModuleManager &Mgr = Chain->getModuleManager(); Record.clear(); - for (auto *M : Mgr) { + for (ModuleFile &M : Mgr) { // Skip modules that weren't directly imported. - if (!M->isDirectlyImported()) + if (!M.isDirectlyImported()) continue; - Record.push_back((unsigned)M->Kind); // FIXME: Stable encoding - AddSourceLocation(M->ImportLoc, Record); - Record.push_back(M->File->getSize()); - Record.push_back(getTimestampForOutput(M->File)); - Record.push_back(M->Signature); - AddPath(M->FileName, Record); + Record.push_back((unsigned)M.Kind); // FIXME: Stable encoding + AddSourceLocation(M.ImportLoc, Record); + + // If we have calculated signature, there is no need to store + // the size or timestamp. + Record.push_back(M.Signature ? 0 : M.File->getSize()); + Record.push_back(M.Signature ? 0 : getTimestampForOutput(M.File)); + + for (auto I : M.Signature) + Record.push_back(I); + + AddPath(M.FileName, Record); } Stream.EmitRecord(IMPORTS, Record); } @@ -1492,24 +1567,6 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, } Stream.EmitRecord(TARGET_OPTIONS, Record); - // Diagnostic options. - Record.clear(); - const DiagnosticOptions &DiagOpts - = Context.getDiagnostics().getDiagnosticOptions(); -#define DIAGOPT(Name, Bits, Default) Record.push_back(DiagOpts.Name); -#define ENUM_DIAGOPT(Name, Type, Bits, Default) \ - Record.push_back(static_cast<unsigned>(DiagOpts.get##Name())); -#include "clang/Basic/DiagnosticOptions.def" - Record.push_back(DiagOpts.Warnings.size()); - for (unsigned I = 0, N = DiagOpts.Warnings.size(); I != N; ++I) - AddString(DiagOpts.Warnings[I], Record); - Record.push_back(DiagOpts.Remarks.size()); - for (unsigned I = 0, N = DiagOpts.Remarks.size(); I != N; ++I) - AddString(DiagOpts.Remarks[I], Record); - // Note: we don't serialize the log or serialization file names, because they - // are generally transient files and will almost always be overridden. - Stream.EmitRecord(DIAGNOSTIC_OPTIONS, Record); - // File system options. Record.clear(); const FileSystemOptions &FSOpts = @@ -1544,6 +1601,8 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, AddString(HSOpts.ModuleCachePath, Record); AddString(HSOpts.ModuleUserBuildPath, Record); Record.push_back(HSOpts.DisableModuleHash); + Record.push_back(HSOpts.ImplicitModuleMaps); + Record.push_back(HSOpts.ModuleMapFileHomeIsCwd); Record.push_back(HSOpts.UseBuiltinIncludes); Record.push_back(HSOpts.UseStandardSystemIncludes); Record.push_back(HSOpts.UseStandardCXXIncludes); @@ -1623,7 +1682,6 @@ uint64_t ASTWriter::WriteControlBlock(Preprocessor &PP, PP.getHeaderSearchInfo().getHeaderSearchOpts(), PP.getLangOpts().Modules); Stream.ExitBlock(); - return Signature; } namespace { @@ -1634,6 +1692,7 @@ namespace { bool IsSystemFile; bool IsTransient; bool BufferOverridden; + bool IsTopLevelModuleMap; }; } // end anonymous namespace @@ -1652,6 +1711,7 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 32)); // Modification time IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Overridden IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Transient + IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Module map IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // File name unsigned IFAbbrevCode = Stream.EmitAbbrev(std::move(IFAbbrev)); @@ -1666,7 +1726,8 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, // We only care about file entries that were not overridden. if (!SLoc->isFile()) continue; - const SrcMgr::ContentCache *Cache = SLoc->getFile().getContentCache(); + const SrcMgr::FileInfo &File = SLoc->getFile(); + const SrcMgr::ContentCache *Cache = File.getContentCache(); if (!Cache->OrigEntry) continue; @@ -1675,6 +1736,8 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, Entry.IsSystemFile = Cache->IsSystemFile; Entry.IsTransient = Cache->IsTransient; Entry.BufferOverridden = Cache->BufferOverridden; + Entry.IsTopLevelModuleMap = isModuleMap(File.getFileCharacteristic()) && + File.getIncludeLoc().isInvalid(); if (Cache->IsSystemFile) SortedFiles.push_back(Entry); else @@ -1705,7 +1768,8 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, (uint64_t)Entry.File->getSize(), (uint64_t)getTimestampForOutput(Entry.File), Entry.BufferOverridden, - Entry.IsTransient}; + Entry.IsTransient, + Entry.IsTopLevelModuleMap}; EmitRecordWithPath(IFAbbrevCode, Record, Entry.File->getName()); } @@ -1740,7 +1804,7 @@ static unsigned CreateSLocFileAbbrev(llvm::BitstreamWriter &Stream) { Abbrev->Add(BitCodeAbbrevOp(SM_SLOC_FILE_ENTRY)); Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Offset Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Include location - Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2)); // Characteristic + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Characteristic Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Line directives // FileEntry fields. Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6)); // Input File ID @@ -1759,7 +1823,7 @@ static unsigned CreateSLocBufferAbbrev(llvm::BitstreamWriter &Stream) { Abbrev->Add(BitCodeAbbrevOp(SM_SLOC_BUFFER_ENTRY)); Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Offset Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 8)); // Include location - Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2)); // Characteristic + Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Characteristic Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Line directives Abbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // Buffer name blob return Stream.EmitAbbrev(std::move(Abbrev)); @@ -1800,24 +1864,31 @@ namespace { // Trait used for the on-disk hash table of header search information. class HeaderFileInfoTrait { ASTWriter &Writer; - const HeaderSearch &HS; // Keep track of the framework names we've used during serialization. SmallVector<char, 128> FrameworkStringData; llvm::StringMap<unsigned> FrameworkNameOffset; public: - HeaderFileInfoTrait(ASTWriter &Writer, const HeaderSearch &HS) - : Writer(Writer), HS(HS) { } - + HeaderFileInfoTrait(ASTWriter &Writer) : Writer(Writer) {} + struct key_type { - const FileEntry *FE; StringRef Filename; + off_t Size; + time_t ModTime; }; typedef const key_type &key_type_ref; + + using UnresolvedModule = + llvm::PointerIntPair<Module *, 2, ModuleMap::ModuleHeaderRole>; - typedef HeaderFileInfo data_type; + struct data_type { + const HeaderFileInfo &HFI; + ArrayRef<ModuleMap::KnownHeader> KnownHeaders; + UnresolvedModule Unresolved; + }; typedef const data_type &data_type_ref; + typedef unsigned hash_value_type; typedef unsigned offset_type; @@ -1825,8 +1896,7 @@ namespace { // The hash is based only on size/time of the file, so that the reader can // match even when symlinking or excess path elements ("foo/../", "../") // change the form of the name. However, complete path is still the key. - return llvm::hash_combine(key.FE->getSize(), - Writer.getTimestampForOutput(key.FE)); + return llvm::hash_combine(key.Size, key.ModTime); } std::pair<unsigned,unsigned> @@ -1836,68 +1906,74 @@ namespace { unsigned KeyLen = key.Filename.size() + 1 + 8 + 8; LE.write<uint16_t>(KeyLen); unsigned DataLen = 1 + 2 + 4 + 4; - for (auto ModInfo : HS.getModuleMap().findAllModulesForHeader(key.FE)) + for (auto ModInfo : Data.KnownHeaders) if (Writer.getLocalOrImportedSubmoduleID(ModInfo.getModule())) DataLen += 4; + if (Data.Unresolved.getPointer()) + DataLen += 4; LE.write<uint8_t>(DataLen); return std::make_pair(KeyLen, DataLen); } - + void EmitKey(raw_ostream& Out, key_type_ref key, unsigned KeyLen) { using namespace llvm::support; endian::Writer<little> LE(Out); - LE.write<uint64_t>(key.FE->getSize()); + LE.write<uint64_t>(key.Size); KeyLen -= 8; - LE.write<uint64_t>(Writer.getTimestampForOutput(key.FE)); + LE.write<uint64_t>(key.ModTime); KeyLen -= 8; Out.write(key.Filename.data(), KeyLen); } - + void EmitData(raw_ostream &Out, key_type_ref key, data_type_ref Data, unsigned DataLen) { using namespace llvm::support; endian::Writer<little> LE(Out); uint64_t Start = Out.tell(); (void)Start; - unsigned char Flags = (Data.isImport << 4) - | (Data.isPragmaOnce << 3) - | (Data.DirInfo << 1) - | Data.IndexHeaderMapHeader; + unsigned char Flags = (Data.HFI.isImport << 5) + | (Data.HFI.isPragmaOnce << 4) + | (Data.HFI.DirInfo << 1) + | Data.HFI.IndexHeaderMapHeader; LE.write<uint8_t>(Flags); - LE.write<uint16_t>(Data.NumIncludes); + LE.write<uint16_t>(Data.HFI.NumIncludes); - if (!Data.ControllingMacro) - LE.write<uint32_t>(Data.ControllingMacroID); + if (!Data.HFI.ControllingMacro) + LE.write<uint32_t>(Data.HFI.ControllingMacroID); else - LE.write<uint32_t>(Writer.getIdentifierRef(Data.ControllingMacro)); - + LE.write<uint32_t>(Writer.getIdentifierRef(Data.HFI.ControllingMacro)); + unsigned Offset = 0; - if (!Data.Framework.empty()) { + if (!Data.HFI.Framework.empty()) { // If this header refers into a framework, save the framework name. llvm::StringMap<unsigned>::iterator Pos - = FrameworkNameOffset.find(Data.Framework); + = FrameworkNameOffset.find(Data.HFI.Framework); if (Pos == FrameworkNameOffset.end()) { Offset = FrameworkStringData.size() + 1; - FrameworkStringData.append(Data.Framework.begin(), - Data.Framework.end()); + FrameworkStringData.append(Data.HFI.Framework.begin(), + Data.HFI.Framework.end()); FrameworkStringData.push_back(0); - FrameworkNameOffset[Data.Framework] = Offset; + FrameworkNameOffset[Data.HFI.Framework] = Offset; } else Offset = Pos->second; } LE.write<uint32_t>(Offset); - // FIXME: If the header is excluded, we should write out some - // record of that fact. - for (auto ModInfo : HS.getModuleMap().findAllModulesForHeader(key.FE)) { - if (uint32_t ModID = - Writer.getLocalOrImportedSubmoduleID(ModInfo.getModule())) { - uint32_t Value = (ModID << 2) | (unsigned)ModInfo.getRole(); + auto EmitModule = [&](Module *M, ModuleMap::ModuleHeaderRole Role) { + if (uint32_t ModID = Writer.getLocalOrImportedSubmoduleID(M)) { + uint32_t Value = (ModID << 2) | (unsigned)Role; assert((Value >> 2) == ModID && "overflow in header module info"); LE.write<uint32_t>(Value); } - } + }; + + // FIXME: If the header is excluded, we should write out some + // record of that fact. + for (auto ModInfo : Data.KnownHeaders) + EmitModule(ModInfo.getModule(), ModInfo.getRole()); + if (Data.Unresolved.getPointer()) + EmitModule(Data.Unresolved.getPointer(), Data.Unresolved.getInt()); assert(Out.tell() - Start == DataLen && "Wrong data length"); } @@ -1912,16 +1988,72 @@ namespace { /// /// \param HS The header search structure to save. void ASTWriter::WriteHeaderSearch(const HeaderSearch &HS) { + HeaderFileInfoTrait GeneratorTrait(*this); + llvm::OnDiskChainedHashTableGenerator<HeaderFileInfoTrait> Generator; + SmallVector<const char *, 4> SavedStrings; + unsigned NumHeaderSearchEntries = 0; + + // Find all unresolved headers for the current module. We generally will + // have resolved them before we get here, but not necessarily: we might be + // compiling a preprocessed module, where there is no requirement for the + // original files to exist any more. + const HeaderFileInfo Empty; // So we can take a reference. + if (WritingModule) { + llvm::SmallVector<Module *, 16> Worklist(1, WritingModule); + while (!Worklist.empty()) { + Module *M = Worklist.pop_back_val(); + if (!M->isAvailable()) + continue; + + // Map to disk files where possible, to pick up any missing stat + // information. This also means we don't need to check the unresolved + // headers list when emitting resolved headers in the first loop below. + // FIXME: It'd be preferable to avoid doing this if we were given + // sufficient stat information in the module map. + HS.getModuleMap().resolveHeaderDirectives(M); + + // If the file didn't exist, we can still create a module if we were given + // enough information in the module map. + for (auto U : M->MissingHeaders) { + // Check that we were given enough information to build a module + // without this file existing on disk. + if (!U.Size || (!U.ModTime && IncludeTimestamps)) { + PP->Diag(U.FileNameLoc, diag::err_module_no_size_mtime_for_header) + << WritingModule->getFullModuleName() << U.Size.hasValue() + << U.FileName; + continue; + } + + // Form the effective relative pathname for the file. + SmallString<128> Filename(M->Directory->getName()); + llvm::sys::path::append(Filename, U.FileName); + PreparePathForOutput(Filename); + + StringRef FilenameDup = strdup(Filename.c_str()); + SavedStrings.push_back(FilenameDup.data()); + + HeaderFileInfoTrait::key_type Key = { + FilenameDup, *U.Size, IncludeTimestamps ? *U.ModTime : 0 + }; + HeaderFileInfoTrait::data_type Data = { + Empty, {}, {M, ModuleMap::headerKindToRole(U.Kind)} + }; + // FIXME: Deal with cases where there are multiple unresolved header + // directives in different submodules for the same header. + Generator.insert(Key, Data, GeneratorTrait); + ++NumHeaderSearchEntries; + } + + Worklist.append(M->submodule_begin(), M->submodule_end()); + } + } + SmallVector<const FileEntry *, 16> FilesByUID; HS.getFileMgr().GetUniqueIDMapping(FilesByUID); if (FilesByUID.size() > HS.header_file_size()) FilesByUID.resize(HS.header_file_size()); - - HeaderFileInfoTrait GeneratorTrait(*this, HS); - llvm::OnDiskChainedHashTableGenerator<HeaderFileInfoTrait> Generator; - SmallVector<const char *, 4> SavedStrings; - unsigned NumHeaderSearchEntries = 0; + for (unsigned UID = 0, LastUID = FilesByUID.size(); UID != LastUID; ++UID) { const FileEntry *File = FilesByUID[UID]; if (!File) @@ -1948,11 +2080,16 @@ void ASTWriter::WriteHeaderSearch(const HeaderSearch &HS) { SavedStrings.push_back(Filename.data()); } - HeaderFileInfoTrait::key_type key = { File, Filename }; - Generator.insert(key, *HFI, GeneratorTrait); + HeaderFileInfoTrait::key_type Key = { + Filename, File->getSize(), getTimestampForOutput(File) + }; + HeaderFileInfoTrait::data_type Data = { + *HFI, HS.getModuleMap().findAllModulesForHeader(File), {} + }; + Generator.insert(Key, Data, GeneratorTrait); ++NumHeaderSearchEntries; } - + // Create the on-disk hash table in a buffer. SmallString<4096> TableData; uint32_t BucketOffset; @@ -1986,6 +2123,30 @@ void ASTWriter::WriteHeaderSearch(const HeaderSearch &HS) { free(const_cast<char *>(SavedStrings[I])); } +static void emitBlob(llvm::BitstreamWriter &Stream, StringRef Blob, + unsigned SLocBufferBlobCompressedAbbrv, + unsigned SLocBufferBlobAbbrv) { + typedef ASTWriter::RecordData::value_type RecordDataType; + + // Compress the buffer if possible. We expect that almost all PCM + // consumers will not want its contents. + SmallString<0> CompressedBuffer; + if (llvm::zlib::isAvailable()) { + llvm::Error E = llvm::zlib::compress(Blob.drop_back(1), CompressedBuffer); + if (!E) { + RecordDataType Record[] = {SM_SLOC_BUFFER_BLOB_COMPRESSED, + Blob.size() - 1}; + Stream.EmitRecordWithBlob(SLocBufferBlobCompressedAbbrv, Record, + CompressedBuffer); + return; + } + llvm::consumeError(std::move(E)); + } + + RecordDataType Record[] = {SM_SLOC_BUFFER_BLOB}; + Stream.EmitRecordWithBlob(SLocBufferBlobAbbrv, Record, Blob); +} + /// \brief Writes the block containing the serialized form of the /// source manager. /// @@ -2094,20 +2255,8 @@ void ASTWriter::WriteSourceManagerBlock(SourceManager &SourceMgr, const llvm::MemoryBuffer *Buffer = Content->getBuffer(PP.getDiagnostics(), PP.getSourceManager()); StringRef Blob(Buffer->getBufferStart(), Buffer->getBufferSize() + 1); - - // Compress the buffer if possible. We expect that almost all PCM - // consumers will not want its contents. - SmallString<0> CompressedBuffer; - if (llvm::zlib::compress(Blob.drop_back(1), CompressedBuffer) == - llvm::zlib::StatusOK) { - RecordData::value_type Record[] = {SM_SLOC_BUFFER_BLOB_COMPRESSED, - Blob.size() - 1}; - Stream.EmitRecordWithBlob(SLocBufferBlobCompressedAbbrv, Record, - CompressedBuffer); - } else { - RecordData::value_type Record[] = {SM_SLOC_BUFFER_BLOB}; - Stream.EmitRecordWithBlob(SLocBufferBlobAbbrv, Record, Blob); - } + emitBlob(Stream, Blob, SLocBufferBlobCompressedAbbrv, + SLocBufferBlobAbbrv); } } else { // The source location entry is a macro expansion. @@ -2236,6 +2385,18 @@ void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) { Stream.EmitRecord(PP_COUNTER_VALUE, Record); } + if (PP.isRecordingPreamble() && PP.hasRecordedPreamble()) { + assert(!IsModule); + for (const auto &Cond : PP.getPreambleConditionalStack()) { + AddSourceLocation(Cond.IfLoc, Record); + Record.push_back(Cond.WasSkipping); + Record.push_back(Cond.FoundNonSkip); + Record.push_back(Cond.FoundElse); + } + Stream.EmitRecord(PP_CONDITIONAL_STACK, Record); + Record.clear(); + } + // Enter the preprocessor block. Stream.EnterSubblock(PREPROCESSOR_BLOCK_ID, 3); @@ -2347,7 +2508,6 @@ void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) { } AddIdentifierRef(Name, Record); - Record.push_back(inferSubmoduleIDFromLocation(MI->getDefinitionLoc())); AddSourceLocation(MI->getDefinitionLoc(), Record); AddSourceLocation(MI->getDefinitionEndLoc(), Record); Record.push_back(MI->isUsed()); @@ -2361,9 +2521,9 @@ void ASTWriter::WritePreprocessor(const Preprocessor &PP, bool IsModule) { Record.push_back(MI->isC99Varargs()); Record.push_back(MI->isGNUVarargs()); Record.push_back(MI->hasCommaPasting()); - Record.push_back(MI->getNumArgs()); - for (const IdentifierInfo *Arg : MI->args()) - AddIdentifierRef(Arg, Record); + Record.push_back(MI->getNumParams()); + for (const IdentifierInfo *Param : MI->params()) + AddIdentifierRef(Param, Record); } // If we have a detailed preprocessing record, record the macro definition @@ -2516,7 +2676,8 @@ unsigned ASTWriter::getLocalOrImportedSubmoduleID(Module *Mod) { auto *Top = Mod->getTopLevelModule(); if (Top != WritingModule && - !Top->fullModuleNameIs(StringRef(getLangOpts().CurrentModule))) + (getLangOpts().CompilingPCH || + !Top->fullModuleNameIs(StringRef(getLangOpts().CurrentModule)))) return 0; return SubmoduleIDs[Mod] = NextSubmoduleID++; @@ -2629,9 +2790,10 @@ void ASTWriter::WriteSubmodules(Module *WritingModule) { unsigned ConflictAbbrev = Stream.EmitAbbrev(std::move(Abbrev)); // Write the submodule metadata block. - RecordData::value_type Record[] = {getNumberOfModules(WritingModule), - FirstSubmoduleID - - NUM_PREDEF_SUBMODULE_IDS}; + RecordData::value_type Record[] = { + getNumberOfModules(WritingModule), + FirstSubmoduleID - NUM_PREDEF_SUBMODULE_IDS, + (unsigned)WritingModule->Kind}; Stream.EmitRecord(SUBMODULE_METADATA, Record); // Write all of the submodules. @@ -2650,11 +2812,17 @@ void ASTWriter::WriteSubmodules(Module *WritingModule) { // Emit the definition of the block. { - RecordData::value_type Record[] = { - SUBMODULE_DEFINITION, ID, ParentID, Mod->IsFramework, Mod->IsExplicit, - Mod->IsSystem, Mod->IsExternC, Mod->InferSubmodules, - Mod->InferExplicitSubmodules, Mod->InferExportWildcard, - Mod->ConfigMacrosExhaustive}; + RecordData::value_type Record[] = {SUBMODULE_DEFINITION, + ID, + ParentID, + Mod->IsFramework, + Mod->IsExplicit, + Mod->IsSystem, + Mod->IsExternC, + Mod->InferSubmodules, + Mod->InferExplicitSubmodules, + Mod->InferExportWildcard, + Mod->ConfigMacrosExhaustive}; Stream.EmitRecordWithBlob(DefinitionAbbrev, Record, Mod->Name); } @@ -2768,64 +2936,89 @@ void ASTWriter::WriteSubmodules(Module *WritingModule) { "non-imported submodule?"); } -serialization::SubmoduleID -ASTWriter::inferSubmoduleIDFromLocation(SourceLocation Loc) { - if (Loc.isInvalid() || !WritingModule) - return 0; // No submodule - - // Find the module that owns this location. - ModuleMap &ModMap = PP->getHeaderSearchInfo().getModuleMap(); - Module *OwningMod - = ModMap.inferModuleFromLocation(FullSourceLoc(Loc,PP->getSourceManager())); - if (!OwningMod) - return 0; - - // Check whether this submodule is part of our own module. - if (WritingModule != OwningMod && !OwningMod->isSubModuleOf(WritingModule)) - return 0; - - return getSubmoduleID(OwningMod); -} - void ASTWriter::WritePragmaDiagnosticMappings(const DiagnosticsEngine &Diag, bool isModule) { - // Make sure set diagnostic pragmas don't affect the translation unit that - // imports the module. - // FIXME: Make diagnostic pragma sections work properly with modules. - if (isModule) - return; - llvm::SmallDenseMap<const DiagnosticsEngine::DiagState *, unsigned, 64> DiagStateIDMap; unsigned CurrID = 0; - DiagStateIDMap[&Diag.DiagStates.front()] = ++CurrID; // the command-line one. RecordData Record; - for (DiagnosticsEngine::DiagStatePointsTy::const_iterator - I = Diag.DiagStatePoints.begin(), E = Diag.DiagStatePoints.end(); - I != E; ++I) { - const DiagnosticsEngine::DiagStatePoint &point = *I; - if (point.Loc.isInvalid()) - continue; - AddSourceLocation(point.Loc, Record); - unsigned &DiagStateID = DiagStateIDMap[point.State]; + auto EncodeDiagStateFlags = + [](const DiagnosticsEngine::DiagState *DS) -> unsigned { + unsigned Result = (unsigned)DS->ExtBehavior; + for (unsigned Val : + {(unsigned)DS->IgnoreAllWarnings, (unsigned)DS->EnableAllWarnings, + (unsigned)DS->WarningsAsErrors, (unsigned)DS->ErrorsAsFatal, + (unsigned)DS->SuppressSystemWarnings}) + Result = (Result << 1) | Val; + return Result; + }; + + unsigned Flags = EncodeDiagStateFlags(Diag.DiagStatesByLoc.FirstDiagState); + Record.push_back(Flags); + + auto AddDiagState = [&](const DiagnosticsEngine::DiagState *State, + bool IncludeNonPragmaStates) { + // Ensure that the diagnostic state wasn't modified since it was created. + // We will not correctly round-trip this information otherwise. + assert(Flags == EncodeDiagStateFlags(State) && + "diag state flags vary in single AST file"); + + unsigned &DiagStateID = DiagStateIDMap[State]; Record.push_back(DiagStateID); - + if (DiagStateID == 0) { DiagStateID = ++CurrID; - for (const auto &I : *(point.State)) { - if (I.second.isPragma()) { + + // Add a placeholder for the number of mappings. + auto SizeIdx = Record.size(); + Record.emplace_back(); + for (const auto &I : *State) { + if (I.second.isPragma() || IncludeNonPragmaStates) { Record.push_back(I.first); - Record.push_back((unsigned)I.second.getSeverity()); + Record.push_back(I.second.serialize()); } } - Record.push_back(-1); // mark the end of the diag/map pairs for this - // location. + // Update the placeholder. + Record[SizeIdx] = (Record.size() - SizeIdx) / 2; + } + }; + + AddDiagState(Diag.DiagStatesByLoc.FirstDiagState, isModule); + + // Reserve a spot for the number of locations with state transitions. + auto NumLocationsIdx = Record.size(); + Record.emplace_back(); + + // Emit the state transitions. + unsigned NumLocations = 0; + for (auto &FileIDAndFile : Diag.DiagStatesByLoc.Files) { + if (!FileIDAndFile.first.isValid() || + !FileIDAndFile.second.HasLocalTransitions) + continue; + ++NumLocations; + AddSourceLocation(Diag.SourceMgr->getLocForStartOfFile(FileIDAndFile.first), + Record); + Record.push_back(FileIDAndFile.second.StateTransitions.size()); + for (auto &StatePoint : FileIDAndFile.second.StateTransitions) { + Record.push_back(StatePoint.Offset); + AddDiagState(StatePoint.State, false); } } - if (!Record.empty()) - Stream.EmitRecord(DIAG_PRAGMA_MAPPINGS, Record); + // Backpatch the number of locations. + Record[NumLocationsIdx] = NumLocations; + + // Emit CurDiagStateLoc. Do it last in order to match source order. + // + // This also protects against a hypothetical corner case with simulating + // -Werror settings for implicit modules in the ASTReader, where reading + // CurDiagState out of context could change whether warning pragmas are + // treated as errors. + AddSourceLocation(Diag.DiagStatesByLoc.CurDiagStateLoc, Record); + AddDiagState(Diag.DiagStatesByLoc.CurDiagState, false); + + Stream.EmitRecord(DIAG_PRAGMA_MAPPINGS, Record); } //===----------------------------------------------------------------------===// @@ -3559,6 +3752,7 @@ public: case DeclarationName::ObjCOneArgSelector: case DeclarationName::ObjCMultiArgSelector: case DeclarationName::CXXLiteralOperatorName: + case DeclarationName::CXXDeductionGuideName: KeyLen += 4; break; case DeclarationName::CXXOperatorName: @@ -3588,6 +3782,7 @@ public: switch (Name.getKind()) { case DeclarationName::Identifier: case DeclarationName::CXXLiteralOperatorName: + case DeclarationName::CXXDeductionGuideName: LE.write<uint32_t>(Writer.getIdentifierRef(Name.getIdentifier())); return; case DeclarationName::ObjCZeroArgSelector: @@ -3931,7 +4126,7 @@ void ASTWriter::WriteDeclContextVisibleUpdate(const DeclContext *DC) { /// \brief Write an FP_PRAGMA_OPTIONS block for the given FPOptions. void ASTWriter::WriteFPPragmaOptions(const FPOptions &Opts) { - RecordData::value_type Record[] = {Opts.fp_contract}; + RecordData::value_type Record[] = {Opts.getInt()}; Stream.EmitRecord(FP_PRAGMA_OPTIONS, Record); } @@ -4086,6 +4281,25 @@ void ASTWriter::WriteMSPointersToMembersPragmaOptions(Sema &SemaRef) { Stream.EmitRecord(POINTERS_TO_MEMBERS_PRAGMA_OPTIONS, Record); } +/// \brief Write the state of 'pragma pack' at the end of the module. +void ASTWriter::WritePackPragmaOptions(Sema &SemaRef) { + // Don't serialize pragma pack state for modules, since it should only take + // effect on a per-submodule basis. + if (WritingModule) + return; + + RecordData Record; + Record.push_back(SemaRef.PackStack.CurrentValue); + AddSourceLocation(SemaRef.PackStack.CurrentPragmaLocation, Record); + Record.push_back(SemaRef.PackStack.Stack.size()); + for (const auto &StackEntry : SemaRef.PackStack.Stack) { + Record.push_back(StackEntry.Value); + AddSourceLocation(StackEntry.PragmaLocation, Record); + AddString(StackEntry.StackSlotLabel, Record); + } + Stream.EmitRecord(PACK_PRAGMA_OPTIONS, Record); +} + void ASTWriter::WriteModuleFileExtension(Sema &SemaRef, ModuleFileExtensionWriter &Writer) { // Enter the extension block. @@ -4223,9 +4437,11 @@ void ASTWriter::SetSelectorOffset(Selector Sel, uint32_t Offset) { } ASTWriter::ASTWriter(llvm::BitstreamWriter &Stream, + SmallVectorImpl<char> &Buffer, MemoryBufferCache &PCMCache, ArrayRef<std::shared_ptr<ModuleFileExtension>> Extensions, bool IncludeTimestamps) - : Stream(Stream), IncludeTimestamps(IncludeTimestamps) { + : Stream(Stream), Buffer(Buffer), PCMCache(PCMCache), + IncludeTimestamps(IncludeTimestamps) { for (const auto &Ext : Extensions) { if (auto Writer = Ext->createExtensionWriter(*this)) ModuleFileExtensionWriters.push_back(std::move(Writer)); @@ -4245,9 +4461,10 @@ time_t ASTWriter::getTimestampForOutput(const FileEntry *E) const { return IncludeTimestamps ? E->getModificationTime() : 0; } -uint64_t ASTWriter::WriteAST(Sema &SemaRef, const std::string &OutputFile, - Module *WritingModule, StringRef isysroot, - bool hasErrors) { +ASTFileSignature ASTWriter::WriteAST(Sema &SemaRef, + const std::string &OutputFile, + Module *WritingModule, StringRef isysroot, + bool hasErrors) { WritingAST = true; ASTHasCompilerErrors = hasErrors; @@ -4271,6 +4488,12 @@ uint64_t ASTWriter::WriteAST(Sema &SemaRef, const std::string &OutputFile, this->BaseDirectory.clear(); WritingAST = false; + if (SemaRef.Context.getLangOpts().ImplicitModules && WritingModule) { + // Construct MemoryBuffer and update buffer manager. + PCMCache.addBuffer(OutputFile, + llvm::MemoryBuffer::getMemBufferCopy( + StringRef(Buffer.begin(), Buffer.size()))); + } return Signature; } @@ -4283,9 +4506,9 @@ static void AddLazyVectorDecls(ASTWriter &Writer, Vector &Vec, } } -uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, - const std::string &OutputFile, - Module *WritingModule) { +ASTFileSignature ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, + const std::string &OutputFile, + Module *WritingModule) { using namespace llvm; bool isModule = WritingModule != nullptr; @@ -4433,7 +4656,7 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, } // Write the control block - uint64_t Signature = WriteControlBlock(PP, Context, isysroot, OutputFile); + WriteControlBlock(PP, Context, isysroot, OutputFile); // Write the remaining AST contents. Stream.EnterSubblock(AST_BLOCK_ID, 5); @@ -4573,10 +4796,10 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, SmallString<2048> Buffer; { llvm::raw_svector_ostream Out(Buffer); - for (ModuleFile *M : Chain->ModuleMgr) { + for (ModuleFile &M : Chain->ModuleMgr) { using namespace llvm::support; endian::Writer<little> LE(Out); - StringRef FileName = M->FileName; + StringRef FileName = M.FileName; LE.write<uint16_t>(FileName.size()); Out.write(FileName.data(), FileName.size()); @@ -4594,15 +4817,15 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, // These values should be unique within a chain, since they will be read // as keys into ContinuousRangeMaps. - writeBaseIDOrNone(M->SLocEntryBaseOffset, M->LocalNumSLocEntries); - writeBaseIDOrNone(M->BaseIdentifierID, M->LocalNumIdentifiers); - writeBaseIDOrNone(M->BaseMacroID, M->LocalNumMacros); - writeBaseIDOrNone(M->BasePreprocessedEntityID, - M->NumPreprocessedEntities); - writeBaseIDOrNone(M->BaseSubmoduleID, M->LocalNumSubmodules); - writeBaseIDOrNone(M->BaseSelectorID, M->LocalNumSelectors); - writeBaseIDOrNone(M->BaseDeclID, M->LocalNumDecls); - writeBaseIDOrNone(M->BaseTypeIndex, M->LocalNumTypes); + writeBaseIDOrNone(M.SLocEntryBaseOffset, M.LocalNumSLocEntries); + writeBaseIDOrNone(M.BaseIdentifierID, M.LocalNumIdentifiers); + writeBaseIDOrNone(M.BaseMacroID, M.LocalNumMacros); + writeBaseIDOrNone(M.BasePreprocessedEntityID, + M.NumPreprocessedEntities); + writeBaseIDOrNone(M.BaseSubmoduleID, M.LocalNumSubmodules); + writeBaseIDOrNone(M.BaseSelectorID, M.LocalNumSelectors); + writeBaseIDOrNone(M.BaseDeclID, M.LocalNumDecls); + writeBaseIDOrNone(M.BaseTypeIndex, M.LocalNumTypes); } } RecordData::value_type Record[] = {MODULE_OFFSET_MAP}; @@ -4650,7 +4873,6 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, WriteOpenCLExtensionTypes(SemaRef); WriteOpenCLExtensionDecls(SemaRef); WriteCUDAPragmas(SemaRef); - WritePragmaDiagnosticMappings(Context.getDiagnostics(), isModule); // If we're emitting a module, write out the submodule information. if (WritingModule) @@ -4662,6 +4884,9 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, if (!EagerlyDeserializedDecls.empty()) Stream.EmitRecord(EAGERLY_DESERIALIZED_DECLS, EagerlyDeserializedDecls); + if (!ModularCodegenDecls.empty()) + Stream.EmitRecord(MODULAR_CODEGEN_DECLS, ModularCodegenDecls); + // Write the record containing tentative definitions. if (!TentativeDefinitions.empty()) Stream.EmitRecord(TENTATIVE_DEFINITIONS, TentativeDefinitions); @@ -4765,6 +4990,7 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, WriteMSStructPragmaOptions(SemaRef); WriteMSPointersToMembersPragmaOptions(SemaRef); } + WritePackPragmaOptions(SemaRef); // Some simple statistics RecordData::value_type Record[] = { @@ -4776,7 +5002,7 @@ uint64_t ASTWriter::WriteASTCore(Sema &SemaRef, StringRef isysroot, for (const auto &ExtWriter : ModuleFileExtensionWriters) WriteModuleFileExtension(SemaRef, *ExtWriter); - return Signature; + return writeUnhashedControlBlock(PP, Context); } void ASTWriter::WriteDeclUpdatesBlocks(RecordDataImpl &OffsetsRecord) { @@ -4813,9 +5039,18 @@ void ASTWriter::WriteDeclUpdatesBlocks(RecordDataImpl &OffsetsRecord) { case UPD_CXX_ADDED_FUNCTION_DEFINITION: break; - case UPD_CXX_INSTANTIATED_STATIC_DATA_MEMBER: + case UPD_CXX_INSTANTIATED_STATIC_DATA_MEMBER: { + const VarDecl *VD = cast<VarDecl>(D); Record.AddSourceLocation(Update.getLoc()); + if (VD->getInit()) { + Record.push_back(!VD->isInitKnownICE() ? 1 + : (VD->isInitICE() ? 3 : 2)); + Record.AddStmt(const_cast<Expr*>(VD->getInit())); + } else { + Record.push_back(0); + } break; + } case UPD_CXX_INSTANTIATED_DEFAULT_ARGUMENT: Record.AddStmt(const_cast<Expr *>( @@ -5229,6 +5464,10 @@ void ASTRecordWriter::AddDeclarationName(DeclarationName Name) { AddTypeRef(Name.getCXXNameType()); break; + case DeclarationName::CXXDeductionGuideName: + AddDeclRef(Name.getCXXDeductionGuideTemplate()); + break; + case DeclarationName::CXXOperatorName: Record->push_back(Name.getCXXOverloadedOperator()); break; @@ -5290,6 +5529,7 @@ void ASTRecordWriter::AddDeclarationNameLoc(const DeclarationNameLoc &DNLoc, case DeclarationName::ObjCOneArgSelector: case DeclarationName::ObjCMultiArgSelector: case DeclarationName::CXXUsingDirective: + case DeclarationName::CXXDeductionGuideName: break; } } @@ -5634,9 +5874,11 @@ void ASTRecordWriter::AddCXXDefinitionData(const CXXRecordDecl *D) { Record->push_back(Data.HasUninitializedFields); Record->push_back(Data.HasInheritedConstructor); Record->push_back(Data.HasInheritedAssignment); + Record->push_back(Data.NeedOverloadResolutionForCopyConstructor); Record->push_back(Data.NeedOverloadResolutionForMoveConstructor); Record->push_back(Data.NeedOverloadResolutionForMoveAssignment); Record->push_back(Data.NeedOverloadResolutionForDestructor); + Record->push_back(Data.DefaultedCopyConstructorIsDeleted); Record->push_back(Data.DefaultedMoveConstructorIsDeleted); Record->push_back(Data.DefaultedMoveAssignmentIsDeleted); Record->push_back(Data.DefaultedDestructorIsDeleted); @@ -5645,16 +5887,27 @@ void ASTRecordWriter::AddCXXDefinitionData(const CXXRecordDecl *D) { Record->push_back(Data.HasIrrelevantDestructor); Record->push_back(Data.HasConstexprNonCopyMoveConstructor); Record->push_back(Data.HasDefaultedDefaultConstructor); + Record->push_back(Data.CanPassInRegisters); Record->push_back(Data.DefaultedDefaultConstructorIsConstexpr); Record->push_back(Data.HasConstexprDefaultConstructor); Record->push_back(Data.HasNonLiteralTypeFieldsOrBases); Record->push_back(Data.ComputedVisibleConversions); Record->push_back(Data.UserProvidedDefaultConstructor); Record->push_back(Data.DeclaredSpecialMembers); - Record->push_back(Data.ImplicitCopyConstructorHasConstParam); + Record->push_back(Data.ImplicitCopyConstructorCanHaveConstParamForVBase); + Record->push_back(Data.ImplicitCopyConstructorCanHaveConstParamForNonVBase); Record->push_back(Data.ImplicitCopyAssignmentHasConstParam); Record->push_back(Data.HasDeclaredCopyConstructorWithConstParam); Record->push_back(Data.HasDeclaredCopyAssignmentWithConstParam); + + // getODRHash will compute the ODRHash if it has not been previously computed. + Record->push_back(D->getODRHash()); + bool ModulesDebugInfo = Writer->Context->getLangOpts().ModulesDebugInfo && + Writer->WritingModule && !D->isDependentType(); + Record->push_back(ModulesDebugInfo); + if (ModulesDebugInfo) + Writer->ModularCodegenDecls.push_back(Writer->GetDeclRef(D)); + // IsLambda bit is already saved. Record->push_back(Data.NumBases); diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTWriterDecl.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTWriterDecl.cpp index d8466e9..ec21ca2 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTWriterDecl.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTWriterDecl.cpp @@ -86,6 +86,7 @@ namespace clang { void VisitUnresolvedUsingValueDecl(UnresolvedUsingValueDecl *D); void VisitDeclaratorDecl(DeclaratorDecl *D); void VisitFunctionDecl(FunctionDecl *D); + void VisitCXXDeductionGuideDecl(CXXDeductionGuideDecl *D); void VisitCXXMethodDecl(CXXMethodDecl *D); void VisitCXXConstructorDecl(CXXConstructorDecl *D); void VisitCXXDestructorDecl(CXXDestructorDecl *D); @@ -298,7 +299,7 @@ void ASTDeclWriter::VisitDecl(Decl *D) { Record.push_back(D->isTopLevelDeclInObjCContainer()); Record.push_back(D->getAccess()); Record.push_back(D->isModulePrivate()); - Record.push_back(Writer.inferSubmoduleIDFromLocation(D->getLocation())); + Record.push_back(Writer.getSubmoduleID(D->getOwningModule())); // If this declaration injected a name into a context different from its // lexical context, and that context is an imported namespace, we need to @@ -368,6 +369,7 @@ void ASTDeclWriter::VisitTypedefNameDecl(TypedefNameDecl *D) { Record.push_back(D->isModed()); if (D->isModed()) Record.AddTypeRef(D->getUnderlyingType()); + Record.AddDeclRef(D->getAnonDeclWithTypedefName(false)); } void ASTDeclWriter::VisitTypedefDecl(TypedefDecl *D) { @@ -519,6 +521,7 @@ void ASTDeclWriter::VisitFunctionDecl(FunctionDecl *D) { Record.push_back((int)D->SClass); // FIXME: stable encoding Record.push_back(D->IsInline); Record.push_back(D->IsInlineSpecified); + Record.push_back(D->IsExplicitSpecified); Record.push_back(D->IsVirtualAsWritten); Record.push_back(D->IsPure); Record.push_back(D->HasInheritedPrototype); @@ -607,6 +610,11 @@ void ASTDeclWriter::VisitFunctionDecl(FunctionDecl *D) { Code = serialization::DECL_FUNCTION; } +void ASTDeclWriter::VisitCXXDeductionGuideDecl(CXXDeductionGuideDecl *D) { + VisitFunctionDecl(D); + Code = serialization::DECL_CXX_DEDUCTION_GUIDE; +} + void ASTDeclWriter::VisitObjCMethodDecl(ObjCMethodDecl *D) { VisitNamedDecl(D); // FIXME: convert to LazyStmtPtr? @@ -791,7 +799,9 @@ void ASTDeclWriter::VisitObjCPropertyDecl(ObjCPropertyDecl *D) { // FIXME: stable encoding Record.push_back((unsigned)D->getPropertyImplementation()); Record.AddDeclarationName(D->getGetterName()); + Record.AddSourceLocation(D->getGetterNameLoc()); Record.AddDeclarationName(D->getSetterName()); + Record.AddSourceLocation(D->getSetterNameLoc()); Record.AddDeclRef(D->getGetterMethodDecl()); Record.AddDeclRef(D->getSetterMethodDecl()); Record.AddDeclRef(D->getPropertyIvarDecl()); @@ -806,7 +816,6 @@ void ASTDeclWriter::VisitObjCImplDecl(ObjCImplDecl *D) { void ASTDeclWriter::VisitObjCCategoryImplDecl(ObjCCategoryImplDecl *D) { VisitObjCImplDecl(D); - Record.AddIdentifierRef(D->getIdentifier()); Record.AddSourceLocation(D->getCategoryNameLoc()); Code = serialization::DECL_OBJC_CATEGORY_IMPL; } @@ -906,6 +915,10 @@ void ASTDeclWriter::VisitVarDecl(VarDecl *D) { Record.push_back(D->isConstexpr()); Record.push_back(D->isInitCapture()); Record.push_back(D->isPreviousDeclInSameBlockScope()); + if (const auto *IPD = dyn_cast<ImplicitParamDecl>(D)) + Record.push_back(static_cast<unsigned>(IPD->getParameterKind())); + else + Record.push_back(0); } Record.push_back(D->getLinkageInternal()); @@ -1268,8 +1281,6 @@ void ASTDeclWriter::VisitCXXConstructorDecl(CXXConstructorDecl *D) { VisitCXXMethodDecl(D); - Record.push_back(D->IsExplicitSpecified); - Code = D->isInheritingConstructor() ? serialization::DECL_CXX_INHERITED_CONSTRUCTOR : serialization::DECL_CXX_CONSTRUCTOR; @@ -1285,7 +1296,6 @@ void ASTDeclWriter::VisitCXXDestructorDecl(CXXDestructorDecl *D) { void ASTDeclWriter::VisitCXXConversionDecl(CXXConversionDecl *D) { VisitCXXMethodDecl(D); - Record.push_back(D->IsExplicitSpecified); Code = serialization::DECL_CXX_CONVERSION; } @@ -1983,6 +1993,7 @@ void ASTWriter::WriteDeclAbbrevs() { Abv->Add(BitCodeAbbrevOp(0)); // isConstexpr Abv->Add(BitCodeAbbrevOp(0)); // isInitCapture Abv->Add(BitCodeAbbrevOp(0)); // isPrevDeclInSameScope + Abv->Add(BitCodeAbbrevOp(0)); // ImplicitParamKind Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // Linkage Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2)); // IsInitICE (local) Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 2)); // VarKind (local enum) @@ -2023,6 +2034,7 @@ void ASTWriter::WriteDeclAbbrevs() { Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 3)); // StorageClass Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Inline Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // InlineSpecified + Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // ExplicitSpecified Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // VirtualAsWritten Abv->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Fixed, 1)); // Pure Abv->Add(BitCodeAbbrevOp(0)); // HasInheritedProto @@ -2221,6 +2233,21 @@ void ASTRecordWriter::AddFunctionDefinition(const FunctionDecl *FD) { Writer->ClearSwitchCaseIDs(); assert(FD->doesThisDeclarationHaveABody()); + bool ModulesCodegen = false; + if (Writer->WritingModule && !FD->isDependentContext()) { + // Under -fmodules-codegen, codegen is performed for all defined functions. + // When building a C++ Modules TS module interface unit, a strong definition + // in the module interface is provided by the compilation of that module + // interface unit, not by its users. (Inline functions are still emitted + // in module users.) + ModulesCodegen = + Writer->Context->getLangOpts().ModulesCodegen || + (Writer->WritingModule->Kind == Module::ModuleInterfaceUnit && + Writer->Context->GetGVALinkageForFunction(FD) == GVA_StrongExternal); + } + Record->push_back(ModulesCodegen); + if (ModulesCodegen) + Writer->ModularCodegenDecls.push_back(Writer->GetDeclRef(FD)); if (auto *CD = dyn_cast<CXXConstructorDecl>(FD)) { Record->push_back(CD->getNumCtorInitializers()); if (CD->getNumCtorInitializers()) diff --git a/contrib/llvm/tools/clang/lib/Serialization/ASTWriterStmt.cpp b/contrib/llvm/tools/clang/lib/Serialization/ASTWriterStmt.cpp index 01fd98c..6971339 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ASTWriterStmt.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ASTWriterStmt.cpp @@ -286,7 +286,7 @@ void ASTStmtWriter::VisitMSAsmStmt(MSAsmStmt *S) { } // Outputs - for (unsigned I = 0, N = S->getNumOutputs(); I != N; ++I) { + for (unsigned I = 0, N = S->getNumOutputs(); I != N; ++I) { Record.AddStmt(S->getOutputExpr(I)); Record.AddString(S->getOutputConstraint(I)); } @@ -300,24 +300,48 @@ void ASTStmtWriter::VisitMSAsmStmt(MSAsmStmt *S) { Code = serialization::STMT_MSASM; } -void ASTStmtWriter::VisitCoroutineBodyStmt(CoroutineBodyStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoroutineBodyStmt(CoroutineBodyStmt *CoroStmt) { + VisitStmt(CoroStmt); + Record.push_back(CoroStmt->getParamMoves().size()); + for (Stmt *S : CoroStmt->children()) + Record.AddStmt(S); + Code = serialization::STMT_COROUTINE_BODY; } void ASTStmtWriter::VisitCoreturnStmt(CoreturnStmt *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); + VisitStmt(S); + Record.AddSourceLocation(S->getKeywordLoc()); + Record.AddStmt(S->getOperand()); + Record.AddStmt(S->getPromiseCall()); + Record.push_back(S->isImplicit()); + Code = serialization::STMT_CORETURN; } -void ASTStmtWriter::VisitCoawaitExpr(CoawaitExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoroutineSuspendExpr(CoroutineSuspendExpr *E) { + VisitExpr(E); + Record.AddSourceLocation(E->getKeywordLoc()); + for (Stmt *S : E->children()) + Record.AddStmt(S); + Record.AddStmt(E->getOpaqueValue()); } -void ASTStmtWriter::VisitCoyieldExpr(CoyieldExpr *S) { - // FIXME: Implement coroutine serialization. - llvm_unreachable("unimplemented"); +void ASTStmtWriter::VisitCoawaitExpr(CoawaitExpr *E) { + VisitCoroutineSuspendExpr(E); + Record.push_back(E->isImplicit()); + Code = serialization::EXPR_COAWAIT; +} + +void ASTStmtWriter::VisitCoyieldExpr(CoyieldExpr *E) { + VisitCoroutineSuspendExpr(E); + Code = serialization::EXPR_COYIELD; +} + +void ASTStmtWriter::VisitDependentCoawaitExpr(DependentCoawaitExpr *E) { + VisitExpr(E); + Record.AddSourceLocation(E->getKeywordLoc()); + for (Stmt *S : E->children()) + Record.AddStmt(S); + Code = serialization::EXPR_DEPENDENT_COAWAIT; } void ASTStmtWriter::VisitCapturedStmt(CapturedStmt *S) { @@ -645,7 +669,7 @@ void ASTStmtWriter::VisitBinaryOperator(BinaryOperator *E) { Record.AddStmt(E->getRHS()); Record.push_back(E->getOpcode()); // FIXME: stable encoding Record.AddSourceLocation(E->getOperatorLoc()); - Record.push_back(E->isFPContractable()); + Record.push_back(E->getFPFeatures().getInt()); Code = serialization::EXPR_BINARY_OPERATOR; } @@ -1213,7 +1237,7 @@ void ASTStmtWriter::VisitCXXOperatorCallExpr(CXXOperatorCallExpr *E) { VisitCallExpr(E); Record.push_back(E->getOperator()); Record.AddSourceRange(E->Range); - Record.push_back(E->isFPContractable()); + Record.push_back(E->getFPFeatures().getInt()); Code = serialization::EXPR_CXX_OPERATOR_CALL; } @@ -1794,6 +1818,7 @@ void OMPClauseWriter::writeClause(OMPClause *C) { } void OMPClauseWriter::VisitOMPClauseWithPreInit(OMPClauseWithPreInit *C) { + Record.push_back(C->getCaptureRegion()); Record.AddStmt(C->getPreInitStmt()); } @@ -1803,6 +1828,7 @@ void OMPClauseWriter::VisitOMPClauseWithPostUpdate(OMPClauseWithPostUpdate *C) { } void OMPClauseWriter::VisitOMPIfClause(OMPIfClause *C) { + VisitOMPClauseWithPreInit(C); Record.push_back(C->getNameModifier()); Record.AddSourceLocation(C->getNameModifierLoc()); Record.AddSourceLocation(C->getColonLoc()); @@ -1816,6 +1842,7 @@ void OMPClauseWriter::VisitOMPFinalClause(OMPFinalClause *C) { } void OMPClauseWriter::VisitOMPNumThreadsClause(OMPNumThreadsClause *C) { + VisitOMPClauseWithPreInit(C); Record.AddStmt(C->getNumThreads()); Record.AddSourceLocation(C->getLParenLoc()); } @@ -1955,6 +1982,25 @@ void OMPClauseWriter::VisitOMPReductionClause(OMPReductionClause *C) { Record.AddStmt(E); } +void OMPClauseWriter::VisitOMPTaskReductionClause(OMPTaskReductionClause *C) { + Record.push_back(C->varlist_size()); + VisitOMPClauseWithPostUpdate(C); + Record.AddSourceLocation(C->getLParenLoc()); + Record.AddSourceLocation(C->getColonLoc()); + Record.AddNestedNameSpecifierLoc(C->getQualifierLoc()); + Record.AddDeclarationNameInfo(C->getNameInfo()); + for (auto *VE : C->varlists()) + Record.AddStmt(VE); + for (auto *VE : C->privates()) + Record.AddStmt(VE); + for (auto *E : C->lhs_exprs()) + Record.AddStmt(E); + for (auto *E : C->rhs_exprs()) + Record.AddStmt(E); + for (auto *E : C->reduction_ops()) + Record.AddStmt(E); +} + void OMPClauseWriter::VisitOMPLinearClause(OMPLinearClause *C) { Record.push_back(C->varlist_size()); VisitOMPClauseWithPostUpdate(C); @@ -2064,11 +2110,13 @@ void OMPClauseWriter::VisitOMPMapClause(OMPMapClause *C) { } void OMPClauseWriter::VisitOMPNumTeamsClause(OMPNumTeamsClause *C) { + VisitOMPClauseWithPreInit(C); Record.AddStmt(C->getNumTeams()); Record.AddSourceLocation(C->getLParenLoc()); } void OMPClauseWriter::VisitOMPThreadLimitClause(OMPThreadLimitClause *C) { + VisitOMPClauseWithPreInit(C); Record.AddStmt(C->getThreadLimit()); Record.AddSourceLocation(C->getLParenLoc()); } @@ -2236,6 +2284,15 @@ void ASTStmtWriter::VisitOMPLoopDirective(OMPLoopDirective *D) { if (isOpenMPLoopBoundSharingDirective(D->getDirectiveKind())) { Record.AddStmt(D->getPrevLowerBoundVariable()); Record.AddStmt(D->getPrevUpperBoundVariable()); + Record.AddStmt(D->getDistInc()); + Record.AddStmt(D->getPrevEnsureUpperBound()); + Record.AddStmt(D->getCombinedLowerBoundVariable()); + Record.AddStmt(D->getCombinedUpperBoundVariable()); + Record.AddStmt(D->getCombinedEnsureUpperBound()); + Record.AddStmt(D->getCombinedInit()); + Record.AddStmt(D->getCombinedCond()); + Record.AddStmt(D->getCombinedNextLowerBound()); + Record.AddStmt(D->getCombinedNextUpperBound()); } for (auto I : D->counters()) { Record.AddStmt(I); @@ -2421,6 +2478,7 @@ void ASTStmtWriter::VisitOMPTaskwaitDirective(OMPTaskwaitDirective *D) { void ASTStmtWriter::VisitOMPTaskgroupDirective(OMPTaskgroupDirective *D) { VisitStmt(D); + Record.push_back(D->getNumClauses()); VisitOMPExecutableDirective(D); Code = serialization::STMT_OMP_TASKGROUP_DIRECTIVE; } diff --git a/contrib/llvm/tools/clang/lib/Serialization/GeneratePCH.cpp b/contrib/llvm/tools/clang/lib/Serialization/GeneratePCH.cpp index 7f1b750..2e00765 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/GeneratePCH.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/GeneratePCH.cpp @@ -27,10 +27,11 @@ PCHGenerator::PCHGenerator( ArrayRef<std::shared_ptr<ModuleFileExtension>> Extensions, bool AllowASTWithErrors, bool IncludeTimestamps) : PP(PP), OutputFile(OutputFile), isysroot(isysroot.str()), - SemaPtr(nullptr), Buffer(Buffer), Stream(Buffer->Data), - Writer(Stream, Extensions, IncludeTimestamps), + SemaPtr(nullptr), Buffer(std::move(Buffer)), Stream(this->Buffer->Data), + Writer(Stream, this->Buffer->Data, PP.getPCMCache(), Extensions, + IncludeTimestamps), AllowASTWithErrors(AllowASTWithErrors) { - Buffer->IsComplete = false; + this->Buffer->IsComplete = false; } PCHGenerator::~PCHGenerator() { diff --git a/contrib/llvm/tools/clang/lib/Serialization/GlobalModuleIndex.cpp b/contrib/llvm/tools/clang/lib/Serialization/GlobalModuleIndex.cpp index ae5796e..6978e7e 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/GlobalModuleIndex.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/GlobalModuleIndex.cpp @@ -376,6 +376,15 @@ namespace { /// \brief The set of modules on which this module depends. Each entry is /// a module ID. SmallVector<unsigned, 4> Dependencies; + ASTFileSignature Signature; + }; + + struct ImportedModuleFileInfo { + off_t StoredSize; + time_t StoredModTime; + ASTFileSignature StoredSignature; + ImportedModuleFileInfo(off_t Size, time_t ModTime, ASTFileSignature Sig) + : StoredSize(Size), StoredModTime(ModTime), StoredSignature(Sig) {} }; /// \brief Builder that generates the global module index file. @@ -383,12 +392,20 @@ namespace { FileManager &FileMgr; const PCHContainerReader &PCHContainerRdr; - /// \brief Mapping from files to module file information. + /// Mapping from files to module file information. typedef llvm::MapVector<const FileEntry *, ModuleFileInfo> ModuleFilesMap; - /// \brief Information about each of the known module files. + /// Information about each of the known module files. ModuleFilesMap ModuleFiles; + /// \brief Mapping from the imported module file to the imported + /// information. + typedef std::multimap<const FileEntry *, ImportedModuleFileInfo> + ImportedModuleFilesMap; + + /// \brief Information about each importing of a module file. + ImportedModuleFilesMap ImportedModuleFiles; + /// \brief Mapping from identifiers to the list of module file IDs that /// consider this identifier to be interesting. typedef llvm::StringMap<SmallVector<unsigned, 2> > InterestingIdentifierMap; @@ -424,7 +441,8 @@ namespace { bool loadModuleFile(const FileEntry *File); /// \brief Write the index to the given bitstream. - void writeIndex(llvm::BitstreamWriter &Stream); + /// \returns true if an error occurred, false otherwise. + bool writeIndex(llvm::BitstreamWriter &Stream); }; } @@ -515,7 +533,7 @@ bool GlobalModuleIndexBuilder::loadModuleFile(const FileEntry *File) { unsigned ID = getModuleFileInfo(File).ID; // Search for the blocks and records we care about. - enum { Other, ControlBlock, ASTBlock } State = Other; + enum { Other, ControlBlock, ASTBlock, DiagnosticOptionsBlock } State = Other; bool Done = false; while (!Done) { llvm::BitstreamEntry Entry = InStream.advance(); @@ -553,6 +571,15 @@ bool GlobalModuleIndexBuilder::loadModuleFile(const FileEntry *File) { continue; } + if (Entry.ID == UNHASHED_CONTROL_BLOCK_ID) { + if (InStream.EnterSubBlock(UNHASHED_CONTROL_BLOCK_ID)) + return true; + + // Found the Diagnostic Options block. + State = DiagnosticOptionsBlock; + continue; + } + if (InStream.SkipBlock()) return true; @@ -587,7 +614,10 @@ bool GlobalModuleIndexBuilder::loadModuleFile(const FileEntry *File) { // Skip the stored signature. // FIXME: we could read the signature out of the import and validate it. - Idx++; + ASTFileSignature StoredSignature = { + {{(uint32_t)Record[Idx++], (uint32_t)Record[Idx++], + (uint32_t)Record[Idx++], (uint32_t)Record[Idx++], + (uint32_t)Record[Idx++]}}}; // Retrieve the imported file name. unsigned Length = Record[Idx++]; @@ -599,11 +629,16 @@ bool GlobalModuleIndexBuilder::loadModuleFile(const FileEntry *File) { const FileEntry *DependsOnFile = FileMgr.getFile(ImportedFile, /*openFile=*/false, /*cacheFailure=*/false); - if (!DependsOnFile || - (StoredSize != DependsOnFile->getSize()) || - (StoredModTime != DependsOnFile->getModificationTime())) + + if (!DependsOnFile) return true; + // Save the information in ImportedModuleFileInfo so we can verify after + // loading all pcms. + ImportedModuleFiles.insert(std::make_pair( + DependsOnFile, ImportedModuleFileInfo(StoredSize, StoredModTime, + StoredSignature))); + // Record the dependency. unsigned DependsOnID = getModuleFileInfo(DependsOnFile).ID; getModuleFileInfo(File).Dependencies.push_back(DependsOnID); @@ -632,6 +667,12 @@ bool GlobalModuleIndexBuilder::loadModuleFile(const FileEntry *File) { } } + // Get Signature. + if (State == DiagnosticOptionsBlock && Code == SIGNATURE) + getModuleFileInfo(File).Signature = { + {{(uint32_t)Record[0], (uint32_t)Record[1], (uint32_t)Record[2], + (uint32_t)Record[3], (uint32_t)Record[4]}}}; + // We don't care about this record. } @@ -680,7 +721,20 @@ public: } -void GlobalModuleIndexBuilder::writeIndex(llvm::BitstreamWriter &Stream) { +bool GlobalModuleIndexBuilder::writeIndex(llvm::BitstreamWriter &Stream) { + for (auto MapEntry : ImportedModuleFiles) { + auto *File = MapEntry.first; + ImportedModuleFileInfo &Info = MapEntry.second; + if (getModuleFileInfo(File).Signature) { + if (getModuleFileInfo(File).Signature != Info.StoredSignature) + // Verify Signature. + return true; + } else if (Info.StoredSize != File->getSize() || + Info.StoredModTime != File->getModificationTime()) + // Verify Size and ModTime. + return true; + } + using namespace llvm; // Emit the file header. @@ -756,6 +810,7 @@ void GlobalModuleIndexBuilder::writeIndex(llvm::BitstreamWriter &Stream) { } Stream.ExitBlock(); + return false; } GlobalModuleIndex::ErrorCode @@ -816,7 +871,8 @@ GlobalModuleIndex::writeIndex(FileManager &FileMgr, SmallVector<char, 16> OutputBuffer; { llvm::BitstreamWriter OutputStream(OutputBuffer); - Builder.writeIndex(OutputStream); + if (Builder.writeIndex(OutputStream)) + return EC_IOError; } // Write the global index file to a temporary file. diff --git a/contrib/llvm/tools/clang/lib/Serialization/Module.cpp b/contrib/llvm/tools/clang/lib/Serialization/Module.cpp index 72b0861..5a44d26 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/Module.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/Module.cpp @@ -19,28 +19,6 @@ using namespace clang; using namespace serialization; using namespace reader; -ModuleFile::ModuleFile(ModuleKind Kind, unsigned Generation) - : Kind(Kind), File(nullptr), Signature(0), DirectlyImported(false), - Generation(Generation), SizeInBits(0), - LocalNumSLocEntries(0), SLocEntryBaseID(0), - SLocEntryBaseOffset(0), SLocEntryOffsets(nullptr), - LocalNumIdentifiers(0), - IdentifierOffsets(nullptr), BaseIdentifierID(0), - IdentifierTableData(nullptr), IdentifierLookupTable(nullptr), - LocalNumMacros(0), MacroOffsets(nullptr), - BasePreprocessedEntityID(0), - PreprocessedEntityOffsets(nullptr), NumPreprocessedEntities(0), - LocalNumHeaderFileInfos(0), - HeaderFileInfoTableData(nullptr), HeaderFileInfoTable(nullptr), - LocalNumSubmodules(0), BaseSubmoduleID(0), - LocalNumSelectors(0), SelectorOffsets(nullptr), BaseSelectorID(0), - SelectorLookupTableData(nullptr), SelectorLookupTable(nullptr), - LocalNumDecls(0), DeclOffsets(nullptr), BaseDeclID(0), - FileSortedDecls(nullptr), NumFileSortedDecls(0), - ObjCCategoriesMap(nullptr), LocalNumObjCCategoriesInMap(0), - LocalNumTypes(0), TypeOffsets(nullptr), BaseTypeIndex(0) -{} - ModuleFile::~ModuleFile() { delete static_cast<ASTIdentifierLookupTable *>(IdentifierLookupTable); delete static_cast<HeaderFileInfoLookupTable *>(HeaderFileInfoTable); diff --git a/contrib/llvm/tools/clang/lib/Serialization/ModuleManager.cpp b/contrib/llvm/tools/clang/lib/Serialization/ModuleManager.cpp index 722b547..1dee4d0 100644 --- a/contrib/llvm/tools/clang/lib/Serialization/ModuleManager.cpp +++ b/contrib/llvm/tools/clang/lib/Serialization/ModuleManager.cpp @@ -12,6 +12,7 @@ // //===----------------------------------------------------------------------===// #include "clang/Serialization/ModuleManager.h" +#include "clang/Basic/MemoryBufferCache.h" #include "clang/Frontend/PCHContainerOperations.h" #include "clang/Lex/HeaderSearch.h" #include "clang/Lex/ModuleMap.h" @@ -27,7 +28,7 @@ using namespace clang; using namespace serialization; -ModuleFile *ModuleManager::lookup(StringRef Name) { +ModuleFile *ModuleManager::lookup(StringRef Name) const { const FileEntry *Entry = FileMgr.getFile(Name, /*openFile=*/false, /*cacheFailure=*/false); if (Entry) @@ -36,9 +37,8 @@ ModuleFile *ModuleManager::lookup(StringRef Name) { return nullptr; } -ModuleFile *ModuleManager::lookup(const FileEntry *File) { - llvm::DenseMap<const FileEntry *, ModuleFile *>::iterator Known - = Modules.find(File); +ModuleFile *ModuleManager::lookup(const FileEntry *File) const { + auto Known = Modules.find(File); if (Known == Modules.end()) return nullptr; @@ -52,6 +52,30 @@ ModuleManager::lookupBuffer(StringRef Name) { return std::move(InMemoryBuffers[Entry]); } +static bool checkSignature(ASTFileSignature Signature, + ASTFileSignature ExpectedSignature, + std::string &ErrorStr) { + if (!ExpectedSignature || Signature == ExpectedSignature) + return false; + + ErrorStr = + Signature ? "signature mismatch" : "could not read module signature"; + return true; +} + +static void updateModuleImports(ModuleFile &MF, ModuleFile *ImportedBy, + SourceLocation ImportLoc) { + if (ImportedBy) { + MF.ImportedBy.insert(ImportedBy); + ImportedBy->Imports.insert(&MF); + } else { + if (!MF.DirectlyImported) + MF.ImportLoc = ImportLoc; + + MF.DirectlyImported = true; + } +} + ModuleManager::AddModuleResult ModuleManager::addModule(StringRef FileName, ModuleKind Type, SourceLocation ImportLoc, ModuleFile *ImportedBy, @@ -84,141 +108,133 @@ ModuleManager::addModule(StringRef FileName, ModuleKind Type, } // Check whether we already loaded this module, before - ModuleFile *ModuleEntry = Modules[Entry]; - bool NewModule = false; - if (!ModuleEntry) { - // Allocate a new module. - NewModule = true; - ModuleEntry = new ModuleFile(Type, Generation); - ModuleEntry->Index = Chain.size(); - ModuleEntry->FileName = FileName.str(); - ModuleEntry->File = Entry; - ModuleEntry->ImportLoc = ImportLoc; - ModuleEntry->InputFilesValidationTimestamp = 0; - - if (ModuleEntry->Kind == MK_ImplicitModule) { - std::string TimestampFilename = ModuleEntry->getTimestampFilename(); - vfs::Status Status; - // A cached stat value would be fine as well. - if (!FileMgr.getNoncachedStatValue(TimestampFilename, Status)) - ModuleEntry->InputFilesValidationTimestamp = - llvm::sys::toTimeT(Status.getLastModificationTime()); - } - - // Load the contents of the module - if (std::unique_ptr<llvm::MemoryBuffer> Buffer = lookupBuffer(FileName)) { - // The buffer was already provided for us. - ModuleEntry->Buffer = std::move(Buffer); - } else { - // Open the AST file. - llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> Buf( - (std::error_code())); - if (FileName == "-") { - Buf = llvm::MemoryBuffer::getSTDIN(); - } else { - // Leave the FileEntry open so if it gets read again by another - // ModuleManager it must be the same underlying file. - // FIXME: Because FileManager::getFile() doesn't guarantee that it will - // give us an open file, this may not be 100% reliable. - Buf = FileMgr.getBufferForFile(ModuleEntry->File, - /*IsVolatile=*/false, - /*ShouldClose=*/false); - } - - if (!Buf) { - ErrorStr = Buf.getError().message(); - delete ModuleEntry; - return Missing; - } - - ModuleEntry->Buffer = std::move(*Buf); - } + if (ModuleFile *ModuleEntry = Modules.lookup(Entry)) { + // Check the stored signature. + if (checkSignature(ModuleEntry->Signature, ExpectedSignature, ErrorStr)) + return OutOfDate; - // Initialize the stream. - ModuleEntry->Data = PCHContainerRdr.ExtractPCH(*ModuleEntry->Buffer); + Module = ModuleEntry; + updateModuleImports(*ModuleEntry, ImportedBy, ImportLoc); + return AlreadyLoaded; } - if (ExpectedSignature) { - // If we've not read the control block yet, read the signature eagerly now - // so that we can check it. - if (!ModuleEntry->Signature) - ModuleEntry->Signature = ReadSignature(ModuleEntry->Data); + // Allocate a new module. + auto NewModule = llvm::make_unique<ModuleFile>(Type, Generation); + NewModule->Index = Chain.size(); + NewModule->FileName = FileName.str(); + NewModule->File = Entry; + NewModule->ImportLoc = ImportLoc; + NewModule->InputFilesValidationTimestamp = 0; + + if (NewModule->Kind == MK_ImplicitModule) { + std::string TimestampFilename = NewModule->getTimestampFilename(); + vfs::Status Status; + // A cached stat value would be fine as well. + if (!FileMgr.getNoncachedStatValue(TimestampFilename, Status)) + NewModule->InputFilesValidationTimestamp = + llvm::sys::toTimeT(Status.getLastModificationTime()); + } - if (ModuleEntry->Signature != ExpectedSignature) { - ErrorStr = ModuleEntry->Signature ? "signature mismatch" - : "could not read module signature"; + // Load the contents of the module + if (std::unique_ptr<llvm::MemoryBuffer> Buffer = lookupBuffer(FileName)) { + // The buffer was already provided for us. + NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(Buffer)); + } else if (llvm::MemoryBuffer *Buffer = PCMCache->lookupBuffer(FileName)) { + NewModule->Buffer = Buffer; + } else { + // Open the AST file. + llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> Buf((std::error_code())); + if (FileName == "-") { + Buf = llvm::MemoryBuffer::getSTDIN(); + } else { + // Leave the FileEntry open so if it gets read again by another + // ModuleManager it must be the same underlying file. + // FIXME: Because FileManager::getFile() doesn't guarantee that it will + // give us an open file, this may not be 100% reliable. + Buf = FileMgr.getBufferForFile(NewModule->File, + /*IsVolatile=*/false, + /*ShouldClose=*/false); + } - if (NewModule) - delete ModuleEntry; - return OutOfDate; + if (!Buf) { + ErrorStr = Buf.getError().message(); + return Missing; } - } - if (ImportedBy) { - ModuleEntry->ImportedBy.insert(ImportedBy); - ImportedBy->Imports.insert(ModuleEntry); - } else { - if (!ModuleEntry->DirectlyImported) - ModuleEntry->ImportLoc = ImportLoc; - - ModuleEntry->DirectlyImported = true; + NewModule->Buffer = &PCMCache->addBuffer(FileName, std::move(*Buf)); } - Module = ModuleEntry; + // Initialize the stream. + NewModule->Data = PCHContainerRdr.ExtractPCH(*NewModule->Buffer); + + // Read the signature eagerly now so that we can check it. Avoid calling + // ReadSignature unless there's something to check though. + if (ExpectedSignature && checkSignature(ReadSignature(NewModule->Data), + ExpectedSignature, ErrorStr)) { + // Try to remove the buffer. If it can't be removed, then it was already + // validated by this process. + if (!PCMCache->tryToRemoveBuffer(NewModule->FileName)) + FileMgr.invalidateCache(NewModule->File); + return OutOfDate; + } - if (!NewModule) - return AlreadyLoaded; + // We're keeping this module. Store it everywhere. + Module = Modules[Entry] = NewModule.get(); - assert(!Modules[Entry] && "module loaded twice"); - Modules[Entry] = ModuleEntry; + updateModuleImports(*NewModule, ImportedBy, ImportLoc); - Chain.push_back(ModuleEntry); - if (!ModuleEntry->isModule()) - PCHChain.push_back(ModuleEntry); + if (!NewModule->isModule()) + PCHChain.push_back(NewModule.get()); if (!ImportedBy) - Roots.push_back(ModuleEntry); + Roots.push_back(NewModule.get()); + Chain.push_back(std::move(NewModule)); return NewlyLoaded; } void ModuleManager::removeModules( - ModuleIterator first, ModuleIterator last, + ModuleIterator First, llvm::SmallPtrSetImpl<ModuleFile *> &LoadedSuccessfully, ModuleMap *modMap) { - if (first == last) + auto Last = end(); + if (First == Last) return; + // Explicitly clear VisitOrder since we might not notice it is stale. VisitOrder.clear(); // Collect the set of module file pointers that we'll be removing. - llvm::SmallPtrSet<ModuleFile *, 4> victimSet(first, last); + llvm::SmallPtrSet<ModuleFile *, 4> victimSet( + (llvm::pointer_iterator<ModuleIterator>(First)), + (llvm::pointer_iterator<ModuleIterator>(Last))); auto IsVictim = [&](ModuleFile *MF) { return victimSet.count(MF); }; // Remove any references to the now-destroyed modules. - for (unsigned i = 0, n = Chain.size(); i != n; ++i) { - Chain[i]->ImportedBy.remove_if(IsVictim); + for (auto I = begin(); I != First; ++I) { + I->Imports.remove_if(IsVictim); + I->ImportedBy.remove_if(IsVictim); } Roots.erase(std::remove_if(Roots.begin(), Roots.end(), IsVictim), Roots.end()); // Remove the modules from the PCH chain. - for (auto I = first; I != last; ++I) { - if (!(*I)->isModule()) { - PCHChain.erase(std::find(PCHChain.begin(), PCHChain.end(), *I), + for (auto I = First; I != Last; ++I) { + if (!I->isModule()) { + PCHChain.erase(std::find(PCHChain.begin(), PCHChain.end(), &*I), PCHChain.end()); break; } } // Delete the modules and erase them from the various structures. - for (ModuleIterator victim = first; victim != last; ++victim) { - Modules.erase((*victim)->File); + for (ModuleIterator victim = First; victim != Last; ++victim) { + Modules.erase(victim->File); if (modMap) { - StringRef ModuleName = (*victim)->ModuleName; + StringRef ModuleName = victim->ModuleName; if (Module *mod = modMap->findModule(ModuleName)) { mod->setASTFile(nullptr); } @@ -227,14 +243,17 @@ void ModuleManager::removeModules( // Files that didn't make it through ReadASTCore successfully will be // rebuilt (or there was an error). Invalidate them so that we can load the // new files that will be renamed over the old ones. - if (LoadedSuccessfully.count(*victim) == 0) - FileMgr.invalidateCache((*victim)->File); - - delete *victim; + // + // The PCMCache tracks whether the module was successfully loaded in another + // thread/context; in that case, it won't need to be rebuilt (and we can't + // safely invalidate it anyway). + if (LoadedSuccessfully.count(&*victim) == 0 && + !PCMCache->tryToRemoveBuffer(victim->FileName)) + FileMgr.invalidateCache(victim->File); } - // Remove the modules from the chain. - Chain.erase(first, last); + // Delete the modules. + Chain.erase(Chain.begin() + (First - begin()), Chain.end()); } void @@ -274,11 +293,9 @@ void ModuleManager::setGlobalIndex(GlobalModuleIndex *Index) { // Notify the global module index about all of the modules we've already // loaded. - for (unsigned I = 0, N = Chain.size(); I != N; ++I) { - if (!GlobalIndex->loadedModuleFile(Chain[I])) { - ModulesInCommonWithGlobalIndex.push_back(Chain[I]); - } - } + for (ModuleFile &M : *this) + if (!GlobalIndex->loadedModuleFile(&M)) + ModulesInCommonWithGlobalIndex.push_back(&M); } void ModuleManager::moduleFileAccepted(ModuleFile *MF) { @@ -288,16 +305,12 @@ void ModuleManager::moduleFileAccepted(ModuleFile *MF) { ModulesInCommonWithGlobalIndex.push_back(MF); } -ModuleManager::ModuleManager(FileManager &FileMgr, +ModuleManager::ModuleManager(FileManager &FileMgr, MemoryBufferCache &PCMCache, const PCHContainerReader &PCHContainerRdr) - : FileMgr(FileMgr), PCHContainerRdr(PCHContainerRdr), GlobalIndex(), - FirstVisitState(nullptr) {} + : FileMgr(FileMgr), PCMCache(&PCMCache), PCHContainerRdr(PCHContainerRdr), + GlobalIndex(), FirstVisitState(nullptr) {} -ModuleManager::~ModuleManager() { - for (unsigned i = 0, e = Chain.size(); i != e; ++i) - delete Chain[e - i - 1]; - delete FirstVisitState; -} +ModuleManager::~ModuleManager() { delete FirstVisitState; } void ModuleManager::visit(llvm::function_ref<bool(ModuleFile &M)> Visitor, llvm::SmallPtrSetImpl<ModuleFile *> *ModuleFilesHit) { @@ -314,11 +327,11 @@ void ModuleManager::visit(llvm::function_ref<bool(ModuleFile &M)> Visitor, Queue.reserve(N); llvm::SmallVector<unsigned, 4> UnusedIncomingEdges; UnusedIncomingEdges.resize(size()); - for (ModuleFile *M : llvm::reverse(*this)) { - unsigned Size = M->ImportedBy.size(); - UnusedIncomingEdges[M->Index] = Size; + for (ModuleFile &M : llvm::reverse(*this)) { + unsigned Size = M.ImportedBy.size(); + UnusedIncomingEdges[M.Index] = Size; if (!Size) - Queue.push_back(M); + Queue.push_back(&M); } // Traverse the graph, making sure to visit a module before visiting any @@ -433,7 +446,7 @@ namespace llvm { struct GraphTraits<ModuleManager> { typedef ModuleFile *NodeRef; typedef llvm::SetVector<ModuleFile *>::const_iterator ChildIteratorType; - typedef ModuleManager::ModuleConstIterator nodes_iterator; + typedef pointer_iterator<ModuleManager::ModuleConstIterator> nodes_iterator; static ChildIteratorType child_begin(NodeRef Node) { return Node->Imports.begin(); @@ -444,11 +457,11 @@ namespace llvm { } static nodes_iterator nodes_begin(const ModuleManager &Manager) { - return Manager.begin(); + return nodes_iterator(Manager.begin()); } static nodes_iterator nodes_end(const ModuleManager &Manager) { - return Manager.end(); + return nodes_iterator(Manager.end()); } }; |
