diff options
| author | dim <dim@FreeBSD.org> | 2017-04-02 17:24:58 +0000 |
|---|---|---|
| committer | dim <dim@FreeBSD.org> | 2017-04-02 17:24:58 +0000 |
| commit | 60b571e49a90d38697b3aca23020d9da42fc7d7f (patch) | |
| tree | 99351324c24d6cb146b6285b6caffa4d26fce188 /contrib/llvm/lib/DebugInfo/PDB | |
| parent | bea1b22c7a9bce1dfdd73e6e5b65bc4752215180 (diff) | |
| download | FreeBSD-src-60b571e49a90d38697b3aca23020d9da42fc7d7f.zip FreeBSD-src-60b571e49a90d38697b3aca23020d9da42fc7d7f.tar.gz | |
Update clang, llvm, lld, lldb, compiler-rt and libc++ to 4.0.0 release:
MFC r309142 (by emaste):
Add WITH_LLD_AS_LD build knob
If set it installs LLD as /usr/bin/ld. LLD (as of version 3.9) is not
capable of linking the world and kernel, but can self-host and link many
substantial applications. GNU ld continues to be used for the world and
kernel build, regardless of how this knob is set.
It is on by default for arm64, and off for all other CPU architectures.
Sponsored by: The FreeBSD Foundation
MFC r310840:
Reapply 310775, now it also builds correctly if lldb is disabled:
Move llvm-objdump from CLANG_EXTRAS to installed by default
We currently install three tools from binutils 2.17.50: as, ld, and
objdump. Work is underway to migrate to a permissively-licensed
tool-chain, with one goal being the retirement of binutils 2.17.50.
LLVM's llvm-objdump is intended to be compatible with GNU objdump
although it is currently missing some options and may have formatting
differences. Enable it by default for testing and further investigation.
It may later be changed to install as /usr/bin/objdump, it becomes a
fully viable replacement.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D8879
MFC r312855 (by emaste):
Rename LLD_AS_LD to LLD_IS_LD, for consistency with CLANG_IS_CC
Reported by: Dan McGregor <dan.mcgregor usask.ca>
MFC r313559 | glebius | 2017-02-10 18:34:48 +0100 (Fri, 10 Feb 2017) | 5 lines
Don't check struct rtentry on FreeBSD, it is an internal kernel structure.
On other systems it may be API structure for SIOCADDRT/SIOCDELRT.
Reviewed by: emaste, dim
MFC r314152 (by jkim):
Remove an assembler flag, which is redundant since r309124. The upstream
took care of it by introducing a macro NO_EXEC_STACK_DIRECTIVE.
http://llvm.org/viewvc/llvm-project?rev=273500&view=rev
Reviewed by: dim
MFC r314564:
Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
4.0.0 (branches/release_40 296509). The release will follow soon.
Please note that from 3.5.0 onwards, clang, llvm and lldb require C++11
support to build; see UPDATING for more information.
Also note that as of 4.0.0, lld should be able to link the base system
on amd64 and aarch64. See the WITH_LLD_IS_LLD setting in src.conf(5).
Though please be aware that this is work in progress.
Release notes for llvm, clang and lld will be available here:
<http://releases.llvm.org/4.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/4.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/4.0.0/tools/lld/docs/ReleaseNotes.html>
Thanks to Ed Maste, Jan Beich, Antoine Brodin and Eric Fiselier for
their help.
Relnotes: yes
Exp-run: antoine
PR: 215969, 216008
MFC r314708:
For now, revert r287232 from upstream llvm trunk (by Daniil Fukalov):
[SCEV] limit recursion depth of CompareSCEVComplexity
Summary:
CompareSCEVComplexity goes too deep (50+ on a quite a big unrolled
loop) and runs almost infinite time.
Added cache of "equal" SCEV pairs to earlier cutoff of further
estimation. Recursion depth limit was also introduced as a parameter.
Reviewers: sanjoy
Subscribers: mzolotukhin, tstellarAMD, llvm-commits
Differential Revision: https://reviews.llvm.org/D26389
This commit is the cause of excessive compile times on skein_block.c
(and possibly other files) during kernel builds on amd64.
We never saw the problematic behavior described in this upstream commit,
so for now it is better to revert it. An upstream bug has been filed
here: https://bugs.llvm.org/show_bug.cgi?id=32142
Reported by: mjg
MFC r314795:
Reapply r287232 from upstream llvm trunk (by Daniil Fukalov):
[SCEV] limit recursion depth of CompareSCEVComplexity
Summary:
CompareSCEVComplexity goes too deep (50+ on a quite a big unrolled
loop) and runs almost infinite time.
Added cache of "equal" SCEV pairs to earlier cutoff of further
estimation. Recursion depth limit was also introduced as a parameter.
Reviewers: sanjoy
Subscribers: mzolotukhin, tstellarAMD, llvm-commits
Differential Revision: https://reviews.llvm.org/D26389
Pull in r296992 from upstream llvm trunk (by Sanjoy Das):
[SCEV] Decrease the recursion threshold for CompareValueComplexity
Fixes PR32142.
r287232 accidentally increased the recursion threshold for
CompareValueComplexity from 2 to 32. This change reverses that
change by introducing a separate flag for CompareValueComplexity's
threshold.
The latter revision fixes the excessive compile times for skein_block.c.
MFC r314907 | mmel | 2017-03-08 12:40:27 +0100 (Wed, 08 Mar 2017) | 7 lines
Unbreak ARMv6 world.
The new compiler_rt library imported with clang 4.0.0 have several fatal
issues (non-functional __udivsi3 for example) with ARM specific instrict
functions. As temporary workaround, until upstream solve these problems,
disable all thumb[1][2] related feature.
MFC r315016:
Update clang, llvm, lld, lldb, compiler-rt and libc++ to 4.0.0 release.
We were already very close to the last release candidate, so this is a
pretty minor update.
Relnotes: yes
MFC r316005:
Revert r314907, and pull in r298713 from upstream compiler-rt trunk (by
Weiming Zhao):
builtins: Select correct code fragments when compiling for Thumb1/Thum2/ARM ISA.
Summary:
Value of __ARM_ARCH_ISA_THUMB isn't based on the actual compilation
mode (-mthumb, -marm), it reflect's capability of given CPU.
Due to this:
- use __tbumb__ and __thumb2__ insteand of __ARM_ARCH_ISA_THUMB
- use '.thumb' directive consistently in all affected files
- decorate all thumb functions using
DEFINE_COMPILERRT_THUMB_FUNCTION()
---------
Note: This patch doesn't fix broken Thumb1 variant of __udivsi3 !
Reviewers: weimingz, rengolin, compnerd
Subscribers: aemerson, dim
Differential Revision: https://reviews.llvm.org/D30938
Discussed with: mmel
Diffstat (limited to 'contrib/llvm/lib/DebugInfo/PDB')
35 files changed, 1475 insertions, 1548 deletions
diff --git a/contrib/llvm/lib/DebugInfo/PDB/DIA/DIAError.cpp b/contrib/llvm/lib/DebugInfo/PDB/DIA/DIAError.cpp index 1d72a92..0da877b 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/DIA/DIAError.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/DIA/DIAError.cpp @@ -10,7 +10,7 @@ using namespace llvm::pdb; // deal with the Error value directly, rather than converting to error_code. class DIAErrorCategory : public std::error_category { public: - const char *name() const LLVM_NOEXCEPT override { return "llvm.pdb.dia"; } + const char *name() const noexcept override { return "llvm.pdb.dia"; } std::string message(int Condition) const override { switch (static_cast<dia_error_code>(Condition)) { @@ -38,21 +38,20 @@ char DIAError::ID = 0; DIAError::DIAError(dia_error_code C) : DIAError(C, "") {} -DIAError::DIAError(const std::string &Context) +DIAError::DIAError(StringRef Context) : DIAError(dia_error_code::unspecified, Context) {} -DIAError::DIAError(dia_error_code C, const std::string &Context) : Code(C) { +DIAError::DIAError(dia_error_code C, StringRef Context) : Code(C) { ErrMsg = "DIA Error: "; std::error_code EC = convertToErrorCode(); - if (Code != dia_error_code::unspecified) - ErrMsg += EC.message() + " "; + ErrMsg += EC.message() + " "; if (!Context.empty()) ErrMsg += Context; } void DIAError::log(raw_ostream &OS) const { OS << ErrMsg << "\n"; } -const std::string &DIAError::getErrorMessage() const { return ErrMsg; } +StringRef DIAError::getErrorMessage() const { return ErrMsg; } std::error_code DIAError::convertToErrorCode() const { return std::error_code(static_cast<int>(Code), *Category); diff --git a/contrib/llvm/lib/DebugInfo/PDB/DIA/DIASession.cpp b/contrib/llvm/lib/DebugInfo/PDB/DIA/DIASession.cpp index fa224af..6ecf335 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/DIA/DIASession.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/DIA/DIASession.cpp @@ -20,31 +20,36 @@ #include "llvm/DebugInfo/PDB/PDBSymbolCompiland.h" #include "llvm/DebugInfo/PDB/PDBSymbolExe.h" #include "llvm/Support/ConvertUTF.h" +#include "llvm/Support/Format.h" +#include "llvm/Support/raw_ostream.h" using namespace llvm; using namespace llvm::pdb; -namespace { - -Error ErrorFromHResult(HRESULT Result) { +static Error ErrorFromHResult(HRESULT Result, StringRef Context) { switch (Result) { case E_PDB_NOT_FOUND: - return make_error<GenericError>(generic_error_code::invalid_path); + return make_error<GenericError>(generic_error_code::invalid_path, Context); case E_PDB_FORMAT: - return make_error<DIAError>(dia_error_code::invalid_file_format); + return make_error<DIAError>(dia_error_code::invalid_file_format, Context); case E_INVALIDARG: - return make_error<DIAError>(dia_error_code::invalid_parameter); + return make_error<DIAError>(dia_error_code::invalid_parameter, Context); case E_UNEXPECTED: - return make_error<DIAError>(dia_error_code::already_loaded); + return make_error<DIAError>(dia_error_code::already_loaded, Context); case E_PDB_INVALID_SIG: case E_PDB_INVALID_AGE: - return make_error<DIAError>(dia_error_code::debug_info_mismatch); - default: - return make_error<DIAError>(dia_error_code::unspecified); + return make_error<DIAError>(dia_error_code::debug_info_mismatch, Context); + default: { + std::string S; + raw_string_ostream OS(S); + OS << "HRESULT: " << format_hex(static_cast<DWORD>(Result), 10, true) + << ": " << Context; + return make_error<DIAError>(dia_error_code::unspecified, OS.str()); + } } } -Error LoadDIA(CComPtr<IDiaDataSource> &DiaDataSource) { +static Error LoadDIA(CComPtr<IDiaDataSource> &DiaDataSource) { if (SUCCEEDED(CoCreateInstance(CLSID_DiaSource, nullptr, CLSCTX_INPROC_SERVER, IID_IDiaDataSource, reinterpret_cast<LPVOID *>(&DiaDataSource)))) @@ -55,12 +60,11 @@ Error LoadDIA(CComPtr<IDiaDataSource> &DiaDataSource) { #if !defined(_MSC_VER) return llvm::make_error<GenericError>( "DIA is only supported when using MSVC."); -#endif - +#else const wchar_t *msdia_dll = nullptr; -#if _MSC_VER == 1900 +#if _MSC_VER >= 1900 && _MSC_VER < 2000 msdia_dll = L"msdia140.dll"; // VS2015 -#elif _MSC_VER == 1800 +#elif _MSC_VER >= 1800 msdia_dll = L"msdia120.dll"; // VS2013 #else #error "Unknown Visual Studio version." @@ -69,10 +73,9 @@ Error LoadDIA(CComPtr<IDiaDataSource> &DiaDataSource) { HRESULT HR; if (FAILED(HR = NoRegCoCreate(msdia_dll, CLSID_DiaSource, IID_IDiaDataSource, reinterpret_cast<LPVOID *>(&DiaDataSource)))) - return ErrorFromHResult(HR); + return ErrorFromHResult(HR, "Calling NoRegCoCreate"); return Error::success(); -} - +#endif } DIASession::DIASession(CComPtr<IDiaSession> DiaSession) : Session(DiaSession) {} @@ -93,10 +96,10 @@ Error DIASession::createFromPdb(StringRef Path, const wchar_t *Path16Str = reinterpret_cast<const wchar_t*>(Path16.data()); HRESULT HR; if (FAILED(HR = DiaDataSource->loadDataFromPdb(Path16Str))) - return ErrorFromHResult(HR); + return ErrorFromHResult(HR, "Calling loadDataFromPdb"); if (FAILED(HR = DiaDataSource->openSession(&DiaSession))) - return ErrorFromHResult(HR); + return ErrorFromHResult(HR, "Calling openSession"); Session.reset(new DIASession(DiaSession)); return Error::success(); @@ -118,10 +121,10 @@ Error DIASession::createFromExe(StringRef Path, const wchar_t *Path16Str = reinterpret_cast<const wchar_t *>(Path16.data()); HRESULT HR; if (FAILED(HR = DiaDataSource->loadDataForExe(Path16Str, nullptr, nullptr))) - return ErrorFromHResult(HR); + return ErrorFromHResult(HR, "Calling loadDataForExe"); if (FAILED(HR = DiaDataSource->openSession(&DiaSession))) - return ErrorFromHResult(HR); + return ErrorFromHResult(HR, "Calling openSession"); Session.reset(new DIASession(DiaSession)); return Error::success(); diff --git a/contrib/llvm/lib/DebugInfo/PDB/GenericError.cpp b/contrib/llvm/lib/DebugInfo/PDB/GenericError.cpp index 34e1799..789f3b8 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/GenericError.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/GenericError.cpp @@ -20,7 +20,7 @@ namespace { // deal with the Error value directly, rather than converting to error_code. class GenericErrorCategory : public std::error_category { public: - const char *name() const LLVM_NOEXCEPT override { return "llvm.pdb"; } + const char *name() const noexcept override { return "llvm.pdb"; } std::string message(int Condition) const override { switch (static_cast<generic_error_code>(Condition)) { @@ -45,11 +45,10 @@ char GenericError::ID = 0; GenericError::GenericError(generic_error_code C) : GenericError(C, "") {} -GenericError::GenericError(const std::string &Context) +GenericError::GenericError(StringRef Context) : GenericError(generic_error_code::unspecified, Context) {} -GenericError::GenericError(generic_error_code C, const std::string &Context) - : Code(C) { +GenericError::GenericError(generic_error_code C, StringRef Context) : Code(C) { ErrMsg = "PDB Error: "; std::error_code EC = convertToErrorCode(); if (Code != generic_error_code::unspecified) @@ -60,7 +59,7 @@ GenericError::GenericError(generic_error_code C, const std::string &Context) void GenericError::log(raw_ostream &OS) const { OS << ErrMsg << "\n"; } -const std::string &GenericError::getErrorMessage() const { return ErrMsg; } +StringRef GenericError::getErrorMessage() const { return ErrMsg; } std::error_code GenericError::convertToErrorCode() const { return std::error_code(static_cast<int>(Code), *Category); diff --git a/contrib/llvm/lib/DebugInfo/PDB/IPDBSourceFile.cpp b/contrib/llvm/lib/DebugInfo/PDB/IPDBSourceFile.cpp index 46b422f..8cb1fbe 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/IPDBSourceFile.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/IPDBSourceFile.cpp @@ -1,4 +1,4 @@ -//===- IPDBSourceFile.cpp - base interface for a PDB source file *- C++ -*-===// +//===- IPDBSourceFile.cpp - base interface for a PDB source file ----------===// // // The LLVM Compiler Infrastructure // @@ -8,15 +8,17 @@ //===----------------------------------------------------------------------===// #include "llvm/DebugInfo/PDB/IPDBSourceFile.h" - #include "llvm/DebugInfo/PDB/PDBExtras.h" +#include "llvm/DebugInfo/PDB/PDBTypes.h" #include "llvm/Support/Format.h" #include "llvm/Support/raw_ostream.h" +#include <cstdint> +#include <string> using namespace llvm; using namespace llvm::pdb; -IPDBSourceFile::~IPDBSourceFile() {} +IPDBSourceFile::~IPDBSourceFile() = default; void IPDBSourceFile::dump(raw_ostream &OS, int Indent) const { OS.indent(Indent); diff --git a/contrib/llvm/lib/DebugInfo/PDB/PDB.cpp b/contrib/llvm/lib/DebugInfo/PDB/PDB.cpp index 69a908e..0d72059 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/PDB.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/PDB.cpp @@ -14,7 +14,7 @@ #include "llvm/DebugInfo/PDB/GenericError.h" #include "llvm/DebugInfo/PDB/IPDBSession.h" #include "llvm/DebugInfo/PDB/PDB.h" -#if HAVE_DIA_SDK +#if LLVM_ENABLE_DIA_SDK #include "llvm/DebugInfo/PDB/DIA/DIASession.h" #endif #include "llvm/DebugInfo/PDB/Raw/RawSession.h" @@ -30,7 +30,7 @@ Error llvm::pdb::loadDataForPDB(PDB_ReaderType Type, StringRef Path, if (Type == PDB_ReaderType::Raw) return RawSession::createFromPdb(Path, Session); -#if HAVE_DIA_SDK +#if LLVM_ENABLE_DIA_SDK return DIASession::createFromPdb(Path, Session); #else return llvm::make_error<GenericError>("DIA is not installed on the system"); @@ -43,7 +43,7 @@ Error llvm::pdb::loadDataForEXE(PDB_ReaderType Type, StringRef Path, if (Type == PDB_ReaderType::Raw) return RawSession::createFromExe(Path, Session); -#if HAVE_DIA_SDK +#if LLVM_ENABLE_DIA_SDK return DIASession::createFromExe(Path, Session); #else return llvm::make_error<GenericError>("DIA is not installed on the system"); diff --git a/contrib/llvm/lib/DebugInfo/PDB/PDBContext.cpp b/contrib/llvm/lib/DebugInfo/PDB/PDBContext.cpp index 7732302..94b81ec 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/PDBContext.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/PDBContext.cpp @@ -29,8 +29,8 @@ PDBContext::PDBContext(const COFFObjectFile &Object, Session->setLoadAddress(ImageBase.get()); } -void PDBContext::dump(raw_ostream &OS, DIDumpType DumpType, - bool DumpEH) {} +void PDBContext::dump(raw_ostream &OS, DIDumpType DumpType, bool DumpEH, + bool SummarizeTypes) {} DILineInfo PDBContext::getLineInfoForAddress(uint64_t Address, DILineInfoSpecifier Specifier) { diff --git a/contrib/llvm/lib/DebugInfo/PDB/PDBInterfaceAnchors.cpp b/contrib/llvm/lib/DebugInfo/PDB/PDBInterfaceAnchors.cpp index a347c67..541fcda 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/PDBInterfaceAnchors.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/PDBInterfaceAnchors.cpp @@ -15,15 +15,14 @@ #include "llvm/DebugInfo/PDB/IPDBLineNumber.h" #include "llvm/DebugInfo/PDB/IPDBRawSymbol.h" #include "llvm/DebugInfo/PDB/IPDBSession.h" -#include "llvm/DebugInfo/PDB/IPDBRawSymbol.h" using namespace llvm; using namespace llvm::pdb; -IPDBSession::~IPDBSession() {} +IPDBSession::~IPDBSession() = default; -IPDBDataStream::~IPDBDataStream() {} +IPDBDataStream::~IPDBDataStream() = default; -IPDBRawSymbol::~IPDBRawSymbol() {} +IPDBRawSymbol::~IPDBRawSymbol() = default; -IPDBLineNumber::~IPDBLineNumber() {} +IPDBLineNumber::~IPDBLineNumber() = default; diff --git a/contrib/llvm/lib/DebugInfo/PDB/PDBSymDumper.cpp b/contrib/llvm/lib/DebugInfo/PDB/PDBSymDumper.cpp index 9450a98..2f81931 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/PDBSymDumper.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/PDBSymDumper.cpp @@ -20,7 +20,7 @@ using namespace llvm::pdb; PDBSymDumper::PDBSymDumper(bool ShouldRequireImpl) : RequireImpl(ShouldRequireImpl) {} -PDBSymDumper::~PDBSymDumper() {} +PDBSymDumper::~PDBSymDumper() = default; void PDBSymDumper::dump(const PDBSymbolAnnotation &Symbol) { PDB_SYMDUMP_UNREACHABLE(PDBSymbolAnnotation) diff --git a/contrib/llvm/lib/DebugInfo/PDB/PDBSymbol.cpp b/contrib/llvm/lib/DebugInfo/PDB/PDBSymbol.cpp index 78b3afc..633e11a 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/PDBSymbol.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/PDBSymbol.cpp @@ -8,9 +8,9 @@ //===----------------------------------------------------------------------===// #include "llvm/DebugInfo/PDB/PDBSymbol.h" - #include "llvm/DebugInfo/PDB/IPDBEnumChildren.h" #include "llvm/DebugInfo/PDB/IPDBRawSymbol.h" +#include "llvm/DebugInfo/PDB/PDBExtras.h" #include "llvm/DebugInfo/PDB/PDBSymbolAnnotation.h" #include "llvm/DebugInfo/PDB/PDBSymbolBlock.h" #include "llvm/DebugInfo/PDB/PDBSymbolCompiland.h" @@ -42,12 +42,9 @@ #include "llvm/DebugInfo/PDB/PDBSymbolTypeVTableShape.h" #include "llvm/DebugInfo/PDB/PDBSymbolUnknown.h" #include "llvm/DebugInfo/PDB/PDBSymbolUsingNamespace.h" -#include "llvm/DebugInfo/PDB/PDBSymDumper.h" -#include <memory> -#include <utility> - +#include "llvm/DebugInfo/PDB/PDBTypes.h" +#include <algorithm> #include <memory> -#include <utility> using namespace llvm; using namespace llvm::pdb; @@ -56,7 +53,7 @@ PDBSymbol::PDBSymbol(const IPDBSession &PDBSession, std::unique_ptr<IPDBRawSymbol> Symbol) : Session(PDBSession), RawSymbol(std::move(Symbol)) {} -PDBSymbol::~PDBSymbol() {} +PDBSymbol::~PDBSymbol() = default; #define FACTORY_SYMTAG_CASE(Tag, Type) \ case PDB_SymType::Tag: \ diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStream.cpp index 3c0586c..4f4a0cf 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStream.cpp @@ -7,13 +7,13 @@ // //===----------------------------------------------------------------------===// +#include "llvm/ADT/StringRef.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamArray.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/PDB/PDBTypes.h" #include "llvm/DebugInfo/PDB/Raw/DbiStream.h" - -#include "llvm/DebugInfo/CodeView/StreamArray.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" #include "llvm/DebugInfo/PDB/Raw/ISectionContribVisitor.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" #include "llvm/DebugInfo/PDB/Raw/InfoStream.h" #include "llvm/DebugInfo/PDB/Raw/ModInfo.h" #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h" @@ -22,49 +22,17 @@ #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/DebugInfo/PDB/Raw/RawTypes.h" #include "llvm/Object/COFF.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstddef> +#include <cstdint> using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; using namespace llvm::support; -namespace { -// Some of the values are stored in bitfields. Since this needs to be portable -// across compilers and architectures (big / little endian in particular) we -// can't use the actual structures below, but must instead do the shifting -// and masking ourselves. The struct definitions are provided for reference. - -// struct DbiFlags { -// uint16_t IncrementalLinking : 1; // True if linked incrementally -// uint16_t IsStripped : 1; // True if private symbols were stripped. -// uint16_t HasCTypes : 1; // True if linked with /debug:ctypes. -// uint16_t Reserved : 13; -//}; -const uint16_t FlagIncrementalMask = 0x0001; -const uint16_t FlagStrippedMask = 0x0002; -const uint16_t FlagHasCTypesMask = 0x0004; - -// struct DbiBuildNo { -// uint16_t MinorVersion : 8; -// uint16_t MajorVersion : 7; -// uint16_t NewVersionFormat : 1; -//}; -const uint16_t BuildMinorMask = 0x00FF; -const uint16_t BuildMinorShift = 0; - -const uint16_t BuildMajorMask = 0x7F00; -const uint16_t BuildMajorShift = 8; - -struct FileInfoSubstreamHeader { - ulittle16_t NumModules; // Total # of modules, should match number of - // records in the ModuleInfo substream. - ulittle16_t NumSourceFiles; // Total # of source files. This value is not - // accurate because PDB actually supports more - // than 64k source files, so we ignore it and - // compute the value from other stream fields. -}; -} - template <typename ContribType> static Error loadSectionContribs(FixedStreamArray<ContribType> &Output, StreamReader &Reader) { @@ -81,15 +49,14 @@ static Error loadSectionContribs(FixedStreamArray<ContribType> &Output, DbiStream::DbiStream(PDBFile &File, std::unique_ptr<MappedBlockStream> Stream) : Pdb(File), Stream(std::move(Stream)), Header(nullptr) { - static_assert(sizeof(HeaderInfo) == 64, "Invalid HeaderInfo size!"); } -DbiStream::~DbiStream() {} +DbiStream::~DbiStream() = default; Error DbiStream::reload() { StreamReader Reader(*Stream); - if (Stream->getLength() < sizeof(HeaderInfo)) + if (Stream->getLength() < sizeof(DbiStreamHeader)) return make_error<RawError>(raw_error_code::corrupt_file, "DBI Stream does not contain a header."); if (auto EC = Reader.readObject(Header)) @@ -116,7 +83,7 @@ Error DbiStream::reload() { "DBI Age does not match PDB Age."); if (Stream->getLength() != - sizeof(HeaderInfo) + Header->ModiSubstreamSize + + sizeof(DbiStreamHeader) + Header->ModiSubstreamSize + Header->SecContrSubstreamSize + Header->SectionMapSize + Header->FileInfoSize + Header->TypeServerSize + Header->OptionalDbgHdrSize + Header->ECSubstreamSize) @@ -142,14 +109,11 @@ Error DbiStream::reload() { return make_error<RawError>(raw_error_code::corrupt_file, "DBI type server substream not aligned."); - // Since each ModInfo in the stream is a variable length, we have to iterate - // them to know how many there actually are. - VarStreamArray<ModInfo> ModInfoArray; - if (auto EC = Reader.readArray(ModInfoArray, Header->ModiSubstreamSize)) + if (auto EC = + Reader.readStreamRef(ModInfoSubstream, Header->ModiSubstreamSize)) + return EC; + if (auto EC = initializeModInfoArray()) return EC; - for (auto &Info : ModInfoArray) { - ModuleInfos.emplace_back(Info); - } if (auto EC = Reader.readStreamRef(SecContrSubstream, Header->SecContrSubstreamSize)) @@ -209,25 +173,27 @@ uint16_t DbiStream::getGlobalSymbolStreamIndex() const { uint16_t DbiStream::getFlags() const { return Header->Flags; } bool DbiStream::isIncrementallyLinked() const { - return (Header->Flags & FlagIncrementalMask) != 0; + return (Header->Flags & DbiFlags::FlagIncrementalMask) != 0; } bool DbiStream::hasCTypes() const { - return (Header->Flags & FlagHasCTypesMask) != 0; + return (Header->Flags & DbiFlags::FlagHasCTypesMask) != 0; } bool DbiStream::isStripped() const { - return (Header->Flags & FlagStrippedMask) != 0; + return (Header->Flags & DbiFlags::FlagStrippedMask) != 0; } uint16_t DbiStream::getBuildNumber() const { return Header->BuildNumber; } uint16_t DbiStream::getBuildMajorVersion() const { - return (Header->BuildNumber & BuildMajorMask) >> BuildMajorShift; + return (Header->BuildNumber & DbiBuildNo::BuildMajorMask) >> + DbiBuildNo::BuildMajorShift; } uint16_t DbiStream::getBuildMinorVersion() const { - return (Header->BuildNumber & BuildMinorMask) >> BuildMinorShift; + return (Header->BuildNumber & DbiBuildNo::BuildMinorMask) >> + DbiBuildNo::BuildMinorShift; } uint16_t DbiStream::getPdbDllRbld() const { return Header->PdbDllRbld; } @@ -243,21 +209,20 @@ PDB_Machine DbiStream::getMachineType() const { return static_cast<PDB_Machine>(Machine); } -codeview::FixedStreamArray<object::coff_section> -DbiStream::getSectionHeaders() { +msf::FixedStreamArray<object::coff_section> DbiStream::getSectionHeaders() { return SectionHeaders; } -codeview::FixedStreamArray<object::FpoData> DbiStream::getFpoRecords() { +msf::FixedStreamArray<object::FpoData> DbiStream::getFpoRecords() { return FpoRecords; } ArrayRef<ModuleInfoEx> DbiStream::modules() const { return ModuleInfos; } -codeview::FixedStreamArray<SecMapEntry> DbiStream::getSectionMap() const { +msf::FixedStreamArray<SecMapEntry> DbiStream::getSectionMap() const { return SectionMap; } -void llvm::pdb::DbiStream::visitSectionContributions( +void DbiStream::visitSectionContributions( ISectionContribVisitor &Visitor) const { if (SectionContribVersion == DbiSecContribVer60) { for (auto &SC : SectionContribs) @@ -285,6 +250,24 @@ Error DbiStream::initializeSectionContributionData() { "Unsupported DBI Section Contribution version"); } +Error DbiStream::initializeModInfoArray() { + if (ModInfoSubstream.getLength() == 0) + return Error::success(); + + // Since each ModInfo in the stream is a variable length, we have to iterate + // them to know how many there actually are. + StreamReader Reader(ModInfoSubstream); + + VarStreamArray<ModInfo> ModInfoArray; + if (auto EC = Reader.readArray(ModInfoArray, ModInfoSubstream.getLength())) + return EC; + for (auto &Info : ModInfoArray) { + ModuleInfos.emplace_back(Info); + } + + return Error::success(); +} + // Initializes this->SectionHeaders. Error DbiStream::initializeSectionHeadersData() { if (DbgStreams.size() == 0) @@ -294,22 +277,21 @@ Error DbiStream::initializeSectionHeadersData() { if (StreamNum >= Pdb.getNumStreams()) return make_error<RawError>(raw_error_code::no_stream); - auto SHS = MappedBlockStream::createIndexedStream(StreamNum, Pdb); - if (!SHS) - return SHS.takeError(); + auto SHS = MappedBlockStream::createIndexedStream( + Pdb.getMsfLayout(), Pdb.getMsfBuffer(), StreamNum); - size_t StreamLen = (*SHS)->getLength(); + size_t StreamLen = SHS->getLength(); if (StreamLen % sizeof(object::coff_section)) return make_error<RawError>(raw_error_code::corrupt_file, "Corrupted section header stream."); size_t NumSections = StreamLen / sizeof(object::coff_section); - codeview::StreamReader Reader(**SHS); + msf::StreamReader Reader(*SHS); if (auto EC = Reader.readArray(SectionHeaders, NumSections)) return make_error<RawError>(raw_error_code::corrupt_file, "Could not read a bitmap."); - SectionHeaderStream = std::move(*SHS); + SectionHeaderStream = std::move(SHS); return Error::success(); } @@ -321,27 +303,26 @@ Error DbiStream::initializeFpoRecords() { uint32_t StreamNum = getDebugStreamIndex(DbgHeaderType::NewFPO); // This means there is no FPO data. - if (StreamNum == InvalidStreamIndex) + if (StreamNum == kInvalidStreamIndex) return Error::success(); if (StreamNum >= Pdb.getNumStreams()) return make_error<RawError>(raw_error_code::no_stream); - auto FS = MappedBlockStream::createIndexedStream(StreamNum, Pdb); - if (!FS) - return FS.takeError(); + auto FS = MappedBlockStream::createIndexedStream( + Pdb.getMsfLayout(), Pdb.getMsfBuffer(), StreamNum); - size_t StreamLen = (*FS)->getLength(); + size_t StreamLen = FS->getLength(); if (StreamLen % sizeof(object::FpoData)) return make_error<RawError>(raw_error_code::corrupt_file, "Corrupted New FPO stream."); size_t NumRecords = StreamLen / sizeof(object::FpoData); - codeview::StreamReader Reader(**FS); + msf::StreamReader Reader(*FS); if (auto EC = Reader.readArray(FpoRecords, NumRecords)) return make_error<RawError>(raw_error_code::corrupt_file, "Corrupted New FPO stream."); - FpoStream = std::move(*FS); + FpoStream = std::move(FS); return Error::success(); } @@ -359,18 +340,6 @@ Error DbiStream::initializeSectionMapData() { } Error DbiStream::initializeFileInfo() { - // The layout of the FileInfoSubstream is like this: - // struct { - // ulittle16_t NumModules; - // ulittle16_t NumSourceFiles; - // ulittle16_t ModIndices[NumModules]; - // ulittle16_t ModFileCounts[NumModules]; - // ulittle32_t FileNameOffsets[NumSourceFiles]; - // char Names[][NumSourceFiles]; - // }; - // with the caveat that `NumSourceFiles` cannot be trusted, so - // it is computed by summing `ModFileCounts`. - // if (FileInfoSubstream.getLength() == 0) return Error::success(); @@ -437,7 +406,10 @@ Error DbiStream::initializeFileInfo() { } uint32_t DbiStream::getDebugStreamIndex(DbgHeaderType Type) const { - return DbgStreams[static_cast<uint16_t>(Type)]; + uint16_t T = static_cast<uint16_t>(Type); + if (T >= DbgStreams.size()) + return kInvalidStreamIndex; + return DbgStreams[T]; } Expected<StringRef> DbiStream::getFileNameForIndex(uint32_t Index) const { @@ -452,11 +424,3 @@ Expected<StringRef> DbiStream::getFileNameForIndex(uint32_t Index) const { return std::move(EC); return Name; } - -Error DbiStream::commit() { - StreamWriter Writer(*Stream); - if (auto EC = Writer.writeObject(*Header)) - return EC; - - return Error::success(); -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStreamBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStreamBuilder.cpp index 34ff8ae..1d5b8d6 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStreamBuilder.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/DbiStreamBuilder.cpp @@ -9,18 +9,28 @@ #include "llvm/DebugInfo/PDB/Raw/DbiStreamBuilder.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" +#include "llvm/ADT/ArrayRef.h" +#include "llvm/DebugInfo/MSF/MSFBuilder.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" #include "llvm/DebugInfo/PDB/Raw/DbiStream.h" -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/Object/COFF.h" +#include "llvm/Support/COFF.h" using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; -DbiStreamBuilder::DbiStreamBuilder() - : Age(1), BuildNumber(0), PdbDllVersion(0), PdbDllRbld(0), Flags(0), - MachineType(PDB_Machine::x86) {} +namespace { +class ModiSubstreamBuilder {}; +} + +DbiStreamBuilder::DbiStreamBuilder(msf::MSFBuilder &Msf) + : Msf(Msf), Allocator(Msf.getAllocator()), Age(1), BuildNumber(0), + PdbDllVersion(0), PdbDllRbld(0), Flags(0), MachineType(PDB_Machine::x86), + Header(nullptr), DbgStreams((int)DbgHeaderType::Max) {} void DbiStreamBuilder::setVersionHeader(PdbRaw_DbiVer V) { VerHeader = V; } @@ -36,24 +46,207 @@ void DbiStreamBuilder::setFlags(uint16_t F) { Flags = F; } void DbiStreamBuilder::setMachineType(PDB_Machine M) { MachineType = M; } +void DbiStreamBuilder::setSectionContribs(ArrayRef<SectionContrib> Arr) { + SectionContribs = Arr; +} + +void DbiStreamBuilder::setSectionMap(ArrayRef<SecMapEntry> SecMap) { + SectionMap = SecMap; +} + +Error DbiStreamBuilder::addDbgStream(pdb::DbgHeaderType Type, + ArrayRef<uint8_t> Data) { + if (DbgStreams[(int)Type].StreamNumber) + return make_error<RawError>(raw_error_code::duplicate_entry, + "The specified stream type already exists"); + auto ExpectedIndex = Msf.addStream(Data.size()); + if (!ExpectedIndex) + return ExpectedIndex.takeError(); + uint32_t Index = std::move(*ExpectedIndex); + DbgStreams[(int)Type].Data = Data; + DbgStreams[(int)Type].StreamNumber = Index; + return Error::success(); +} + uint32_t DbiStreamBuilder::calculateSerializedLength() const { // For now we only support serializing the header. - return sizeof(DbiStream::HeaderInfo); -} - -Expected<std::unique_ptr<DbiStream>> DbiStreamBuilder::build(PDBFile &File) { - if (!VerHeader.hasValue()) - return make_error<RawError>(raw_error_code::unspecified, - "Missing DBI Stream Version"); - - auto DbiS = MappedBlockStream::createIndexedStream(StreamDBI, File); - if (!DbiS) - return DbiS.takeError(); - auto DS = std::move(*DbiS); - DbiStream::HeaderInfo *H = - static_cast<DbiStream::HeaderInfo *>(DS->getAllocator().Allocate( - sizeof(DbiStream::HeaderInfo), - llvm::AlignOf<DbiStream::HeaderInfo>::Alignment)); + return sizeof(DbiStreamHeader) + calculateFileInfoSubstreamSize() + + calculateModiSubstreamSize() + calculateSectionContribsStreamSize() + + calculateSectionMapStreamSize() + calculateDbgStreamsSize(); +} + +Error DbiStreamBuilder::addModuleInfo(StringRef ObjFile, StringRef Module) { + auto Entry = llvm::make_unique<ModuleInfo>(); + ModuleInfo *M = Entry.get(); + Entry->Mod = Module; + Entry->Obj = ObjFile; + auto Result = ModuleInfos.insert(std::make_pair(Module, std::move(Entry))); + if (!Result.second) + return make_error<RawError>(raw_error_code::duplicate_entry, + "The specified module already exists"); + ModuleInfoList.push_back(M); + return Error::success(); +} + +Error DbiStreamBuilder::addModuleSourceFile(StringRef Module, StringRef File) { + auto ModIter = ModuleInfos.find(Module); + if (ModIter == ModuleInfos.end()) + return make_error<RawError>(raw_error_code::no_entry, + "The specified module was not found"); + uint32_t Index = SourceFileNames.size(); + SourceFileNames.insert(std::make_pair(File, Index)); + auto &ModEntry = *ModIter; + ModEntry.second->SourceFiles.push_back(File); + return Error::success(); +} + +uint32_t DbiStreamBuilder::calculateModiSubstreamSize() const { + uint32_t Size = 0; + for (const auto &M : ModuleInfoList) { + Size += sizeof(ModuleInfoHeader); + Size += M->Mod.size() + 1; + Size += M->Obj.size() + 1; + } + return alignTo(Size, sizeof(uint32_t)); +} + +uint32_t DbiStreamBuilder::calculateSectionContribsStreamSize() const { + if (SectionContribs.empty()) + return 0; + return sizeof(enum PdbRaw_DbiSecContribVer) + + sizeof(SectionContribs[0]) * SectionContribs.size(); +} + +uint32_t DbiStreamBuilder::calculateSectionMapStreamSize() const { + if (SectionMap.empty()) + return 0; + return sizeof(SecMapHeader) + sizeof(SecMapEntry) * SectionMap.size(); +} + +uint32_t DbiStreamBuilder::calculateFileInfoSubstreamSize() const { + uint32_t Size = 0; + Size += sizeof(ulittle16_t); // NumModules + Size += sizeof(ulittle16_t); // NumSourceFiles + Size += ModuleInfoList.size() * sizeof(ulittle16_t); // ModIndices + Size += ModuleInfoList.size() * sizeof(ulittle16_t); // ModFileCounts + uint32_t NumFileInfos = 0; + for (const auto &M : ModuleInfoList) + NumFileInfos += M->SourceFiles.size(); + Size += NumFileInfos * sizeof(ulittle32_t); // FileNameOffsets + Size += calculateNamesBufferSize(); + return alignTo(Size, sizeof(uint32_t)); +} + +uint32_t DbiStreamBuilder::calculateNamesBufferSize() const { + uint32_t Size = 0; + for (const auto &F : SourceFileNames) { + Size += F.getKeyLength() + 1; // Names[I]; + } + return Size; +} + +uint32_t DbiStreamBuilder::calculateDbgStreamsSize() const { + return DbgStreams.size() * sizeof(uint16_t); +} + +Error DbiStreamBuilder::generateModiSubstream() { + uint32_t Size = calculateModiSubstreamSize(); + auto Data = Allocator.Allocate<uint8_t>(Size); + + ModInfoBuffer = MutableByteStream(MutableArrayRef<uint8_t>(Data, Size)); + + StreamWriter ModiWriter(ModInfoBuffer); + for (const auto &M : ModuleInfoList) { + ModuleInfoHeader Layout = {}; + Layout.ModDiStream = kInvalidStreamIndex; + Layout.NumFiles = M->SourceFiles.size(); + if (auto EC = ModiWriter.writeObject(Layout)) + return EC; + if (auto EC = ModiWriter.writeZeroString(M->Mod)) + return EC; + if (auto EC = ModiWriter.writeZeroString(M->Obj)) + return EC; + } + if (ModiWriter.bytesRemaining() > sizeof(uint32_t)) + return make_error<RawError>(raw_error_code::invalid_format, + "Unexpected bytes in Modi Stream Data"); + return Error::success(); +} + +Error DbiStreamBuilder::generateFileInfoSubstream() { + uint32_t Size = calculateFileInfoSubstreamSize(); + uint32_t NameSize = calculateNamesBufferSize(); + auto Data = Allocator.Allocate<uint8_t>(Size); + uint32_t NamesOffset = Size - NameSize; + + FileInfoBuffer = MutableByteStream(MutableArrayRef<uint8_t>(Data, Size)); + + WritableStreamRef MetadataBuffer = + WritableStreamRef(FileInfoBuffer).keep_front(NamesOffset); + StreamWriter MetadataWriter(MetadataBuffer); + + uint16_t ModiCount = std::min<uint32_t>(UINT16_MAX, ModuleInfos.size()); + uint16_t FileCount = std::min<uint32_t>(UINT16_MAX, SourceFileNames.size()); + if (auto EC = MetadataWriter.writeInteger(ModiCount)) // NumModules + return EC; + if (auto EC = MetadataWriter.writeInteger(FileCount)) // NumSourceFiles + return EC; + for (uint16_t I = 0; I < ModiCount; ++I) { + if (auto EC = MetadataWriter.writeInteger(I)) // Mod Indices + return EC; + } + for (const auto MI : ModuleInfoList) { + FileCount = static_cast<uint16_t>(MI->SourceFiles.size()); + if (auto EC = MetadataWriter.writeInteger(FileCount)) // Mod File Counts + return EC; + } + + // Before writing the FileNameOffsets array, write the NamesBuffer array. + // A side effect of this is that this will actually compute the various + // file name offsets, so we can then go back and write the FileNameOffsets + // array to the other substream. + NamesBuffer = WritableStreamRef(FileInfoBuffer).drop_front(NamesOffset); + StreamWriter NameBufferWriter(NamesBuffer); + for (auto &Name : SourceFileNames) { + Name.second = NameBufferWriter.getOffset(); + if (auto EC = NameBufferWriter.writeZeroString(Name.getKey())) + return EC; + } + + for (const auto MI : ModuleInfoList) { + for (StringRef Name : MI->SourceFiles) { + auto Result = SourceFileNames.find(Name); + if (Result == SourceFileNames.end()) + return make_error<RawError>(raw_error_code::no_entry, + "The source file was not found."); + if (auto EC = MetadataWriter.writeInteger(Result->second)) + return EC; + } + } + + if (NameBufferWriter.bytesRemaining() > 0) + return make_error<RawError>(raw_error_code::invalid_format, + "The names buffer contained unexpected data."); + + if (MetadataWriter.bytesRemaining() > sizeof(uint32_t)) + return make_error<RawError>( + raw_error_code::invalid_format, + "The metadata buffer contained unexpected data."); + + return Error::success(); +} + +Error DbiStreamBuilder::finalize() { + if (Header) + return Error::success(); + + DbiStreamHeader *H = Allocator.Allocate<DbiStreamHeader>(); + + if (auto EC = generateModiSubstream()) + return EC; + if (auto EC = generateFileInfoSubstream()) + return EC; + H->VersionHeader = *VerHeader; H->VersionSignature = -1; H->Age = Age; @@ -64,18 +257,156 @@ Expected<std::unique_ptr<DbiStream>> DbiStreamBuilder::build(PDBFile &File) { H->MachineType = static_cast<uint16_t>(MachineType); H->ECSubstreamSize = 0; - H->FileInfoSize = 0; - H->ModiSubstreamSize = 0; - H->OptionalDbgHdrSize = 0; - H->SecContrSubstreamSize = 0; - H->SectionMapSize = 0; + H->FileInfoSize = FileInfoBuffer.getLength(); + H->ModiSubstreamSize = ModInfoBuffer.getLength(); + H->OptionalDbgHdrSize = DbgStreams.size() * sizeof(uint16_t); + H->SecContrSubstreamSize = calculateSectionContribsStreamSize(); + H->SectionMapSize = calculateSectionMapStreamSize(); H->TypeServerSize = 0; - H->SymRecordStreamIndex = DbiStream::InvalidStreamIndex; - H->PublicSymbolStreamIndex = DbiStream::InvalidStreamIndex; - H->MFCTypeServerIndex = DbiStream::InvalidStreamIndex; - H->GlobalSymbolStreamIndex = DbiStream::InvalidStreamIndex; - - auto Dbi = llvm::make_unique<DbiStream>(File, std::move(DS)); - Dbi->Header = H; - return std::move(Dbi); + H->SymRecordStreamIndex = kInvalidStreamIndex; + H->PublicSymbolStreamIndex = kInvalidStreamIndex; + H->MFCTypeServerIndex = kInvalidStreamIndex; + H->GlobalSymbolStreamIndex = kInvalidStreamIndex; + + Header = H; + return Error::success(); +} + +Error DbiStreamBuilder::finalizeMsfLayout() { + uint32_t Length = calculateSerializedLength(); + if (auto EC = Msf.setStreamSize(StreamDBI, Length)) + return EC; + return Error::success(); +} + +static uint16_t toSecMapFlags(uint32_t Flags) { + uint16_t Ret = 0; + if (Flags & COFF::IMAGE_SCN_MEM_READ) + Ret |= static_cast<uint16_t>(OMFSegDescFlags::Read); + if (Flags & COFF::IMAGE_SCN_MEM_WRITE) + Ret |= static_cast<uint16_t>(OMFSegDescFlags::Write); + if (Flags & COFF::IMAGE_SCN_MEM_EXECUTE) + Ret |= static_cast<uint16_t>(OMFSegDescFlags::Execute); + if (Flags & COFF::IMAGE_SCN_MEM_EXECUTE) + Ret |= static_cast<uint16_t>(OMFSegDescFlags::Execute); + if (!(Flags & COFF::IMAGE_SCN_MEM_16BIT)) + Ret |= static_cast<uint16_t>(OMFSegDescFlags::AddressIs32Bit); + + // This seems always 1. + Ret |= static_cast<uint16_t>(OMFSegDescFlags::IsSelector); + + return Ret; +} + +// A utility function to create Section Contributions +// for a given input sections. +std::vector<SectionContrib> DbiStreamBuilder::createSectionContribs( + ArrayRef<object::coff_section> SecHdrs) { + std::vector<SectionContrib> Ret; + + // Create a SectionContrib for each input section. + for (auto &Sec : SecHdrs) { + Ret.emplace_back(); + auto &Entry = Ret.back(); + memset(&Entry, 0, sizeof(Entry)); + + Entry.Off = Sec.PointerToRawData; + Entry.Size = Sec.SizeOfRawData; + Entry.Characteristics = Sec.Characteristics; + } + return Ret; +} + +// A utility function to create a Section Map for a given list of COFF sections. +// +// A Section Map seem to be a copy of a COFF section list in other format. +// I don't know why a PDB file contains both a COFF section header and +// a Section Map, but it seems it must be present in a PDB. +std::vector<SecMapEntry> DbiStreamBuilder::createSectionMap( + ArrayRef<llvm::object::coff_section> SecHdrs) { + std::vector<SecMapEntry> Ret; + int Idx = 0; + + auto Add = [&]() -> SecMapEntry & { + Ret.emplace_back(); + auto &Entry = Ret.back(); + memset(&Entry, 0, sizeof(Entry)); + + Entry.Frame = Idx + 1; + + // We don't know the meaning of these fields yet. + Entry.SecName = UINT16_MAX; + Entry.ClassName = UINT16_MAX; + + return Entry; + }; + + for (auto &Hdr : SecHdrs) { + auto &Entry = Add(); + Entry.Flags = toSecMapFlags(Hdr.Characteristics); + Entry.SecByteLength = Hdr.VirtualSize; + ++Idx; + } + + // The last entry is for absolute symbols. + auto &Entry = Add(); + Entry.Flags = static_cast<uint16_t>(OMFSegDescFlags::AddressIs32Bit) | + static_cast<uint16_t>(OMFSegDescFlags::IsAbsoluteAddress); + Entry.SecByteLength = UINT32_MAX; + + return Ret; +} + +Error DbiStreamBuilder::commit(const msf::MSFLayout &Layout, + const msf::WritableStream &Buffer) { + if (auto EC = finalize()) + return EC; + + auto InfoS = + WritableMappedBlockStream::createIndexedStream(Layout, Buffer, StreamDBI); + + StreamWriter Writer(*InfoS); + if (auto EC = Writer.writeObject(*Header)) + return EC; + + if (auto EC = Writer.writeStreamRef(ModInfoBuffer)) + return EC; + + if (!SectionContribs.empty()) { + if (auto EC = Writer.writeEnum(DbiSecContribVer60)) + return EC; + if (auto EC = Writer.writeArray(SectionContribs)) + return EC; + } + + if (!SectionMap.empty()) { + ulittle16_t Size = static_cast<ulittle16_t>(SectionMap.size()); + SecMapHeader SMHeader = {Size, Size}; + if (auto EC = Writer.writeObject(SMHeader)) + return EC; + if (auto EC = Writer.writeArray(SectionMap)) + return EC; + } + + if (auto EC = Writer.writeStreamRef(FileInfoBuffer)) + return EC; + + for (auto &Stream : DbgStreams) + if (auto EC = Writer.writeInteger(Stream.StreamNumber)) + return EC; + + for (auto &Stream : DbgStreams) { + if (Stream.StreamNumber == kInvalidStreamIndex) + continue; + auto WritableStream = WritableMappedBlockStream::createIndexedStream( + Layout, Buffer, Stream.StreamNumber); + StreamWriter DbgStreamWriter(*WritableStream); + if (auto EC = DbgStreamWriter.writeArray(Stream.Data)) + return EC; + } + + if (Writer.bytesRemaining() > 0) + return make_error<RawError>(raw_error_code::invalid_format, + "Unexpected bytes found in DBI Stream"); + return Error::success(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.cpp new file mode 100644 index 0000000..6ecbb5c --- /dev/null +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.cpp @@ -0,0 +1,93 @@ +//===- GSI.cpp - Common Functions for GlobalsStream and PublicsStream ----===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#include "GSI.h" + +#include "llvm/DebugInfo/MSF/StreamArray.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" + +#include "llvm/Support/Error.h" + +namespace llvm { +namespace pdb { + +static Error checkHashHdrVersion(const GSIHashHeader *HashHdr) { + if (HashHdr->VerHdr != GSIHashHeader::HdrVersion) + return make_error<RawError>( + raw_error_code::feature_unsupported, + "Encountered unsupported globals stream version."); + + return Error::success(); +} + +Error readGSIHashBuckets( + msf::FixedStreamArray<support::ulittle32_t> &HashBuckets, + const GSIHashHeader *HashHdr, msf::StreamReader &Reader) { + if (auto EC = checkHashHdrVersion(HashHdr)) + return EC; + + // Before the actual hash buckets, there is a bitmap of length determined by + // IPHR_HASH. + ArrayRef<uint8_t> Bitmap; + size_t BitmapSizeInBits = alignTo(IPHR_HASH + 1, 32); + uint32_t NumBitmapEntries = BitmapSizeInBits / 8; + if (auto EC = Reader.readBytes(Bitmap, NumBitmapEntries)) + return joinErrors(std::move(EC), + make_error<RawError>(raw_error_code::corrupt_file, + "Could not read a bitmap.")); + uint32_t NumBuckets = 0; + for (uint8_t B : Bitmap) + NumBuckets += countPopulation(B); + + // Hash buckets follow. + if (auto EC = Reader.readArray(HashBuckets, NumBuckets)) + return joinErrors(std::move(EC), + make_error<RawError>(raw_error_code::corrupt_file, + "Hash buckets corrupted.")); + + return Error::success(); +} + +Error readGSIHashHeader(const GSIHashHeader *&HashHdr, + msf::StreamReader &Reader) { + if (Reader.readObject(HashHdr)) + return make_error<RawError>(raw_error_code::corrupt_file, + "Stream does not contain a GSIHashHeader."); + + if (HashHdr->VerSignature != GSIHashHeader::HdrSignature) + return make_error<RawError>( + raw_error_code::feature_unsupported, + "GSIHashHeader signature (0xffffffff) not found."); + + return Error::success(); +} + +Error readGSIHashRecords(msf::FixedStreamArray<PSHashRecord> &HashRecords, + const GSIHashHeader *HashHdr, + msf::StreamReader &Reader) { + if (auto EC = checkHashHdrVersion(HashHdr)) + return EC; + + // HashHdr->HrSize specifies the number of bytes of PSHashRecords we have. + // Verify that we can read them all. + if (HashHdr->HrSize % sizeof(PSHashRecord)) + return make_error<RawError>(raw_error_code::corrupt_file, + "Invalid HR array size."); + uint32_t NumHashRecords = HashHdr->HrSize / sizeof(PSHashRecord); + if (auto EC = Reader.readArray(HashRecords, NumHashRecords)) + return joinErrors(std::move(EC), + make_error<RawError>(raw_error_code::corrupt_file, + "Error reading hash records.")); + + return Error::success(); +} +} +} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.h b/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.h new file mode 100644 index 0000000..82cebd9 --- /dev/null +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/GSI.h @@ -0,0 +1,70 @@ +//===- GSI.h - Common Declarations for GlobalsStream and PublicsStream ----===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// The data structures defined in this file are based on the reference +// implementation which is available at +// https://github.com/Microsoft/microsoft-pdb/blob/master/PDB/dbi/gsi.h +// +// When you are reading the reference source code, you'd find the +// information below useful. +// +// - ppdb1->m_fMinimalDbgInfo seems to be always true. +// - SMALLBUCKETS macro is defined. +// +// The reference doesn't compile, so I learned just by reading code. +// It's not guaranteed to be correct. +// +//===----------------------------------------------------------------------===// + +#ifndef LLVM_LIB_DEBUGINFO_PDB_RAW_GSI_H +#define LLVM_LIB_DEBUGINFO_PDB_RAW_GSI_H + +#include "llvm/DebugInfo/MSF/StreamArray.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" + +#include "llvm/Support/Endian.h" +#include "llvm/Support/Error.h" + +namespace llvm { + +namespace msf { +class StreamReader; +} + +namespace pdb { + +/// From https://github.com/Microsoft/microsoft-pdb/blob/master/PDB/dbi/gsi.cpp +static const unsigned IPHR_HASH = 4096; + +/// Header of the hash tables found in the globals and publics sections. +/// Based on GSIHashHeader in +/// https://github.com/Microsoft/microsoft-pdb/blob/master/PDB/dbi/gsi.h +struct GSIHashHeader { + enum : unsigned { + HdrSignature = ~0U, + HdrVersion = 0xeffe0000 + 19990810, + }; + support::ulittle32_t VerSignature; + support::ulittle32_t VerHdr; + support::ulittle32_t HrSize; + support::ulittle32_t NumBuckets; +}; + +Error readGSIHashBuckets( + msf::FixedStreamArray<support::ulittle32_t> &HashBuckets, + const GSIHashHeader *HashHdr, msf::StreamReader &Reader); +Error readGSIHashHeader(const GSIHashHeader *&HashHdr, + msf::StreamReader &Reader); +Error readGSIHashRecords(msf::FixedStreamArray<PSHashRecord> &HashRecords, + const GSIHashHeader *HashHdr, + msf::StreamReader &Reader); +} +} + +#endif diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/GlobalsStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/GlobalsStream.cpp new file mode 100644 index 0000000..31afc92 --- /dev/null +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/GlobalsStream.cpp @@ -0,0 +1,42 @@ +//===- GlobalsStream.cpp - PDB Index of Symbols by Name ---- ----*- C++ -*-===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#include "GSI.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/PDB/Raw/GlobalsStream.h" +#include "llvm/Support/Error.h" +#include <algorithm> + +using namespace llvm; +using namespace llvm::msf; +using namespace llvm::pdb; + +GlobalsStream::GlobalsStream(std::unique_ptr<MappedBlockStream> Stream) + : Stream(std::move(Stream)) {} + +GlobalsStream::~GlobalsStream() = default; + +Error GlobalsStream::reload() { + StreamReader Reader(*Stream); + + const GSIHashHeader *HashHdr; + if (auto EC = readGSIHashHeader(HashHdr, Reader)) + return EC; + + if (auto EC = readGSIHashRecords(HashRecords, HashHdr, Reader)) + return EC; + + if (auto EC = readGSIHashBuckets(HashBuckets, HashHdr, Reader)) + return EC; + NumBuckets = HashBuckets.size(); + + return Error::success(); +} + +Error GlobalsStream::commit() { return Error::success(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/Hash.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/Hash.cpp index 23cb557..b9f685e 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/Hash.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/Hash.cpp @@ -11,6 +11,7 @@ #include "llvm/ADT/ArrayRef.h" #include "llvm/Support/Endian.h" +#include "llvm/Support/JamCRC.h" using namespace llvm; using namespace llvm::support; @@ -73,59 +74,13 @@ uint32_t pdb::hashStringV2(StringRef Str) { Hash ^= (Hash >> 6); } - return Hash * 1664525L + 1013904223L; + return Hash * 1664525U + 1013904223U; } -static const uint32_t V8HashTable[] = { - 0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, - 0xE963A535, 0x9E6495A3, 0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, - 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07, 0x90BF1D91, 0x1DB71064, 0x6AB020F2, - 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7, - 0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9, - 0xFA0F3D63, 0x8D080DF5, 0x3B6E20C8, 0x4C69105E, 0xD56041E4, 0xA2677172, - 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B, 0x35B5A8FA, 0x42B2986C, - 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59, - 0x26D930AC, 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, - 0xCFBA9599, 0xB8BDA50F, 0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, - 0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D, 0x76DC4190, 0x01DB7106, - 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, 0x9FBFE4A5, 0xE8B8D433, - 0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, - 0x91646C97, 0xE6635C01, 0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, - 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457, 0x65B0D9C6, 0x12B7E950, - 0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65, - 0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, - 0xA4D1C46D, 0xD3D6F4FB, 0x4369E96A, 0x346ED9FC, 0xAD678846, 0xDA60B8D0, - 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9, 0x5005713C, 0x270241AA, - 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F, - 0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, - 0xB7BD5C3B, 0xC0BA6CAD, 0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, - 0xEAD54739, 0x9DD277AF, 0x04DB2615, 0x73DC1683, 0xE3630B12, 0x94643B84, - 0x0D6D6A3E, 0x7A6A5AA8, 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1, - 0xF00F9344, 0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB, - 0x196C3671, 0x6E6B06E7, 0xFED41B76, 0x89D32BE0, 0x10DA7A5A, 0x67DD4ACC, - 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5, 0xD6D6A3E8, 0xA1D1937E, - 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B, - 0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, - 0x316E8EEF, 0x4669BE79, 0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, - 0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F, 0xC5BA3BBE, 0xB2BD0B28, - 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D, - 0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, 0x9C0906A9, 0xEB0E363F, - 0x72076785, 0x05005713, 0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, - 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21, 0x86D3D2D4, 0xF1D4E242, - 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777, - 0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, - 0x616BFFD3, 0x166CCF45, 0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, - 0xA7672661, 0xD06016F7, 0x4969474D, 0x3E6E77DB, 0xAED16A4A, 0xD9D65ADC, - 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9, - 0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, 0xCDD70693, - 0x54DE5729, 0x23D967BF, 0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, - 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D, -}; - // Corresponds to `SigForPbCb` in langapi/shared/crc32.h. uint32_t pdb::hashBufferV8(ArrayRef<uint8_t> Buf) { - uint32_t Hash = 0; - for (uint8_t Byte : Buf) - Hash = (Hash >> 8) ^ V8HashTable[(Hash & 0xff) ^ Byte]; - return Hash; + JamCRC JC(/*Init=*/0U); + JC.update(makeArrayRef<char>(reinterpret_cast<const char *>(Buf.data()), + Buf.size())); + return JC.getCRC(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/IndexedStreamData.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/IndexedStreamData.cpp deleted file mode 100644 index 9bd16ea..0000000 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/IndexedStreamData.cpp +++ /dev/null @@ -1,25 +0,0 @@ -//===- IndexedStreamData.cpp - Standard PDB Stream Data ---------*- C++ -*-===// -// -// The LLVM Compiler Infrastructure -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/IPDBFile.h" - -using namespace llvm; -using namespace llvm::pdb; - -IndexedStreamData::IndexedStreamData(uint32_t StreamIdx, const IPDBFile &File) - : StreamIdx(StreamIdx), File(File) {} - -uint32_t IndexedStreamData::getLength() { - return File.getStreamByteSize(StreamIdx); -} - -ArrayRef<support::ulittle32_t> IndexedStreamData::getStreamBlocks() { - return File.getStreamBlockList(StreamIdx); -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStream.cpp index c33a764..f19535d 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStream.cpp @@ -10,24 +10,25 @@ #include "llvm/DebugInfo/PDB/Raw/InfoStream.h" #include "llvm/ADT/BitVector.h" #include "llvm/ADT/SmallVector.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/RawConstants.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; InfoStream::InfoStream(std::unique_ptr<MappedBlockStream> Stream) : Stream(std::move(Stream)) {} Error InfoStream::reload() { - codeview::StreamReader Reader(*Stream); + StreamReader Reader(*Stream); - const HeaderInfo *H; + const InfoStreamHeader *H; if (auto EC = Reader.readObject(H)) return joinErrors( std::move(EC), @@ -74,17 +75,3 @@ uint32_t InfoStream::getSignature() const { return Signature; } uint32_t InfoStream::getAge() const { return Age; } PDB_UniqueId InfoStream::getGuid() const { return Guid; } - -Error InfoStream::commit() { - StreamWriter Writer(*Stream); - - HeaderInfo H; - H.Age = Age; - H.Signature = Signature; - H.Version = Version; - H.Guid = Guid; - if (auto EC = Writer.writeObject(H)) - return EC; - - return NamedStreams.commit(Writer); -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStreamBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStreamBuilder.cpp index 7be9cc3..73fbf85 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStreamBuilder.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/InfoStreamBuilder.cpp @@ -9,16 +9,20 @@ #include "llvm/DebugInfo/PDB/Raw/InfoStreamBuilder.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" +#include "llvm/DebugInfo/MSF/MSFBuilder.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" #include "llvm/DebugInfo/PDB/Raw/InfoStream.h" -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; -InfoStreamBuilder::InfoStreamBuilder() {} +InfoStreamBuilder::InfoStreamBuilder(msf::MSFBuilder &Msf) + : Msf(Msf), Ver(PdbRaw_ImplVer::PdbImplVC70), Sig(-1), Age(0) {} void InfoStreamBuilder::setVersion(PdbRaw_ImplVer V) { Ver = V; } @@ -33,35 +37,29 @@ NameMapBuilder &InfoStreamBuilder::getNamedStreamsBuilder() { } uint32_t InfoStreamBuilder::calculateSerializedLength() const { - return sizeof(InfoStream::HeaderInfo) + - NamedStreams.calculateSerializedLength(); + return sizeof(InfoStreamHeader) + NamedStreams.calculateSerializedLength(); } -Expected<std::unique_ptr<InfoStream>> InfoStreamBuilder::build(PDBFile &File) { - if (!Ver.hasValue()) - return make_error<RawError>(raw_error_code::unspecified, - "Missing PDB Stream Version"); - if (!Sig.hasValue()) - return make_error<RawError>(raw_error_code::unspecified, - "Missing PDB Stream Signature"); - if (!Age.hasValue()) - return make_error<RawError>(raw_error_code::unspecified, - "Missing PDB Stream Age"); - if (!Guid.hasValue()) - return make_error<RawError>(raw_error_code::unspecified, - "Missing PDB Stream Guid"); +Error InfoStreamBuilder::finalizeMsfLayout() { + uint32_t Length = calculateSerializedLength(); + if (auto EC = Msf.setStreamSize(StreamPDB, Length)) + return EC; + return Error::success(); +} + +Error InfoStreamBuilder::commit(const msf::MSFLayout &Layout, + const msf::WritableStream &Buffer) const { + auto InfoS = + WritableMappedBlockStream::createIndexedStream(Layout, Buffer, StreamPDB); + StreamWriter Writer(*InfoS); + + InfoStreamHeader H; + H.Age = Age; + H.Signature = Sig; + H.Version = Ver; + H.Guid = Guid; + if (auto EC = Writer.writeObject(H)) + return EC; - auto InfoS = MappedBlockStream::createIndexedStream(StreamPDB, File); - if (!InfoS) - return InfoS.takeError(); - auto Info = llvm::make_unique<InfoStream>(std::move(*InfoS)); - Info->Version = *Ver; - Info->Signature = *Sig; - Info->Age = *Age; - Info->Guid = *Guid; - auto NS = NamedStreams.build(); - if (!NS) - return NS.takeError(); - Info->NamedStreams = **NS; - return std::move(Info); + return NamedStreams.commit(Writer); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp deleted file mode 100644 index 92b2048..0000000 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/MappedBlockStream.cpp +++ /dev/null @@ -1,310 +0,0 @@ -//===- MappedBlockStream.cpp - Reads stream data from a PDBFile -----------===// -// -// The LLVM Compiler Infrastructure -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" -#include "llvm/DebugInfo/PDB/Raw/DirectoryStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/IPDBStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/PDBFile.h" -#include "llvm/DebugInfo/PDB/Raw/RawError.h" - -using namespace llvm; -using namespace llvm::pdb; - -namespace { -// This exists so that we can use make_unique while still keeping the -// constructor of MappedBlockStream private, forcing users to go through -// the `create` interface. -class MappedBlockStreamImpl : public MappedBlockStream { -public: - MappedBlockStreamImpl(std::unique_ptr<IPDBStreamData> Data, - const IPDBFile &File) - : MappedBlockStream(std::move(Data), File) {} -}; -} - -typedef std::pair<uint32_t, uint32_t> Interval; -static Interval intersect(const Interval &I1, const Interval &I2) { - return std::make_pair(std::max(I1.first, I2.first), - std::min(I1.second, I2.second)); -} - -MappedBlockStream::MappedBlockStream(std::unique_ptr<IPDBStreamData> Data, - const IPDBFile &Pdb) - : Pdb(Pdb), Data(std::move(Data)) {} - -Error MappedBlockStream::readBytes(uint32_t Offset, uint32_t Size, - ArrayRef<uint8_t> &Buffer) const { - // Make sure we aren't trying to read beyond the end of the stream. - if (Size > Data->getLength()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - if (Offset > Data->getLength() - Size) - return make_error<RawError>(raw_error_code::insufficient_buffer); - - if (tryReadContiguously(Offset, Size, Buffer)) - return Error::success(); - - auto CacheIter = CacheMap.find(Offset); - if (CacheIter != CacheMap.end()) { - // Try to find an alloc that was large enough for this request. - for (auto &Entry : CacheIter->second) { - if (Entry.size() >= Size) { - Buffer = Entry.slice(0, Size); - return Error::success(); - } - } - } - - // We couldn't find a buffer that started at the correct offset (the most - // common scenario). Try to see if there is a buffer that starts at some - // other offset but overlaps the desired range. - for (auto &CacheItem : CacheMap) { - Interval RequestExtent = std::make_pair(Offset, Offset + Size); - - // We already checked this one on the fast path above. - if (CacheItem.first == Offset) - continue; - // If the initial extent of the cached item is beyond the ending extent - // of the request, there is no overlap. - if (CacheItem.first >= Offset + Size) - continue; - - // We really only have to check the last item in the list, since we append - // in order of increasing length. - if (CacheItem.second.empty()) - continue; - - auto CachedAlloc = CacheItem.second.back(); - // If the initial extent of the request is beyond the ending extent of - // the cached item, there is no overlap. - Interval CachedExtent = - std::make_pair(CacheItem.first, CacheItem.first + CachedAlloc.size()); - if (RequestExtent.first >= CachedExtent.first + CachedExtent.second) - continue; - - Interval Intersection = intersect(CachedExtent, RequestExtent); - // Only use this if the entire request extent is contained in the cached - // extent. - if (Intersection != RequestExtent) - continue; - - uint32_t CacheRangeOffset = - AbsoluteDifference(CachedExtent.first, Intersection.first); - Buffer = CachedAlloc.slice(CacheRangeOffset, Size); - return Error::success(); - } - - // Otherwise allocate a large enough buffer in the pool, memcpy the data - // into it, and return an ArrayRef to that. Do not touch existing pool - // allocations, as existing clients may be holding a pointer which must - // not be invalidated. - uint8_t *WriteBuffer = static_cast<uint8_t *>(Pool.Allocate(Size, 8)); - if (auto EC = readBytes(Offset, MutableArrayRef<uint8_t>(WriteBuffer, Size))) - return EC; - - if (CacheIter != CacheMap.end()) { - CacheIter->second.emplace_back(WriteBuffer, Size); - } else { - std::vector<CacheEntry> List; - List.emplace_back(WriteBuffer, Size); - CacheMap.insert(std::make_pair(Offset, List)); - } - Buffer = ArrayRef<uint8_t>(WriteBuffer, Size); - return Error::success(); -} - -Error MappedBlockStream::readLongestContiguousChunk( - uint32_t Offset, ArrayRef<uint8_t> &Buffer) const { - // Make sure we aren't trying to read beyond the end of the stream. - if (Offset >= Data->getLength()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - uint32_t First = Offset / Pdb.getBlockSize(); - uint32_t Last = First; - - auto BlockList = Data->getStreamBlocks(); - while (Last < Pdb.getBlockCount() - 1) { - if (BlockList[Last] != BlockList[Last + 1] - 1) - break; - ++Last; - } - - uint32_t OffsetInFirstBlock = Offset % Pdb.getBlockSize(); - uint32_t BytesFromFirstBlock = Pdb.getBlockSize() - OffsetInFirstBlock; - uint32_t BlockSpan = Last - First + 1; - uint32_t ByteSpan = - BytesFromFirstBlock + (BlockSpan - 1) * Pdb.getBlockSize(); - auto Result = Pdb.getBlockData(BlockList[First], Pdb.getBlockSize()); - if (!Result) - return Result.takeError(); - Buffer = Result->drop_front(OffsetInFirstBlock); - Buffer = ArrayRef<uint8_t>(Buffer.data(), ByteSpan); - return Error::success(); -} - -uint32_t MappedBlockStream::getLength() const { return Data->getLength(); } - -Error MappedBlockStream::commit() const { return Error::success(); } - -bool MappedBlockStream::tryReadContiguously(uint32_t Offset, uint32_t Size, - ArrayRef<uint8_t> &Buffer) const { - // Attempt to fulfill the request with a reference directly into the stream. - // This can work even if the request crosses a block boundary, provided that - // all subsequent blocks are contiguous. For example, a 10k read with a 4k - // block size can be filled with a reference if, from the starting offset, - // 3 blocks in a row are contiguous. - uint32_t BlockNum = Offset / Pdb.getBlockSize(); - uint32_t OffsetInBlock = Offset % Pdb.getBlockSize(); - uint32_t BytesFromFirstBlock = - std::min(Size, Pdb.getBlockSize() - OffsetInBlock); - uint32_t NumAdditionalBlocks = - llvm::alignTo(Size - BytesFromFirstBlock, Pdb.getBlockSize()) / - Pdb.getBlockSize(); - - auto BlockList = Data->getStreamBlocks(); - uint32_t RequiredContiguousBlocks = NumAdditionalBlocks + 1; - uint32_t E = BlockList[BlockNum]; - for (uint32_t I = 0; I < RequiredContiguousBlocks; ++I, ++E) { - if (BlockList[I + BlockNum] != E) - return false; - } - - uint32_t FirstBlockAddr = BlockList[BlockNum]; - auto Result = Pdb.getBlockData(FirstBlockAddr, Pdb.getBlockSize()); - if (!Result) { - consumeError(Result.takeError()); - return false; - } - auto Data = Result->drop_front(OffsetInBlock); - Buffer = ArrayRef<uint8_t>(Data.data(), Size); - return true; -} - -Error MappedBlockStream::readBytes(uint32_t Offset, - MutableArrayRef<uint8_t> Buffer) const { - uint32_t BlockNum = Offset / Pdb.getBlockSize(); - uint32_t OffsetInBlock = Offset % Pdb.getBlockSize(); - - // Make sure we aren't trying to read beyond the end of the stream. - if (Buffer.size() > Data->getLength()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - if (Offset > Data->getLength() - Buffer.size()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - - uint32_t BytesLeft = Buffer.size(); - uint32_t BytesWritten = 0; - uint8_t *WriteBuffer = Buffer.data(); - auto BlockList = Data->getStreamBlocks(); - while (BytesLeft > 0) { - uint32_t StreamBlockAddr = BlockList[BlockNum]; - - auto Result = Pdb.getBlockData(StreamBlockAddr, Pdb.getBlockSize()); - if (!Result) - return Result.takeError(); - - auto Data = *Result; - const uint8_t *ChunkStart = Data.data() + OffsetInBlock; - uint32_t BytesInChunk = - std::min(BytesLeft, Pdb.getBlockSize() - OffsetInBlock); - ::memcpy(WriteBuffer + BytesWritten, ChunkStart, BytesInChunk); - - BytesWritten += BytesInChunk; - BytesLeft -= BytesInChunk; - ++BlockNum; - OffsetInBlock = 0; - } - - return Error::success(); -} - -Error MappedBlockStream::writeBytes(uint32_t Offset, - ArrayRef<uint8_t> Buffer) const { - // Make sure we aren't trying to write beyond the end of the stream. - if (Buffer.size() > Data->getLength()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - - if (Offset > Data->getLength() - Buffer.size()) - return make_error<RawError>(raw_error_code::insufficient_buffer); - - uint32_t BlockNum = Offset / Pdb.getBlockSize(); - uint32_t OffsetInBlock = Offset % Pdb.getBlockSize(); - - uint32_t BytesLeft = Buffer.size(); - auto BlockList = Data->getStreamBlocks(); - uint32_t BytesWritten = 0; - while (BytesLeft > 0) { - uint32_t StreamBlockAddr = BlockList[BlockNum]; - uint32_t BytesToWriteInChunk = - std::min(BytesLeft, Pdb.getBlockSize() - OffsetInBlock); - - const uint8_t *Chunk = Buffer.data() + BytesWritten; - ArrayRef<uint8_t> ChunkData(Chunk, BytesToWriteInChunk); - if (auto EC = Pdb.setBlockData(StreamBlockAddr, OffsetInBlock, ChunkData)) - return EC; - - BytesLeft -= BytesToWriteInChunk; - BytesWritten += BytesToWriteInChunk; - ++BlockNum; - OffsetInBlock = 0; - } - - // If this write overlapped a read which previously came from the pool, - // someone may still be holding a pointer to that alloc which is now invalid. - // Compute the overlapping range and update the cache entry, so any - // outstanding buffers are automatically updated. - for (const auto &MapEntry : CacheMap) { - // If the end of the written extent precedes the beginning of the cached - // extent, ignore this map entry. - if (Offset + BytesWritten < MapEntry.first) - continue; - for (const auto &Alloc : MapEntry.second) { - // If the end of the cached extent precedes the beginning of the written - // extent, ignore this alloc. - if (MapEntry.first + Alloc.size() < Offset) - continue; - - // If we get here, they are guaranteed to overlap. - Interval WriteInterval = std::make_pair(Offset, Offset + BytesWritten); - Interval CachedInterval = - std::make_pair(MapEntry.first, MapEntry.first + Alloc.size()); - // If they overlap, we need to write the new data into the overlapping - // range. - auto Intersection = intersect(WriteInterval, CachedInterval); - assert(Intersection.first <= Intersection.second); - - uint32_t Length = Intersection.second - Intersection.first; - uint32_t SrcOffset = - AbsoluteDifference(WriteInterval.first, Intersection.first); - uint32_t DestOffset = - AbsoluteDifference(CachedInterval.first, Intersection.first); - ::memcpy(Alloc.data() + DestOffset, Buffer.data() + SrcOffset, Length); - } - } - - return Error::success(); -} - -uint32_t MappedBlockStream::getNumBytesCopied() const { - return static_cast<uint32_t>(Pool.getBytesAllocated()); -} - -Expected<std::unique_ptr<MappedBlockStream>> -MappedBlockStream::createIndexedStream(uint32_t StreamIdx, - const IPDBFile &File) { - if (StreamIdx >= File.getNumStreams()) - return make_error<RawError>(raw_error_code::no_stream); - - auto Data = llvm::make_unique<IndexedStreamData>(StreamIdx, File); - return llvm::make_unique<MappedBlockStreamImpl>(std::move(Data), File); -} - -Expected<std::unique_ptr<MappedBlockStream>> -MappedBlockStream::createDirectoryStream(const PDBFile &File) { - auto Data = llvm::make_unique<DirectoryStreamData>(File); - return llvm::make_unique<MappedBlockStreamImpl>(std::move(Data), File); -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/ModInfo.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/ModInfo.cpp index bae135f..b34d770 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/ModInfo.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/ModInfo.cpp @@ -7,76 +7,27 @@ // //===----------------------------------------------------------------------===// +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/ModInfo.h" - -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/PDB/Raw/PDBFile.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" #include "llvm/Support/Endian.h" +#include "llvm/Support/Error.h" +#include "llvm/Support/MathExtras.h" +#include <cstdint> using namespace llvm; +using namespace llvm::msf; using namespace llvm::pdb; using namespace llvm::support; -namespace { - -struct SCBytes { - ulittle16_t Section; - char Padding1[2]; - little32_t Offset; - little32_t Size; - ulittle32_t Characteristics; - ulittle16_t ModuleIndex; - char Padding2[2]; - ulittle32_t DataCrc; - ulittle32_t RelocCrc; -}; - -// struct Flags { -// uint16_t fWritten : 1; // True if ModInfo is dirty -// uint16_t fECEnabled : 1; // Is EC symbolic info present? (What is EC?) -// uint16_t unused : 6; // Reserved -// uint16_t iTSM : 8; // Type Server Index for this module -//}; -const uint16_t HasECFlagMask = 0x2; - -const uint16_t TypeServerIndexMask = 0xFF00; -const uint16_t TypeServerIndexShift = 8; -} +ModInfo::ModInfo() = default; + +ModInfo::ModInfo(const ModInfo &Info) = default; -struct ModInfo::FileLayout { - ulittle32_t Mod; // Currently opened module. This field is a - // pointer in the reference implementation, but - // that won't work on 64-bit systems, and anyway - // it doesn't make sense to read a pointer from a - // file. For now it is unused, so just ignore it. - SCBytes SC; // First section contribution of this module. - ulittle16_t Flags; // See Flags definition. - ulittle16_t ModDiStream; // Stream Number of module debug info - ulittle32_t SymBytes; // Size of local symbol debug info in above stream - ulittle32_t LineBytes; // Size of line number debug info in above stream - ulittle32_t C13Bytes; // Size of C13 line number info in above stream - ulittle16_t NumFiles; // Number of files contributing to this module - char Padding1[2]; // Padding so the next field is 4-byte aligned. - ulittle32_t FileNameOffs; // array of [0..NumFiles) DBI name buffer offsets. - // This field is a pointer in the reference - // implementation, but as with `Mod`, we ignore it - // for now since it is unused. - ulittle32_t SrcFileNameNI; // Name Index for src file name - ulittle32_t PdbFilePathNI; // Name Index for path to compiler PDB - // Null terminated Module name - // Null terminated Obj File Name -}; - -ModInfo::ModInfo() : Layout(nullptr) {} - -ModInfo::ModInfo(const ModInfo &Info) - : ModuleName(Info.ModuleName), ObjFileName(Info.ObjFileName), - Layout(Info.Layout) {} - -ModInfo::~ModInfo() {} - -Error ModInfo::initialize(codeview::StreamRef Stream, ModInfo &Info) { - codeview::StreamReader Reader(Stream); +ModInfo::~ModInfo() = default; + +Error ModInfo::initialize(ReadableStreamRef Stream, ModInfo &Info) { + StreamReader Reader(Stream); if (auto EC = Reader.readObject(Info.Layout)) return EC; @@ -88,10 +39,13 @@ Error ModInfo::initialize(codeview::StreamRef Stream, ModInfo &Info) { return Error::success(); } -bool ModInfo::hasECInfo() const { return (Layout->Flags & HasECFlagMask) != 0; } +bool ModInfo::hasECInfo() const { + return (Layout->Flags & ModInfoFlags::HasECFlagMask) != 0; +} uint16_t ModInfo::getTypeServerIndex() const { - return (Layout->Flags & TypeServerIndexMask) >> TypeServerIndexShift; + return (Layout->Flags & ModInfoFlags::TypeServerIndexMask) >> + ModInfoFlags::TypeServerIndexShift; } uint16_t ModInfo::getModuleStreamIndex() const { return Layout->ModDiStream; } @@ -121,7 +75,7 @@ StringRef ModInfo::getObjFileName() const { return ObjFileName; } uint32_t ModInfo::getRecordLength() const { uint32_t M = ModuleName.str().size() + 1; uint32_t O = ObjFileName.str().size() + 1; - uint32_t Size = sizeof(FileLayout) + M + O; - Size = llvm::alignTo(Size, 4); + uint32_t Size = sizeof(ModuleInfoHeader) + M + O; + Size = alignTo(Size, 4); return Size; } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/ModStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/ModStream.cpp index 3415fcd..0ffc5b7 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/ModStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/ModStream.cpp @@ -7,39 +7,43 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/ModStream.h" - -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" +#include "llvm/ADT/iterator_range.h" +#include "llvm/DebugInfo/CodeView/SymbolRecord.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/MSF/StreamRef.h" #include "llvm/DebugInfo/PDB/Raw/ModInfo.h" +#include "llvm/DebugInfo/PDB/Raw/ModStream.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/DebugInfo/PDB/Raw/RawTypes.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> using namespace llvm; +using namespace llvm::msf; using namespace llvm::pdb; ModStream::ModStream(const ModInfo &Module, std::unique_ptr<MappedBlockStream> Stream) : Mod(Module), Stream(std::move(Stream)) {} -ModStream::~ModStream() {} +ModStream::~ModStream() = default; Error ModStream::reload() { - codeview::StreamReader Reader(*Stream); + StreamReader Reader(*Stream); uint32_t SymbolSize = Mod.getSymbolDebugInfoByteSize(); uint32_t C11Size = Mod.getLineInfoByteSize(); uint32_t C13Size = Mod.getC13LineInfoByteSize(); if (C11Size > 0 && C13Size > 0) - return llvm::make_error<RawError>(raw_error_code::corrupt_file, - "Module has both C11 and C13 line info"); + return make_error<RawError>(raw_error_code::corrupt_file, + "Module has both C11 and C13 line info"); - codeview::StreamRef S; + ReadableStreamRef S; - uint32_t SymbolSubstreamSig = 0; - if (auto EC = Reader.readInteger(SymbolSubstreamSig)) + if (auto EC = Reader.readInteger(Signature)) return EC; if (auto EC = Reader.readArray(SymbolsSubstream, SymbolSize - 4)) return EC; @@ -49,7 +53,7 @@ Error ModStream::reload() { if (auto EC = Reader.readStreamRef(C13LinesSubstream, C13Size)) return EC; - codeview::StreamReader LineReader(C13LinesSubstream); + StreamReader LineReader(C13LinesSubstream); if (auto EC = LineReader.readArray(LineInfo, LineReader.bytesRemaining())) return EC; @@ -59,8 +63,8 @@ Error ModStream::reload() { if (auto EC = Reader.readStreamRef(GlobalRefsSubstream, GlobalRefsSize)) return EC; if (Reader.bytesRemaining() > 0) - return llvm::make_error<RawError>(raw_error_code::corrupt_file, - "Unexpected bytes in module stream."); + return make_error<RawError>(raw_error_code::corrupt_file, + "Unexpected bytes in module stream."); return Error::success(); } @@ -69,14 +73,13 @@ iterator_range<codeview::CVSymbolArray::Iterator> ModStream::symbols(bool *HadError) const { // It's OK if the stream is empty. if (SymbolsSubstream.getUnderlyingStream().getLength() == 0) - return llvm::make_range(SymbolsSubstream.end(), SymbolsSubstream.end()); - return llvm::make_range(SymbolsSubstream.begin(HadError), - SymbolsSubstream.end()); + return make_range(SymbolsSubstream.end(), SymbolsSubstream.end()); + return make_range(SymbolsSubstream.begin(HadError), SymbolsSubstream.end()); } iterator_range<codeview::ModuleSubstreamArray::Iterator> ModStream::lines(bool *HadError) const { - return llvm::make_range(LineInfo.begin(HadError), LineInfo.end()); + return make_range(LineInfo.begin(HadError), LineInfo.end()); } Error ModStream::commit() { return Error::success(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfBuilder.cpp deleted file mode 100644 index 16b086b..0000000 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfBuilder.cpp +++ /dev/null @@ -1,279 +0,0 @@ -//===- MSFBuilder.cpp - MSF Directory & Metadata Builder --------*- C++ -*-===// -// -// The LLVM Compiler Infrastructure -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "llvm/DebugInfo/PDB/Raw/MsfBuilder.h" -#include "llvm/DebugInfo/PDB/Raw/RawError.h" - -using namespace llvm; -using namespace llvm::pdb; -using namespace llvm::pdb::msf; -using namespace llvm::support; - -namespace { -const uint32_t kSuperBlockBlock = 0; -const uint32_t kFreePageMap0Block = 1; -const uint32_t kFreePageMap1Block = 2; -const uint32_t kNumReservedPages = 3; - -const uint32_t kDefaultBlockMapAddr = kNumReservedPages; -} - -MsfBuilder::MsfBuilder(uint32_t BlockSize, uint32_t MinBlockCount, bool CanGrow, - BumpPtrAllocator &Allocator) - : Allocator(Allocator), IsGrowable(CanGrow), BlockSize(BlockSize), - MininumBlocks(MinBlockCount), BlockMapAddr(kDefaultBlockMapAddr), - FreeBlocks(MinBlockCount, true) { - FreeBlocks[kSuperBlockBlock] = false; - FreeBlocks[kFreePageMap0Block] = false; - FreeBlocks[kFreePageMap1Block] = false; - FreeBlocks[BlockMapAddr] = false; -} - -Expected<MsfBuilder> MsfBuilder::create(BumpPtrAllocator &Allocator, - uint32_t BlockSize, - uint32_t MinBlockCount, bool CanGrow) { - if (!msf::isValidBlockSize(BlockSize)) - return make_error<RawError>(raw_error_code::unspecified, - "The requested block size is unsupported"); - - return MsfBuilder(BlockSize, - std::max(MinBlockCount, msf::getMinimumBlockCount()), - CanGrow, Allocator); -} - -Error MsfBuilder::setBlockMapAddr(uint32_t Addr) { - if (Addr == BlockMapAddr) - return Error::success(); - - if (Addr >= FreeBlocks.size()) { - if (!IsGrowable) - return make_error<RawError>(raw_error_code::unspecified, - "Cannot grow the number of blocks"); - FreeBlocks.resize(Addr + 1); - } - - if (!isBlockFree(Addr)) - return make_error<RawError>(raw_error_code::unspecified, - "Attempt to reuse an allocated block"); - FreeBlocks[BlockMapAddr] = true; - FreeBlocks[Addr] = false; - BlockMapAddr = Addr; - return Error::success(); -} - -void MsfBuilder::setFreePageMap(uint32_t Fpm) { FreePageMap = Fpm; } - -void MsfBuilder::setUnknown1(uint32_t Unk1) { Unknown1 = Unk1; } - -Error MsfBuilder::setDirectoryBlocksHint(ArrayRef<uint32_t> DirBlocks) { - for (auto B : DirectoryBlocks) - FreeBlocks[B] = true; - for (auto B : DirBlocks) { - if (!isBlockFree(B)) { - return make_error<RawError>(raw_error_code::unspecified, - "Attempt to reuse an allocated block"); - } - FreeBlocks[B] = false; - } - - DirectoryBlocks = DirBlocks; - return Error::success(); -} - -Error MsfBuilder::allocateBlocks(uint32_t NumBlocks, - MutableArrayRef<uint32_t> Blocks) { - if (NumBlocks == 0) - return Error::success(); - - uint32_t NumFreeBlocks = FreeBlocks.count(); - if (NumFreeBlocks < NumBlocks) { - if (!IsGrowable) - return make_error<RawError>(raw_error_code::unspecified, - "There are no free Blocks in the file"); - uint32_t AllocBlocks = NumBlocks - NumFreeBlocks; - FreeBlocks.resize(AllocBlocks + FreeBlocks.size(), true); - } - - int I = 0; - int Block = FreeBlocks.find_first(); - do { - assert(Block != -1 && "We ran out of Blocks!"); - - uint32_t NextBlock = static_cast<uint32_t>(Block); - Blocks[I++] = NextBlock; - FreeBlocks.reset(NextBlock); - Block = FreeBlocks.find_next(Block); - } while (--NumBlocks > 0); - return Error::success(); -} - -uint32_t MsfBuilder::getNumUsedBlocks() const { - return getTotalBlockCount() - getNumFreeBlocks(); -} - -uint32_t MsfBuilder::getNumFreeBlocks() const { return FreeBlocks.count(); } - -uint32_t MsfBuilder::getTotalBlockCount() const { return FreeBlocks.size(); } - -bool MsfBuilder::isBlockFree(uint32_t Idx) const { return FreeBlocks[Idx]; } - -Error MsfBuilder::addStream(uint32_t Size, ArrayRef<uint32_t> Blocks) { - // Add a new stream mapped to the specified blocks. Verify that the specified - // blocks are both necessary and sufficient for holding the requested number - // of bytes, and verify that all requested blocks are free. - uint32_t ReqBlocks = bytesToBlocks(Size, BlockSize); - if (ReqBlocks != Blocks.size()) - return make_error<RawError>( - raw_error_code::unspecified, - "Incorrect number of blocks for requested stream size"); - for (auto Block : Blocks) { - if (Block >= FreeBlocks.size()) - FreeBlocks.resize(Block + 1, true); - - if (!FreeBlocks.test(Block)) - return make_error<RawError>( - raw_error_code::unspecified, - "Attempt to re-use an already allocated block"); - } - // Mark all the blocks occupied by the new stream as not free. - for (auto Block : Blocks) { - FreeBlocks.reset(Block); - } - StreamData.push_back(std::make_pair(Size, Blocks)); - return Error::success(); -} - -Error MsfBuilder::addStream(uint32_t Size) { - uint32_t ReqBlocks = bytesToBlocks(Size, BlockSize); - std::vector<uint32_t> NewBlocks; - NewBlocks.resize(ReqBlocks); - if (auto EC = allocateBlocks(ReqBlocks, NewBlocks)) - return EC; - StreamData.push_back(std::make_pair(Size, NewBlocks)); - return Error::success(); -} - -Error MsfBuilder::setStreamSize(uint32_t Idx, uint32_t Size) { - uint32_t OldSize = getStreamSize(Idx); - if (OldSize == Size) - return Error::success(); - - uint32_t NewBlocks = bytesToBlocks(Size, BlockSize); - uint32_t OldBlocks = bytesToBlocks(OldSize, BlockSize); - - if (NewBlocks > OldBlocks) { - uint32_t AddedBlocks = NewBlocks - OldBlocks; - // If we're growing, we have to allocate new Blocks. - std::vector<uint32_t> AddedBlockList; - AddedBlockList.resize(AddedBlocks); - if (auto EC = allocateBlocks(AddedBlocks, AddedBlockList)) - return EC; - auto &CurrentBlocks = StreamData[Idx].second; - CurrentBlocks.insert(CurrentBlocks.end(), AddedBlockList.begin(), - AddedBlockList.end()); - } else if (OldBlocks > NewBlocks) { - // For shrinking, free all the Blocks in the Block map, update the stream - // data, then shrink the directory. - uint32_t RemovedBlocks = OldBlocks - NewBlocks; - auto CurrentBlocks = ArrayRef<uint32_t>(StreamData[Idx].second); - auto RemovedBlockList = CurrentBlocks.drop_front(NewBlocks); - for (auto P : RemovedBlockList) - FreeBlocks[P] = true; - StreamData[Idx].second = CurrentBlocks.drop_back(RemovedBlocks); - } - - StreamData[Idx].first = Size; - return Error::success(); -} - -uint32_t MsfBuilder::getNumStreams() const { return StreamData.size(); } - -uint32_t MsfBuilder::getStreamSize(uint32_t StreamIdx) const { - return StreamData[StreamIdx].first; -} - -ArrayRef<uint32_t> MsfBuilder::getStreamBlocks(uint32_t StreamIdx) const { - return StreamData[StreamIdx].second; -} - -uint32_t MsfBuilder::computeDirectoryByteSize() const { - // The directory has the following layout, where each item is a ulittle32_t: - // NumStreams - // StreamSizes[NumStreams] - // StreamBlocks[NumStreams][] - uint32_t Size = sizeof(ulittle32_t); // NumStreams - Size += StreamData.size() * sizeof(ulittle32_t); // StreamSizes - for (const auto &D : StreamData) { - uint32_t ExpectedNumBlocks = bytesToBlocks(D.first, BlockSize); - assert(ExpectedNumBlocks == D.second.size() && - "Unexpected number of blocks"); - Size += ExpectedNumBlocks * sizeof(ulittle32_t); - } - return Size; -} - -Expected<Layout> MsfBuilder::build() { - Layout L; - L.SB = Allocator.Allocate<SuperBlock>(); - std::memcpy(L.SB->MagicBytes, Magic, sizeof(Magic)); - L.SB->BlockMapAddr = BlockMapAddr; - L.SB->BlockSize = BlockSize; - L.SB->NumDirectoryBytes = computeDirectoryByteSize(); - L.SB->FreeBlockMapBlock = FreePageMap; - L.SB->Unknown1 = Unknown1; - - uint32_t NumDirectoryBlocks = - bytesToBlocks(L.SB->NumDirectoryBytes, BlockSize); - if (NumDirectoryBlocks > DirectoryBlocks.size()) { - // Our hint wasn't enough to satisfy the entire directory. Allocate - // remaining pages. - std::vector<uint32_t> ExtraBlocks; - uint32_t NumExtraBlocks = NumDirectoryBlocks - DirectoryBlocks.size(); - ExtraBlocks.resize(NumExtraBlocks); - if (auto EC = allocateBlocks(NumExtraBlocks, ExtraBlocks)) - return std::move(EC); - DirectoryBlocks.insert(DirectoryBlocks.end(), ExtraBlocks.begin(), - ExtraBlocks.end()); - } else if (NumDirectoryBlocks < DirectoryBlocks.size()) { - uint32_t NumUnnecessaryBlocks = DirectoryBlocks.size() - NumDirectoryBlocks; - for (auto B : - ArrayRef<uint32_t>(DirectoryBlocks).drop_back(NumUnnecessaryBlocks)) - FreeBlocks[B] = true; - DirectoryBlocks.resize(NumDirectoryBlocks); - } - - // Don't set the number of blocks in the file until after allocating Blocks - // for - // the directory, since the allocation might cause the file to need to grow. - L.SB->NumBlocks = FreeBlocks.size(); - - ulittle32_t *DirBlocks = Allocator.Allocate<ulittle32_t>(NumDirectoryBlocks); - std::uninitialized_copy_n(DirectoryBlocks.begin(), NumDirectoryBlocks, - DirBlocks); - L.DirectoryBlocks = ArrayRef<ulittle32_t>(DirBlocks, NumDirectoryBlocks); - - // The stream sizes should be re-allocated as a stable pointer and the stream - // map should have each of its entries allocated as a separate stable pointer. - if (StreamData.size() > 0) { - ulittle32_t *Sizes = Allocator.Allocate<ulittle32_t>(StreamData.size()); - L.StreamSizes = ArrayRef<ulittle32_t>(Sizes, StreamData.size()); - L.StreamMap.resize(StreamData.size()); - for (uint32_t I = 0; I < StreamData.size(); ++I) { - Sizes[I] = StreamData[I].first; - ulittle32_t *BlockList = - Allocator.Allocate<ulittle32_t>(StreamData[I].second.size()); - std::uninitialized_copy_n(StreamData[I].second.begin(), - StreamData[I].second.size(), BlockList); - L.StreamMap[I] = - ArrayRef<ulittle32_t>(BlockList, StreamData[I].second.size()); - } - } - - return L; -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfCommon.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfCommon.cpp deleted file mode 100644 index 5d97f33..0000000 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/MsfCommon.cpp +++ /dev/null @@ -1,48 +0,0 @@ -//===- MsfCommon.cpp - Common types and functions for MSF files -*- C++ -*-===// -// -// The LLVM Compiler Infrastructure -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "llvm/DebugInfo/PDB/Raw/MsfCommon.h" -#include "llvm/DebugInfo/PDB/Raw/RawError.h" - -using namespace llvm; -using namespace llvm::pdb::msf; - -Error llvm::pdb::msf::validateSuperBlock(const SuperBlock &SB) { - // Check the magic bytes. - if (std::memcmp(SB.MagicBytes, Magic, sizeof(Magic)) != 0) - return make_error<RawError>(raw_error_code::corrupt_file, - "MSF magic header doesn't match"); - - if (!isValidBlockSize(SB.BlockSize)) - return make_error<RawError>(raw_error_code::corrupt_file, - "Unsupported block size."); - - // We don't support directories whose sizes aren't a multiple of four bytes. - if (SB.NumDirectoryBytes % sizeof(support::ulittle32_t) != 0) - return make_error<RawError>(raw_error_code::corrupt_file, - "Directory size is not multiple of 4."); - - // The number of blocks which comprise the directory is a simple function of - // the number of bytes it contains. - uint64_t NumDirectoryBlocks = - bytesToBlocks(SB.NumDirectoryBytes, SB.BlockSize); - - // The directory, as we understand it, is a block which consists of a list of - // block numbers. It is unclear what would happen if the number of blocks - // couldn't fit on a single block. - if (NumDirectoryBlocks > SB.BlockSize / sizeof(support::ulittle32_t)) - return make_error<RawError>(raw_error_code::corrupt_file, - "Too many directory blocks."); - - if (SB.BlockMapAddr == 0) - return make_error<RawError>(raw_error_code::corrupt_file, - "Block 0 is reserved"); - - return Error::success(); -} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameHashTable.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameHashTable.cpp index ae4ebf2..84cccb35 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameHashTable.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameHashTable.cpp @@ -10,18 +10,19 @@ #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h" #include "llvm/ADT/ArrayRef.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/Hash.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/Support/Endian.h" using namespace llvm; +using namespace llvm::msf; using namespace llvm::support; using namespace llvm::pdb; NameHashTable::NameHashTable() : Signature(0), HashVersion(0), NameCount(0) {} -Error NameHashTable::load(codeview::StreamReader &Stream) { +Error NameHashTable::load(StreamReader &Stream) { struct Header { support::ulittle32_t Signature; support::ulittle32_t HashVersion; @@ -72,7 +73,7 @@ StringRef NameHashTable::getStringForID(uint32_t ID) const { // the starting offset of the string we're looking for. So just seek into // the desired offset and a read a null terminated stream from that offset. StringRef Result; - codeview::StreamReader NameReader(NamesBuffer); + StreamReader NameReader(NamesBuffer); NameReader.setOffset(ID); if (auto EC = NameReader.readZeroString(Result)) consumeError(std::move(EC)); @@ -98,7 +99,6 @@ uint32_t NameHashTable::getIDForString(StringRef Str) const { return IDs[0]; } -codeview::FixedStreamArray<support::ulittle32_t> -NameHashTable::name_ids() const { +FixedStreamArray<support::ulittle32_t> NameHashTable::name_ids() const { return IDs; } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMap.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMap.cpp index b8a4eb7..0f55f58 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMap.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMap.cpp @@ -7,20 +7,24 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/NameMap.h" #include "llvm/ADT/SparseBitVector.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" +#include "llvm/ADT/StringMap.h" +#include "llvm/ADT/StringRef.h" +#include "llvm/ADT/iterator_range.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/PDB/Raw/NameMap.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> using namespace llvm; -using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; -NameMap::NameMap() {} - -Error NameMap::load(codeview::StreamReader &Stream) { +NameMap::NameMap() = default; +Error NameMap::load(StreamReader &Stream) { // This is some sort of weird string-set/hash table encoded in the stream. // It starts with the number of bytes in the table. uint32_t NumberOfBytes; @@ -145,63 +149,9 @@ Error NameMap::load(codeview::StreamReader &Stream) { return Error::success(); } -Error NameMap::commit(codeview::StreamWriter &Writer) { - // The first field is the number of bytes of string data. So add - // up the length of all strings plus a null terminator for each - // one. - uint32_t NumBytes = 0; - for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) { - NumBytes += B->getKeyLength() + 1; - } - - if (auto EC = Writer.writeInteger(NumBytes)) // Number of bytes of string data - return EC; - // Now all of the string data itself. - for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) { - if (auto EC = Writer.writeZeroString(B->getKey())) - return EC; - } - - if (auto EC = Writer.writeInteger(Mapping.size())) // Hash Size - return EC; - - if (auto EC = Writer.writeInteger(Mapping.size())) // Max Number of Strings - return EC; - - if (auto EC = Writer.writeInteger(Mapping.size())) // Num Present Words - return EC; - - // For each entry in the mapping, write a bit mask which represents a bucket - // to store it in. We don't use this, so the value we write isn't important - // to us, it just has to be there. - for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) { - if (auto EC = Writer.writeInteger(1U)) - return EC; - } - - if (auto EC = Writer.writeInteger(0U)) // Num Deleted Words - return EC; - - // Mappings of each word. - uint32_t OffsetSoFar = 0; - for (auto B = Mapping.begin(), E = Mapping.end(); B != E; ++B) { - // This is a list of key value pairs where the key is the offset into the - // strings buffer, and the value is a stream number. Write each pair. - if (auto EC = Writer.writeInteger(OffsetSoFar)) - return EC; - - if (auto EC = Writer.writeInteger(B->second)) - return EC; - - OffsetSoFar += B->getKeyLength() + 1; - } - - return Error::success(); -} - iterator_range<StringMapConstIterator<uint32_t>> NameMap::entries() const { - return llvm::make_range<StringMapConstIterator<uint32_t>>(Mapping.begin(), - Mapping.end()); + return make_range<StringMapConstIterator<uint32_t>>(Mapping.begin(), + Mapping.end()); } bool NameMap::tryGetValue(StringRef Name, uint32_t &Value) const { diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMapBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMapBuilder.cpp index 41c6c2c..f570d59 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMapBuilder.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/NameMapBuilder.cpp @@ -7,15 +7,19 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/NameMapBuilder.h" - +#include "llvm/ADT/STLExtras.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" #include "llvm/DebugInfo/PDB/Raw/NameMap.h" +#include "llvm/DebugInfo/PDB/Raw/NameMapBuilder.h" #include "llvm/Support/Endian.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> using namespace llvm; using namespace llvm::pdb; -NameMapBuilder::NameMapBuilder() {} +NameMapBuilder::NameMapBuilder() = default; void NameMapBuilder::addMapping(StringRef Name, uint32_t Mapping) { StringDataBytes += Name.size() + 1; @@ -48,3 +52,57 @@ uint32_t NameMapBuilder::calculateSerializedLength() const { return TotalLength; } + +Error NameMapBuilder::commit(msf::StreamWriter &Writer) const { + // The first field is the number of bytes of string data. So add + // up the length of all strings plus a null terminator for each + // one. + uint32_t NumBytes = 0; + for (auto B = Map.begin(), E = Map.end(); B != E; ++B) { + NumBytes += B->getKeyLength() + 1; + } + + if (auto EC = Writer.writeInteger(NumBytes)) // Number of bytes of string data + return EC; + // Now all of the string data itself. + for (auto B = Map.begin(), E = Map.end(); B != E; ++B) { + if (auto EC = Writer.writeZeroString(B->getKey())) + return EC; + } + + if (auto EC = Writer.writeInteger(Map.size())) // Hash Size + return EC; + + if (auto EC = Writer.writeInteger(Map.size())) // Max Number of Strings + return EC; + + if (auto EC = Writer.writeInteger(Map.size())) // Num Present Words + return EC; + + // For each entry in the mapping, write a bit mask which represents a bucket + // to store it in. We don't use this, so the value we write isn't important + // to us, it just has to be there. + for (auto B = Map.begin(), E = Map.end(); B != E; ++B) { + if (auto EC = Writer.writeInteger(1U)) + return EC; + } + + if (auto EC = Writer.writeInteger(0U)) // Num Deleted Words + return EC; + + // Mappings of each word. + uint32_t OffsetSoFar = 0; + for (auto B = Map.begin(), E = Map.end(); B != E; ++B) { + // This is a list of key value pairs where the key is the offset into the + // strings buffer, and the value is a stream number. Write each pair. + if (auto EC = Writer.writeInteger(OffsetSoFar)) + return EC; + + if (auto EC = Writer.writeInteger(B->second)) + return EC; + + OffsetSoFar += B->getKeyLength() + 1; + } + + return Error::success(); +} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFile.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFile.cpp index 9501675..5349151 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFile.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFile.cpp @@ -7,68 +7,84 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/PDBFile.h" - #include "llvm/ADT/ArrayRef.h" -#include "llvm/DebugInfo/CodeView/StreamArray.h" -#include "llvm/DebugInfo/CodeView/StreamInterface.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" +#include "llvm/ADT/STLExtras.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/MSFCommon.h" +#include "llvm/DebugInfo/MSF/StreamArray.h" +#include "llvm/DebugInfo/MSF/StreamInterface.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/DbiStream.h" -#include "llvm/DebugInfo/PDB/Raw/DirectoryStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" +#include "llvm/DebugInfo/PDB/Raw/GlobalsStream.h" #include "llvm/DebugInfo/PDB/Raw/InfoStream.h" #include "llvm/DebugInfo/PDB/Raw/NameHashTable.h" +#include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/PublicsStream.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/DebugInfo/PDB/Raw/SymbolStream.h" #include "llvm/DebugInfo/PDB/Raw/TpiStream.h" #include "llvm/Support/Endian.h" -#include "llvm/Support/FileOutputBuffer.h" -#include "llvm/Support/MemoryBuffer.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cassert> +#include <cstdint> using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; namespace { typedef FixedStreamArray<support::ulittle32_t> ulittle_array; -} +} // end anonymous namespace -PDBFile::PDBFile(std::unique_ptr<StreamInterface> PdbFileBuffer) - : Buffer(std::move(PdbFileBuffer)), SB(nullptr) {} +PDBFile::PDBFile(std::unique_ptr<ReadableStream> PdbFileBuffer, + BumpPtrAllocator &Allocator) + : Allocator(Allocator), Buffer(std::move(PdbFileBuffer)) {} -PDBFile::~PDBFile() {} +PDBFile::~PDBFile() = default; -uint32_t PDBFile::getBlockSize() const { return SB->BlockSize; } +uint32_t PDBFile::getBlockSize() const { return ContainerLayout.SB->BlockSize; } -uint32_t PDBFile::getFreeBlockMapBlock() const { return SB->FreeBlockMapBlock; } +uint32_t PDBFile::getFreeBlockMapBlock() const { + return ContainerLayout.SB->FreeBlockMapBlock; +} -uint32_t PDBFile::getBlockCount() const { return SB->NumBlocks; } +uint32_t PDBFile::getBlockCount() const { + return ContainerLayout.SB->NumBlocks; +} -uint32_t PDBFile::getNumDirectoryBytes() const { return SB->NumDirectoryBytes; } +uint32_t PDBFile::getNumDirectoryBytes() const { + return ContainerLayout.SB->NumDirectoryBytes; +} -uint32_t PDBFile::getBlockMapIndex() const { return SB->BlockMapAddr; } +uint32_t PDBFile::getBlockMapIndex() const { + return ContainerLayout.SB->BlockMapAddr; +} -uint32_t PDBFile::getUnknown1() const { return SB->Unknown1; } +uint32_t PDBFile::getUnknown1() const { return ContainerLayout.SB->Unknown1; } uint32_t PDBFile::getNumDirectoryBlocks() const { - return msf::bytesToBlocks(SB->NumDirectoryBytes, SB->BlockSize); + return msf::bytesToBlocks(ContainerLayout.SB->NumDirectoryBytes, + ContainerLayout.SB->BlockSize); } uint64_t PDBFile::getBlockMapOffset() const { - return (uint64_t)SB->BlockMapAddr * SB->BlockSize; + return (uint64_t)ContainerLayout.SB->BlockMapAddr * + ContainerLayout.SB->BlockSize; } -uint32_t PDBFile::getNumStreams() const { return StreamSizes.size(); } +uint32_t PDBFile::getNumStreams() const { + return ContainerLayout.StreamSizes.size(); +} uint32_t PDBFile::getStreamByteSize(uint32_t StreamIndex) const { - return StreamSizes[StreamIndex]; + return ContainerLayout.StreamSizes[StreamIndex]; } ArrayRef<support::ulittle32_t> PDBFile::getStreamBlockList(uint32_t StreamIndex) const { - return StreamMap[StreamIndex]; + return ContainerLayout.StreamMap[StreamIndex]; } uint32_t PDBFile::getFileSize() const { return Buffer->getLength(); } @@ -85,41 +101,72 @@ Expected<ArrayRef<uint8_t>> PDBFile::getBlockData(uint32_t BlockIndex, Error PDBFile::setBlockData(uint32_t BlockIndex, uint32_t Offset, ArrayRef<uint8_t> Data) const { - if (Offset >= getBlockSize()) - return make_error<RawError>( - raw_error_code::invalid_block_address, - "setBlockData attempted to write out of block bounds."); - if (Data.size() > getBlockSize() - Offset) - return make_error<RawError>( - raw_error_code::invalid_block_address, - "setBlockData attempted to write out of block bounds."); - - uint64_t StreamBlockOffset = msf::blockToOffset(BlockIndex, getBlockSize()); - StreamBlockOffset += Offset; - return Buffer->writeBytes(StreamBlockOffset, Data); + return make_error<RawError>(raw_error_code::not_writable, + "PDBFile is immutable"); } Error PDBFile::parseFileHeaders() { StreamReader Reader(*Buffer); + // Initialize SB. + const msf::SuperBlock *SB = nullptr; if (auto EC = Reader.readObject(SB)) { consumeError(std::move(EC)); return make_error<RawError>(raw_error_code::corrupt_file, "Does not contain superblock"); } - if (auto EC = setSuperBlock(SB)) + if (auto EC = msf::validateSuperBlock(*SB)) + return EC; + + if (Buffer->getLength() % SB->BlockSize != 0) + return make_error<RawError>(raw_error_code::corrupt_file, + "File size is not a multiple of block size"); + ContainerLayout.SB = SB; + + // Initialize Free Page Map. + ContainerLayout.FreePageMap.resize(SB->NumBlocks); + // The Fpm exists either at block 1 or block 2 of the MSF. However, this + // allows for a maximum of getBlockSize() * 8 blocks bits in the Fpm, and + // thusly an equal number of total blocks in the file. For a block size + // of 4KiB (very common), this would yield 32KiB total blocks in file, for a + // maximum file size of 32KiB * 4KiB = 128MiB. Obviously this won't do, so + // the Fpm is split across the file at `getBlockSize()` intervals. As a + // result, every block whose index is of the form |{1,2} + getBlockSize() * k| + // for any non-negative integer k is an Fpm block. In theory, we only really + // need to reserve blocks of the form |{1,2} + getBlockSize() * 8 * k|, but + // current versions of the MSF format already expect the Fpm to be arranged + // at getBlockSize() intervals, so we have to be compatible. + // See the function fpmPn() for more information: + // https://github.com/Microsoft/microsoft-pdb/blob/master/PDB/msf/msf.cpp#L489 + auto FpmStream = MappedBlockStream::createFpmStream(ContainerLayout, *Buffer); + StreamReader FpmReader(*FpmStream); + ArrayRef<uint8_t> FpmBytes; + if (auto EC = FpmReader.readBytes(FpmBytes, + msf::getFullFpmByteSize(ContainerLayout))) return EC; + uint32_t BlocksRemaining = getBlockCount(); + uint32_t BI = 0; + for (auto Byte : FpmBytes) { + uint32_t BlocksThisByte = std::min(BlocksRemaining, 8U); + for (uint32_t I = 0; I < BlocksThisByte; ++I) { + if (Byte & (1 << I)) + ContainerLayout.FreePageMap[BI] = true; + --BlocksRemaining; + ++BI; + } + } Reader.setOffset(getBlockMapOffset()); - if (auto EC = Reader.readArray(DirectoryBlocks, getNumDirectoryBlocks())) + if (auto EC = Reader.readArray(ContainerLayout.DirectoryBlocks, + getNumDirectoryBlocks())) return EC; return Error::success(); } Error PDBFile::parseStreamData() { - assert(SB); + assert(ContainerLayout.SB); if (DirectoryStream) return Error::success(); @@ -130,21 +177,20 @@ Error PDBFile::parseStreamData() { // is exactly what we are attempting to parse. By specifying a custom // subclass of IPDBStreamData which only accesses the fields that have already // been parsed, we can avoid this and reuse MappedBlockStream. - auto DS = MappedBlockStream::createDirectoryStream(*this); - if (!DS) - return DS.takeError(); - StreamReader Reader(**DS); + auto DS = MappedBlockStream::createDirectoryStream(ContainerLayout, *Buffer); + StreamReader Reader(*DS); if (auto EC = Reader.readInteger(NumStreams)) return EC; - if (auto EC = Reader.readArray(StreamSizes, NumStreams)) + if (auto EC = Reader.readArray(ContainerLayout.StreamSizes, NumStreams)) return EC; for (uint32_t I = 0; I < NumStreams; ++I) { uint32_t StreamSize = getStreamByteSize(I); // FIXME: What does StreamSize ~0U mean? uint64_t NumExpectedStreamBlocks = - StreamSize == UINT32_MAX ? 0 : msf::bytesToBlocks(StreamSize, - SB->BlockSize); + StreamSize == UINT32_MAX + ? 0 + : msf::bytesToBlocks(StreamSize, ContainerLayout.SB->BlockSize); // For convenience, we store the block array contiguously. This is because // if someone calls setStreamMap(), it is more convenient to be able to call @@ -156,29 +202,46 @@ Error PDBFile::parseStreamData() { if (auto EC = Reader.readArray(Blocks, NumExpectedStreamBlocks)) return EC; for (uint32_t Block : Blocks) { - uint64_t BlockEndOffset = (uint64_t)(Block + 1) * SB->BlockSize; + uint64_t BlockEndOffset = + (uint64_t)(Block + 1) * ContainerLayout.SB->BlockSize; if (BlockEndOffset > getFileSize()) return make_error<RawError>(raw_error_code::corrupt_file, "Stream block map is corrupt."); } - StreamMap.push_back(Blocks); + ContainerLayout.StreamMap.push_back(Blocks); } // We should have read exactly SB->NumDirectoryBytes bytes. assert(Reader.bytesRemaining() == 0); - DirectoryStream = std::move(*DS); + DirectoryStream = std::move(DS); return Error::success(); } -llvm::ArrayRef<support::ulittle32_t> PDBFile::getDirectoryBlockArray() const { - return DirectoryBlocks; +ArrayRef<support::ulittle32_t> PDBFile::getDirectoryBlockArray() const { + return ContainerLayout.DirectoryBlocks; +} + +Expected<GlobalsStream &> PDBFile::getPDBGlobalsStream() { + if (!Globals) { + auto DbiS = getPDBDbiStream(); + if (!DbiS) + return DbiS.takeError(); + + auto GlobalS = safelyCreateIndexedStream( + ContainerLayout, *Buffer, DbiS->getGlobalSymbolStreamIndex()); + if (!GlobalS) return GlobalS.takeError(); + auto TempGlobals = llvm::make_unique<GlobalsStream>(std::move(*GlobalS)); + if (auto EC = TempGlobals->reload()) + return std::move(EC); + Globals = std::move(TempGlobals); + } + return *Globals; } Expected<InfoStream &> PDBFile::getPDBInfoStream() { if (!Info) { - auto InfoS = MappedBlockStream::createIndexedStream(StreamPDB, *this); - if (!InfoS) - return InfoS.takeError(); + auto InfoS = safelyCreateIndexedStream(ContainerLayout, *Buffer, StreamPDB); + if (!InfoS) return InfoS.takeError(); auto TempInfo = llvm::make_unique<InfoStream>(std::move(*InfoS)); if (auto EC = TempInfo->reload()) return std::move(EC); @@ -189,9 +252,8 @@ Expected<InfoStream &> PDBFile::getPDBInfoStream() { Expected<DbiStream &> PDBFile::getPDBDbiStream() { if (!Dbi) { - auto DbiS = MappedBlockStream::createIndexedStream(StreamDBI, *this); - if (!DbiS) - return DbiS.takeError(); + auto DbiS = safelyCreateIndexedStream(ContainerLayout, *Buffer, StreamDBI); + if (!DbiS) return DbiS.takeError(); auto TempDbi = llvm::make_unique<DbiStream>(*this, std::move(*DbiS)); if (auto EC = TempDbi->reload()) return std::move(EC); @@ -202,9 +264,8 @@ Expected<DbiStream &> PDBFile::getPDBDbiStream() { Expected<TpiStream &> PDBFile::getPDBTpiStream() { if (!Tpi) { - auto TpiS = MappedBlockStream::createIndexedStream(StreamTPI, *this); - if (!TpiS) - return TpiS.takeError(); + auto TpiS = safelyCreateIndexedStream(ContainerLayout, *Buffer, StreamTPI); + if (!TpiS) return TpiS.takeError(); auto TempTpi = llvm::make_unique<TpiStream>(*this, std::move(*TpiS)); if (auto EC = TempTpi->reload()) return std::move(EC); @@ -215,9 +276,8 @@ Expected<TpiStream &> PDBFile::getPDBTpiStream() { Expected<TpiStream &> PDBFile::getPDBIpiStream() { if (!Ipi) { - auto IpiS = MappedBlockStream::createIndexedStream(StreamIPI, *this); - if (!IpiS) - return IpiS.takeError(); + auto IpiS = safelyCreateIndexedStream(ContainerLayout, *Buffer, StreamIPI); + if (!IpiS) return IpiS.takeError(); auto TempIpi = llvm::make_unique<TpiStream>(*this, std::move(*IpiS)); if (auto EC = TempIpi->reload()) return std::move(EC); @@ -232,12 +292,9 @@ Expected<PublicsStream &> PDBFile::getPDBPublicsStream() { if (!DbiS) return DbiS.takeError(); - uint32_t PublicsStreamNum = DbiS->getPublicSymbolStreamIndex(); - - auto PublicS = - MappedBlockStream::createIndexedStream(PublicsStreamNum, *this); - if (!PublicS) - return PublicS.takeError(); + auto PublicS = safelyCreateIndexedStream( + ContainerLayout, *Buffer, DbiS->getPublicSymbolStreamIndex()); + if (!PublicS) return PublicS.takeError(); auto TempPublics = llvm::make_unique<PublicsStream>(*this, std::move(*PublicS)); if (auto EC = TempPublics->reload()) @@ -254,11 +311,10 @@ Expected<SymbolStream &> PDBFile::getPDBSymbolStream() { return DbiS.takeError(); uint32_t SymbolStreamNum = DbiS->getSymRecordStreamIndex(); - auto SymbolS = - MappedBlockStream::createIndexedStream(SymbolStreamNum, *this); - if (!SymbolS) - return SymbolS.takeError(); + safelyCreateIndexedStream(ContainerLayout, *Buffer, SymbolStreamNum); + if (!SymbolS) return SymbolS.takeError(); + auto TempSymbols = llvm::make_unique<SymbolStream>(std::move(*SymbolS)); if (auto EC = TempSymbols->reload()) return std::move(EC); @@ -275,14 +331,9 @@ Expected<NameHashTable &> PDBFile::getStringTable() { uint32_t NameStreamIndex = IS->getNamedStreamIndex("/names"); - if (NameStreamIndex == 0) - return make_error<RawError>(raw_error_code::no_stream); - if (NameStreamIndex >= getNumStreams()) - return make_error<RawError>(raw_error_code::no_stream); - - auto NS = MappedBlockStream::createIndexedStream(NameStreamIndex, *this); - if (!NS) - return NS.takeError(); + auto NS = + safelyCreateIndexedStream(ContainerLayout, *Buffer, NameStreamIndex); + if (!NS) return NS.takeError(); StreamReader Reader(**NS); auto N = llvm::make_unique<NameHashTable>(); @@ -294,72 +345,47 @@ Expected<NameHashTable &> PDBFile::getStringTable() { return *StringTable; } -Error PDBFile::setSuperBlock(const msf::SuperBlock *Block) { - if (auto EC = msf::validateSuperBlock(*Block)) - return EC; - - if (Buffer->getLength() % SB->BlockSize != 0) - return make_error<RawError>(raw_error_code::corrupt_file, - "File size is not a multiple of block size"); +bool PDBFile::hasPDBDbiStream() const { return StreamDBI < getNumStreams(); } - SB = Block; - return Error::success(); +bool PDBFile::hasPDBGlobalsStream() { + auto DbiS = getPDBDbiStream(); + if (!DbiS) return false; + return DbiS->getGlobalSymbolStreamIndex() < getNumStreams(); } -Error PDBFile::commit() { - StreamWriter Writer(*Buffer); - - if (auto EC = Writer.writeObject(*SB)) - return EC; - Writer.setOffset(getBlockMapOffset()); - if (auto EC = Writer.writeArray(DirectoryBlocks)) - return EC; - - auto DS = MappedBlockStream::createDirectoryStream(*this); - if (!DS) - return DS.takeError(); - auto DirStream = std::move(*DS); - StreamWriter DW(*DirStream); - if (auto EC = DW.writeInteger(this->getNumStreams())) - return EC; - - if (auto EC = DW.writeArray(StreamSizes)) - return EC; - - for (const auto &Blocks : StreamMap) { - if (auto EC = DW.writeArray(Blocks)) - return EC; - } - - if (Info) { - if (auto EC = Info->commit()) - return EC; - } +bool PDBFile::hasPDBInfoStream() { return StreamPDB < getNumStreams(); } - if (Dbi) { - if (auto EC = Dbi->commit()) - return EC; - } +bool PDBFile::hasPDBIpiStream() const { return StreamIPI < getNumStreams(); } - if (Symbols) { - if (auto EC = Symbols->commit()) - return EC; - } +bool PDBFile::hasPDBPublicsStream() { + auto DbiS = getPDBDbiStream(); + if (!DbiS) return false; + return DbiS->getPublicSymbolStreamIndex() < getNumStreams(); +} - if (Publics) { - if (auto EC = Publics->commit()) - return EC; - } +bool PDBFile::hasPDBSymbolStream() { + auto DbiS = getPDBDbiStream(); + if (!DbiS) return false; + return DbiS->getSymRecordStreamIndex() < getNumStreams(); +} - if (Tpi) { - if (auto EC = Tpi->commit()) - return EC; - } +bool PDBFile::hasPDBTpiStream() const { return StreamTPI < getNumStreams(); } - if (Ipi) { - if (auto EC = Ipi->commit()) - return EC; - } +bool PDBFile::hasStringTable() { + auto IS = getPDBInfoStream(); + if (!IS) return false; + return IS->getNamedStreamIndex("/names") < getNumStreams(); +} - return Buffer->commit(); +/// Wrapper around MappedBlockStream::createIndexedStream() +/// that checks if a stream with that index actually exists. +/// If it does not, the return value will have an MSFError with +/// code msf_error_code::no_stream. Else, the return value will +/// contain the stream returned by createIndexedStream(). +Expected<std::unique_ptr<MappedBlockStream>> PDBFile::safelyCreateIndexedStream( + const MSFLayout &Layout, const ReadableStream &MsfData, + uint32_t StreamIndex) const { + if (StreamIndex >= getNumStreams()) + return make_error<RawError>(raw_error_code::no_stream); + return MappedBlockStream::createIndexedStream(Layout, MsfData, StreamIndex); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFileBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFileBuilder.cpp index 9063fd6..6fec0e3 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFileBuilder.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/PDBFileBuilder.cpp @@ -11,92 +11,138 @@ #include "llvm/ADT/BitVector.h" -#include "llvm/DebugInfo/CodeView/StreamInterface.h" -#include "llvm/DebugInfo/CodeView/StreamWriter.h" +#include "llvm/DebugInfo/MSF/MSFBuilder.h" +#include "llvm/DebugInfo/MSF/StreamInterface.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" +#include "llvm/DebugInfo/PDB/GenericError.h" #include "llvm/DebugInfo/PDB/Raw/DbiStream.h" #include "llvm/DebugInfo/PDB/Raw/DbiStreamBuilder.h" #include "llvm/DebugInfo/PDB/Raw/InfoStream.h" #include "llvm/DebugInfo/PDB/Raw/InfoStreamBuilder.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/DebugInfo/PDB/Raw/TpiStream.h" +#include "llvm/DebugInfo/PDB/Raw/TpiStreamBuilder.h" using namespace llvm; using namespace llvm::codeview; +using namespace llvm::msf; using namespace llvm::pdb; using namespace llvm::support; -PDBFileBuilder::PDBFileBuilder( - std::unique_ptr<codeview::StreamInterface> FileBuffer) - : File(llvm::make_unique<PDBFile>(std::move(FileBuffer))) {} +PDBFileBuilder::PDBFileBuilder(BumpPtrAllocator &Allocator) + : Allocator(Allocator) {} -Error PDBFileBuilder::initialize(const msf::SuperBlock &Super) { - auto ExpectedMsf = - MsfBuilder::create(File->Allocator, Super.BlockSize, Super.NumBlocks); +Error PDBFileBuilder::initialize(uint32_t BlockSize) { + auto ExpectedMsf = MSFBuilder::create(Allocator, BlockSize); if (!ExpectedMsf) return ExpectedMsf.takeError(); - - auto &MsfResult = *ExpectedMsf; - if (auto EC = MsfResult.setBlockMapAddr(Super.BlockMapAddr)) - return EC; - Msf = llvm::make_unique<MsfBuilder>(std::move(MsfResult)); - Msf->setFreePageMap(Super.FreeBlockMapBlock); - Msf->setUnknown1(Super.Unknown1); + Msf = llvm::make_unique<MSFBuilder>(std::move(*ExpectedMsf)); return Error::success(); } -MsfBuilder &PDBFileBuilder::getMsfBuilder() { return *Msf; } +MSFBuilder &PDBFileBuilder::getMsfBuilder() { return *Msf; } InfoStreamBuilder &PDBFileBuilder::getInfoBuilder() { if (!Info) - Info = llvm::make_unique<InfoStreamBuilder>(); + Info = llvm::make_unique<InfoStreamBuilder>(*Msf); return *Info; } DbiStreamBuilder &PDBFileBuilder::getDbiBuilder() { if (!Dbi) - Dbi = llvm::make_unique<DbiStreamBuilder>(); + Dbi = llvm::make_unique<DbiStreamBuilder>(*Msf); return *Dbi; } -Expected<std::unique_ptr<PDBFile>> PDBFileBuilder::build() { +TpiStreamBuilder &PDBFileBuilder::getTpiBuilder() { + if (!Tpi) + Tpi = llvm::make_unique<TpiStreamBuilder>(*Msf, StreamTPI); + return *Tpi; +} + +TpiStreamBuilder &PDBFileBuilder::getIpiBuilder() { + if (!Ipi) + Ipi = llvm::make_unique<TpiStreamBuilder>(*Msf, StreamIPI); + return *Ipi; +} + +Expected<msf::MSFLayout> PDBFileBuilder::finalizeMsfLayout() const { if (Info) { - uint32_t Length = Info->calculateSerializedLength(); - if (auto EC = Msf->setStreamSize(StreamPDB, Length)) + if (auto EC = Info->finalizeMsfLayout()) return std::move(EC); } if (Dbi) { - uint32_t Length = Dbi->calculateSerializedLength(); - if (auto EC = Msf->setStreamSize(StreamDBI, Length)) + if (auto EC = Dbi->finalizeMsfLayout()) return std::move(EC); } + if (Tpi) { + if (auto EC = Tpi->finalizeMsfLayout()) + return std::move(EC); + } + if (Ipi) { + if (auto EC = Ipi->finalizeMsfLayout()) + return std::move(EC); + } + + return Msf->build(); +} - auto ExpectedLayout = Msf->build(); +Error PDBFileBuilder::commit(StringRef Filename) { + auto ExpectedLayout = finalizeMsfLayout(); if (!ExpectedLayout) return ExpectedLayout.takeError(); + auto &Layout = *ExpectedLayout; - const msf::Layout &L = *ExpectedLayout; - File->StreamMap = L.StreamMap; - File->StreamSizes = L.StreamSizes; - File->DirectoryBlocks = L.DirectoryBlocks; - File->SB = L.SB; + uint64_t Filesize = Layout.SB->BlockSize * Layout.SB->NumBlocks; + auto OutFileOrError = FileOutputBuffer::create(Filename, Filesize); + if (OutFileOrError.getError()) + return llvm::make_error<pdb::GenericError>(generic_error_code::invalid_path, + Filename); + FileBufferByteStream Buffer(std::move(*OutFileOrError)); + StreamWriter Writer(Buffer); + + if (auto EC = Writer.writeObject(*Layout.SB)) + return EC; + uint32_t BlockMapOffset = + msf::blockToOffset(Layout.SB->BlockMapAddr, Layout.SB->BlockSize); + Writer.setOffset(BlockMapOffset); + if (auto EC = Writer.writeArray(Layout.DirectoryBlocks)) + return EC; + + auto DirStream = + WritableMappedBlockStream::createDirectoryStream(Layout, Buffer); + StreamWriter DW(*DirStream); + if (auto EC = + DW.writeInteger(static_cast<uint32_t>(Layout.StreamSizes.size()))) + return EC; + + if (auto EC = DW.writeArray(Layout.StreamSizes)) + return EC; + + for (const auto &Blocks : Layout.StreamMap) { + if (auto EC = DW.writeArray(Blocks)) + return EC; + } if (Info) { - auto ExpectedInfo = Info->build(*File); - if (!ExpectedInfo) - return ExpectedInfo.takeError(); - File->Info = std::move(*ExpectedInfo); + if (auto EC = Info->commit(Layout, Buffer)) + return EC; } if (Dbi) { - auto ExpectedDbi = Dbi->build(*File); - if (!ExpectedDbi) - return ExpectedDbi.takeError(); - File->Dbi = std::move(*ExpectedDbi); + if (auto EC = Dbi->commit(Layout, Buffer)) + return EC; } - if (File->Info && File->Dbi && File->Info->getAge() != File->Dbi->getAge()) - return llvm::make_error<RawError>( - raw_error_code::corrupt_file, - "PDB Stream Age doesn't match Dbi Stream Age!"); + if (Tpi) { + if (auto EC = Tpi->commit(Layout, Buffer)) + return EC; + } + + if (Ipi) { + if (auto EC = Ipi->commit(Layout, Buffer)) + return EC; + } - return std::move(File); + return Buffer.commit(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/PublicsStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/PublicsStream.cpp index af3d2d0..b31f605 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/PublicsStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/PublicsStream.cpp @@ -22,30 +22,25 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/PublicsStream.h" - -#include "llvm/DebugInfo/CodeView/CodeView.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/TypeRecord.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" +#include "GSI.h" +#include "llvm/ADT/iterator_range.h" +#include "llvm/DebugInfo/CodeView/SymbolRecord.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" -#include "llvm/DebugInfo/PDB/Raw/RawConstants.h" +#include "llvm/DebugInfo/PDB/Raw/PublicsStream.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/DebugInfo/PDB/Raw/SymbolStream.h" - -#include "llvm/ADT/BitVector.h" #include "llvm/Support/Endian.h" -#include "llvm/Support/Format.h" -#include "llvm/Support/MathExtras.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> using namespace llvm; +using namespace llvm::msf; using namespace llvm::support; using namespace llvm::pdb; - -static const unsigned IPHR_HASH = 4096; - // This is PSGSIHDR struct defined in // https://github.com/Microsoft/microsoft-pdb/blob/master/PDB/dbi/gsi.h struct PublicsStream::HeaderInfo { @@ -59,23 +54,11 @@ struct PublicsStream::HeaderInfo { ulittle32_t NumSections; }; -// This is GSIHashHdr. -struct PublicsStream::GSIHashHeader { - enum : unsigned { - HdrSignature = ~0U, - HdrVersion = 0xeffe0000 + 19990810, - }; - ulittle32_t VerSignature; - ulittle32_t VerHdr; - ulittle32_t HrSize; - ulittle32_t NumBuckets; -}; - PublicsStream::PublicsStream(PDBFile &File, std::unique_ptr<MappedBlockStream> Stream) : Pdb(File), Stream(std::move(Stream)) {} -PublicsStream::~PublicsStream() {} +PublicsStream::~PublicsStream() = default; uint32_t PublicsStream::getSymHash() const { return Header->SymHash; } uint32_t PublicsStream::getAddrMap() const { return Header->AddrMap; } @@ -86,7 +69,7 @@ uint32_t PublicsStream::getAddrMap() const { return Header->AddrMap; } // we skip over the hash table which we believe contains information about // public symbols. Error PublicsStream::reload() { - codeview::StreamReader Reader(*Stream); + StreamReader Reader(*Stream); // Check stream size. if (Reader.bytesRemaining() < sizeof(HeaderInfo) + sizeof(GSIHashHeader)) @@ -98,40 +81,15 @@ Error PublicsStream::reload() { return make_error<RawError>(raw_error_code::corrupt_file, "Publics Stream does not contain a header."); - if (Reader.readObject(HashHdr)) - return make_error<RawError>(raw_error_code::corrupt_file, - "Publics Stream does not contain a header."); + if (auto EC = readGSIHashHeader(HashHdr, Reader)) + return EC; - // An array of HashRecord follows. Read them. - if (HashHdr->HrSize % sizeof(PSHashRecord)) - return make_error<RawError>(raw_error_code::corrupt_file, - "Invalid HR array size."); - uint32_t NumHashRecords = HashHdr->HrSize / sizeof(PSHashRecord); - if (auto EC = Reader.readArray(HashRecords, NumHashRecords)) - return joinErrors(std::move(EC), - make_error<RawError>(raw_error_code::corrupt_file, - "Could not read an HR array")); + if (auto EC = readGSIHashRecords(HashRecords, HashHdr, Reader)) + return EC; - // A bitmap of a fixed length follows. - size_t BitmapSizeInBits = alignTo(IPHR_HASH + 1, 32); - uint32_t NumBitmapEntries = BitmapSizeInBits / 8; - if (auto EC = Reader.readBytes(Bitmap, NumBitmapEntries)) - return joinErrors(std::move(EC), - make_error<RawError>(raw_error_code::corrupt_file, - "Could not read a bitmap.")); - for (uint8_t B : Bitmap) - NumBuckets += countPopulation(B); - - // We don't yet understand the following data structures completely, - // but we at least know the types and sizes. Here we are trying - // to read the stream till end so that we at least can detect - // corrupted streams. - - // Hash buckets follow. - if (auto EC = Reader.readArray(HashBuckets, NumBuckets)) - return joinErrors(std::move(EC), - make_error<RawError>(raw_error_code::corrupt_file, - "Hash buckets corrupted.")); + if (auto EC = readGSIHashBuckets(HashBuckets, HashHdr, Reader)) + return EC; + NumBuckets = HashBuckets.size(); // Something called "address map" follows. uint32_t NumAddressMapEntries = Header->AddrMap / sizeof(uint32_t); @@ -163,7 +121,7 @@ PublicsStream::getSymbols(bool *HadError) const { auto SymbolS = Pdb.getPDBSymbolStream(); if (SymbolS.takeError()) { codeview::CVSymbolArray::Iterator Iter; - return llvm::make_range(Iter, Iter); + return make_range(Iter, Iter); } SymbolStream &SS = SymbolS.get(); diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/RawError.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/RawError.cpp index eb169f7..f4a5057 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/RawError.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/RawError.cpp @@ -11,7 +11,7 @@ namespace { // deal with the Error value directly, rather than converting to error_code. class RawErrorCategory : public std::error_category { public: - const char *name() const LLVM_NOEXCEPT override { return "llvm.pdb.raw"; } + const char *name() const noexcept override { return "llvm.pdb.raw"; } std::string message(int Condition) const override { switch (static_cast<raw_error_code>(Condition)) { @@ -19,6 +19,8 @@ public: return "An unknown error has occurred."; case raw_error_code::feature_unsupported: return "The feature is unsupported by the implementation."; + case raw_error_code::invalid_format: + return "The record is in an unexpected format."; case raw_error_code::corrupt_file: return "The PDB file is corrupt."; case raw_error_code::insufficient_buffer: @@ -30,6 +32,10 @@ public: return "The specified item does not exist in the array."; case raw_error_code::invalid_block_address: return "The specified block address is not valid."; + case raw_error_code::duplicate_entry: + return "The entry already exists."; + case raw_error_code::no_entry: + return "The entry does not exist."; case raw_error_code::not_writable: return "The PDB does not support writing."; case raw_error_code::invalid_tpi_hash: diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/RawSession.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/RawSession.cpp index 455d331..cd3a206 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/RawSession.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/RawSession.cpp @@ -7,10 +7,8 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/RawSession.h" - -#include "llvm/DebugInfo/CodeView/ByteStream.h" -#include "llvm/DebugInfo/CodeView/StreamInterface.h" +#include "llvm/ADT/STLExtras.h" +#include "llvm/DebugInfo/MSF/ByteStream.h" #include "llvm/DebugInfo/PDB/GenericError.h" #include "llvm/DebugInfo/PDB/IPDBEnumChildren.h" #include "llvm/DebugInfo/PDB/IPDBSourceFile.h" @@ -18,59 +16,51 @@ #include "llvm/DebugInfo/PDB/PDBSymbolExe.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" - +#include "llvm/DebugInfo/PDB/Raw/RawSession.h" +#include "llvm/Support/Allocator.h" +#include "llvm/Support/Error.h" #include "llvm/Support/ErrorOr.h" #include "llvm/Support/MemoryBuffer.h" +#include <algorithm> +#include <memory> using namespace llvm; +using namespace llvm::msf; using namespace llvm::pdb; -namespace { -// We need a class which behaves like an immutable ByteStream, but whose data -// is backed by an llvm::MemoryBuffer. It also needs to own the underlying -// MemoryBuffer, so this simple adapter is a good way to achieve that. -class InputByteStream : public codeview::ByteStream<false> { -public: - explicit InputByteStream(std::unique_ptr<MemoryBuffer> Buffer) - : ByteStream(ArrayRef<uint8_t>(Buffer->getBuffer().bytes_begin(), - Buffer->getBuffer().bytes_end())), - MemBuffer(std::move(Buffer)) {} - - std::unique_ptr<MemoryBuffer> MemBuffer; -}; -} - -RawSession::RawSession(std::unique_ptr<PDBFile> PdbFile) - : Pdb(std::move(PdbFile)) {} +RawSession::RawSession(std::unique_ptr<PDBFile> PdbFile, + std::unique_ptr<BumpPtrAllocator> Allocator) + : Pdb(std::move(PdbFile)), Allocator(std::move(Allocator)) {} -RawSession::~RawSession() {} +RawSession::~RawSession() = default; Error RawSession::createFromPdb(StringRef Path, std::unique_ptr<IPDBSession> &Session) { - ErrorOr<std::unique_ptr<MemoryBuffer>> ErrorOrBuffer = MemoryBuffer::getFileOrSTDIN(Path, /*FileSize=*/-1, /*RequiresNullTerminator=*/false); if (!ErrorOrBuffer) - return llvm::make_error<GenericError>(generic_error_code::invalid_path); + return make_error<GenericError>(generic_error_code::invalid_path); std::unique_ptr<MemoryBuffer> Buffer = std::move(*ErrorOrBuffer); - auto Stream = llvm::make_unique<InputByteStream>(std::move(Buffer)); + auto Stream = llvm::make_unique<MemoryBufferByteStream>(std::move(Buffer)); - std::unique_ptr<PDBFile> File(new PDBFile(std::move(Stream))); + auto Allocator = llvm::make_unique<BumpPtrAllocator>(); + auto File = llvm::make_unique<PDBFile>(std::move(Stream), *Allocator); if (auto EC = File->parseFileHeaders()) return EC; if (auto EC = File->parseStreamData()) return EC; - Session.reset(new RawSession(std::move(File))); + Session = + llvm::make_unique<RawSession>(std::move(File), std::move(Allocator)); return Error::success(); } Error RawSession::createFromExe(StringRef Path, std::unique_ptr<IPDBSession> &Session) { - return llvm::make_error<RawError>(raw_error_code::feature_unsupported); + return make_error<RawError>(raw_error_code::feature_unsupported); } uint64_t RawSession::getLoadAddress() const { return 0; } @@ -103,26 +93,26 @@ RawSession::findLineNumbersByAddress(uint64_t Address, uint32_t Length) const { std::unique_ptr<IPDBEnumSourceFiles> RawSession::findSourceFiles(const PDBSymbolCompiland *Compiland, - llvm::StringRef Pattern, + StringRef Pattern, PDB_NameSearchFlags Flags) const { return nullptr; } std::unique_ptr<IPDBSourceFile> RawSession::findOneSourceFile(const PDBSymbolCompiland *Compiland, - llvm::StringRef Pattern, + StringRef Pattern, PDB_NameSearchFlags Flags) const { return nullptr; } std::unique_ptr<IPDBEnumChildren<PDBSymbolCompiland>> -RawSession::findCompilandsForSourceFile(llvm::StringRef Pattern, +RawSession::findCompilandsForSourceFile(StringRef Pattern, PDB_NameSearchFlags Flags) const { return nullptr; } std::unique_ptr<PDBSymbolCompiland> -RawSession::findOneCompilandForSourceFile(llvm::StringRef Pattern, +RawSession::findOneCompilandForSourceFile(StringRef Pattern, PDB_NameSearchFlags Flags) const { return nullptr; } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/SymbolStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/SymbolStream.cpp index 41b2a64..2f3ac34 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/SymbolStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/SymbolStream.cpp @@ -10,10 +10,9 @@ #include "llvm/DebugInfo/PDB/Raw/SymbolStream.h" #include "llvm/DebugInfo/CodeView/CodeView.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" #include "llvm/DebugInfo/CodeView/TypeRecord.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/RawConstants.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" @@ -21,6 +20,7 @@ #include "llvm/Support/Endian.h" using namespace llvm; +using namespace llvm::msf; using namespace llvm::support; using namespace llvm::pdb; @@ -30,7 +30,7 @@ SymbolStream::SymbolStream(std::unique_ptr<MappedBlockStream> Stream) SymbolStream::~SymbolStream() {} Error SymbolStream::reload() { - codeview::StreamReader Reader(*Stream); + StreamReader Reader(*Stream); if (auto EC = Reader.readArray(SymbolRecords, Stream->getLength())) return EC; diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiHashing.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiHashing.cpp new file mode 100644 index 0000000..6c3ddb3 --- /dev/null +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiHashing.cpp @@ -0,0 +1,110 @@ +//===- TpiHashing.cpp -----------------------------------------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#include "llvm/DebugInfo/PDB/Raw/TpiHashing.h" + +#include "llvm/DebugInfo/PDB/Raw/Hash.h" +#include "llvm/DebugInfo/PDB/Raw/RawError.h" + +using namespace llvm; +using namespace llvm::codeview; +using namespace llvm::pdb; + +// Corresponds to `fUDTAnon`. +template <typename T> static bool isAnonymous(T &Rec) { + StringRef Name = Rec.getName(); + return Name == "<unnamed-tag>" || Name == "__unnamed" || + Name.endswith("::<unnamed-tag>") || Name.endswith("::__unnamed"); +} + +// Computes a hash for a given TPI record. +template <typename T> +static uint32_t getTpiHash(T &Rec, ArrayRef<uint8_t> FullRecord) { + auto Opts = static_cast<uint16_t>(Rec.getOptions()); + + bool ForwardRef = + Opts & static_cast<uint16_t>(ClassOptions::ForwardReference); + bool Scoped = Opts & static_cast<uint16_t>(ClassOptions::Scoped); + bool UniqueName = Opts & static_cast<uint16_t>(ClassOptions::HasUniqueName); + bool IsAnon = UniqueName && isAnonymous(Rec); + + if (!ForwardRef && !Scoped && !IsAnon) + return hashStringV1(Rec.getName()); + if (!ForwardRef && UniqueName && !IsAnon) + return hashStringV1(Rec.getUniqueName()); + return hashBufferV8(FullRecord); +} + +template <typename T> static uint32_t getSourceLineHash(T &Rec) { + char Buf[4]; + support::endian::write32le(Buf, Rec.getUDT().getIndex()); + return hashStringV1(StringRef(Buf, 4)); +} + +void TpiHashUpdater::visitKnownRecordImpl(CVType &CVR, + UdtSourceLineRecord &Rec) { + CVR.Hash = getSourceLineHash(Rec); +} + +void TpiHashUpdater::visitKnownRecordImpl(CVType &CVR, + UdtModSourceLineRecord &Rec) { + CVR.Hash = getSourceLineHash(Rec); +} + +void TpiHashUpdater::visitKnownRecordImpl(CVType &CVR, ClassRecord &Rec) { + CVR.Hash = getTpiHash(Rec, CVR.data()); +} + +void TpiHashUpdater::visitKnownRecordImpl(CVType &CVR, EnumRecord &Rec) { + CVR.Hash = getTpiHash(Rec, CVR.data()); +} + +void TpiHashUpdater::visitKnownRecordImpl(CVType &CVR, UnionRecord &Rec) { + CVR.Hash = getTpiHash(Rec, CVR.data()); +} + +Error TpiHashVerifier::visitKnownRecord(CVType &CVR, UdtSourceLineRecord &Rec) { + return verifySourceLine(Rec.getUDT()); +} + +Error TpiHashVerifier::visitKnownRecord(CVType &CVR, + UdtModSourceLineRecord &Rec) { + return verifySourceLine(Rec.getUDT()); +} + +Error TpiHashVerifier::visitKnownRecord(CVType &CVR, ClassRecord &Rec) { + if (getTpiHash(Rec, CVR.data()) % NumHashBuckets != HashValues[Index]) + return errorInvalidHash(); + return Error::success(); +} +Error TpiHashVerifier::visitKnownRecord(CVType &CVR, EnumRecord &Rec) { + if (getTpiHash(Rec, CVR.data()) % NumHashBuckets != HashValues[Index]) + return errorInvalidHash(); + return Error::success(); +} +Error TpiHashVerifier::visitKnownRecord(CVType &CVR, UnionRecord &Rec) { + if (getTpiHash(Rec, CVR.data()) % NumHashBuckets != HashValues[Index]) + return errorInvalidHash(); + return Error::success(); +} + +Error TpiHashVerifier::verifySourceLine(codeview::TypeIndex TI) { + char Buf[4]; + support::endian::write32le(Buf, TI.getIndex()); + uint32_t Hash = hashStringV1(StringRef(Buf, 4)); + if (Hash % NumHashBuckets != HashValues[Index]) + return errorInvalidHash(); + return Error::success(); +} + +Error TpiHashVerifier::visitTypeBegin(CVType &Rec) { + ++Index; + RawRecord = Rec; + return Error::success(); +} diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStream.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStream.cpp index 5617e57..a1167cd 100644 --- a/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStream.cpp +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStream.cpp @@ -7,155 +7,55 @@ // //===----------------------------------------------------------------------===// -#include "llvm/DebugInfo/PDB/Raw/TpiStream.h" - +#include "llvm/ADT/iterator_range.h" #include "llvm/DebugInfo/CodeView/CVTypeVisitor.h" -#include "llvm/DebugInfo/CodeView/CodeView.h" -#include "llvm/DebugInfo/CodeView/StreamReader.h" -#include "llvm/DebugInfo/CodeView/TypeIndex.h" +#include "llvm/DebugInfo/CodeView/TypeDeserializer.h" #include "llvm/DebugInfo/CodeView/TypeRecord.h" -#include "llvm/DebugInfo/PDB/Raw/Hash.h" -#include "llvm/DebugInfo/PDB/Raw/IndexedStreamData.h" -#include "llvm/DebugInfo/PDB/Raw/MappedBlockStream.h" +#include "llvm/DebugInfo/CodeView/TypeVisitorCallbackPipeline.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" #include "llvm/DebugInfo/PDB/Raw/PDBFile.h" #include "llvm/DebugInfo/PDB/Raw/RawConstants.h" #include "llvm/DebugInfo/PDB/Raw/RawError.h" #include "llvm/DebugInfo/PDB/Raw/RawTypes.h" - +#include "llvm/DebugInfo/PDB/Raw/TpiHashing.h" +#include "llvm/DebugInfo/PDB/Raw/TpiStream.h" #include "llvm/Support/Endian.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> +#include <vector> using namespace llvm; using namespace llvm::codeview; using namespace llvm::support; +using namespace llvm::msf; using namespace llvm::pdb; -namespace { -const uint32_t MinHashBuckets = 0x1000; -const uint32_t MaxHashBuckets = 0x40000; -} - -// This corresponds to `HDR` in PDB/dbi/tpi.h. -struct TpiStream::HeaderInfo { - struct EmbeddedBuf { - little32_t Off; - ulittle32_t Length; - }; - - ulittle32_t Version; - ulittle32_t HeaderSize; - ulittle32_t TypeIndexBegin; - ulittle32_t TypeIndexEnd; - ulittle32_t TypeRecordBytes; - - // The following members correspond to `TpiHash` in PDB/dbi/tpi.h. - ulittle16_t HashStreamIndex; - ulittle16_t HashAuxStreamIndex; - ulittle32_t HashKeySize; - ulittle32_t NumHashBuckets; - - EmbeddedBuf HashValueBuffer; - EmbeddedBuf IndexOffsetBuffer; - EmbeddedBuf HashAdjBuffer; -}; - TpiStream::TpiStream(const PDBFile &File, std::unique_ptr<MappedBlockStream> Stream) : Pdb(File), Stream(std::move(Stream)) {} -TpiStream::~TpiStream() {} - -// Corresponds to `fUDTAnon`. -template <typename T> static bool isAnonymous(T &Rec) { - StringRef Name = Rec.getName(); - return Name == "<unnamed-tag>" || Name == "__unnamed" || - Name.endswith("::<unnamed-tag>") || Name.endswith("::__unnamed"); -} - -// Computes a hash for a given TPI record. -template <typename T> -static uint32_t getTpiHash(T &Rec, const CVRecord<TypeLeafKind> &RawRec) { - auto Opts = static_cast<uint16_t>(Rec.getOptions()); - - bool ForwardRef = - Opts & static_cast<uint16_t>(ClassOptions::ForwardReference); - bool Scoped = Opts & static_cast<uint16_t>(ClassOptions::Scoped); - bool UniqueName = Opts & static_cast<uint16_t>(ClassOptions::HasUniqueName); - bool IsAnon = UniqueName && isAnonymous(Rec); - - if (!ForwardRef && !Scoped && !IsAnon) - return hashStringV1(Rec.getName()); - if (!ForwardRef && UniqueName && !IsAnon) - return hashStringV1(Rec.getUniqueName()); - return hashBufferV8(RawRec.RawData); -} - -namespace { -class TpiHashVerifier : public TypeVisitorCallbacks { -public: - TpiHashVerifier(FixedStreamArray<support::ulittle32_t> &HashValues, - uint32_t NumHashBuckets) - : HashValues(HashValues), NumHashBuckets(NumHashBuckets) {} - - Error visitUdtSourceLine(UdtSourceLineRecord &Rec) override { - return verifySourceLine(Rec); - } - - Error visitUdtModSourceLine(UdtModSourceLineRecord &Rec) override { - return verifySourceLine(Rec); - } - - Error visitClass(ClassRecord &Rec) override { return verify(Rec); } - Error visitEnum(EnumRecord &Rec) override { return verify(Rec); } - Error visitUnion(UnionRecord &Rec) override { return verify(Rec); } - - Error visitTypeBegin(const CVRecord<TypeLeafKind> &Rec) override { - ++Index; - RawRecord = &Rec; - return Error::success(); - } - -private: - template <typename T> Error verify(T &Rec) { - uint32_t Hash = getTpiHash(Rec, *RawRecord); - if (Hash % NumHashBuckets != HashValues[Index]) - return errorInvalidHash(); - return Error::success(); - } - - template <typename T> Error verifySourceLine(T &Rec) { - char Buf[4]; - support::endian::write32le(Buf, Rec.getUDT().getIndex()); - uint32_t Hash = hashStringV1(StringRef(Buf, 4)); - if (Hash % NumHashBuckets != HashValues[Index]) - return errorInvalidHash(); - return Error::success(); - } - - Error errorInvalidHash() { - return make_error<RawError>( - raw_error_code::invalid_tpi_hash, - "Type index is 0x" + utohexstr(TypeIndex::FirstNonSimpleIndex + Index)); - } - - FixedStreamArray<support::ulittle32_t> HashValues; - const CVRecord<TypeLeafKind> *RawRecord; - uint32_t NumHashBuckets; - uint32_t Index = -1; -}; -} +TpiStream::~TpiStream() = default; // Verifies that a given type record matches with a given hash value. // Currently we only verify SRC_LINE records. Error TpiStream::verifyHashValues() { TpiHashVerifier Verifier(HashValues, Header->NumHashBuckets); - CVTypeVisitor Visitor(Verifier); + TypeDeserializer Deserializer; + + TypeVisitorCallbackPipeline Pipeline; + Pipeline.addCallbackToPipeline(Deserializer); + Pipeline.addCallbackToPipeline(Verifier); + + CVTypeVisitor Visitor(Pipeline); return Visitor.visitTypeStream(TypeRecords); } Error TpiStream::reload() { StreamReader Reader(*Stream); - if (Reader.bytesRemaining() < sizeof(HeaderInfo)) + if (Reader.bytesRemaining() < sizeof(TpiStreamHeader)) return make_error<RawError>(raw_error_code::corrupt_file, "TPI Stream does not contain a header."); @@ -167,7 +67,7 @@ Error TpiStream::reload() { return make_error<RawError>(raw_error_code::corrupt_file, "Unsupported TPI Version."); - if (Header->HeaderSize != sizeof(HeaderInfo)) + if (Header->HeaderSize != sizeof(TpiStreamHeader)) return make_error<RawError>(raw_error_code::corrupt_file, "Corrupt TPI Header size."); @@ -175,8 +75,8 @@ Error TpiStream::reload() { return make_error<RawError>(raw_error_code::corrupt_file, "TPI Stream expected 4 byte hash key size."); - if (Header->NumHashBuckets < MinHashBuckets || - Header->NumHashBuckets > MaxHashBuckets) + if (Header->NumHashBuckets < MinTpiHashBuckets || + Header->NumHashBuckets > MaxTpiHashBuckets) return make_error<RawError>(raw_error_code::corrupt_file, "TPI Stream Invalid number of hash buckets."); @@ -185,43 +85,47 @@ Error TpiStream::reload() { return EC; // Hash indices, hash values, etc come from the hash stream. - if (Header->HashStreamIndex >= Pdb.getNumStreams()) - return make_error<RawError>(raw_error_code::corrupt_file, - "Invalid TPI hash stream index."); - - auto HS = - MappedBlockStream::createIndexedStream(Header->HashStreamIndex, Pdb); - if (!HS) - return HS.takeError(); - StreamReader HSR(**HS); - - uint32_t NumHashValues = Header->HashValueBuffer.Length / sizeof(ulittle32_t); - if (NumHashValues != NumTypeRecords()) - return make_error<RawError>( - raw_error_code::corrupt_file, - "TPI hash count does not match with the number of type records."); - HSR.setOffset(Header->HashValueBuffer.Off); - if (auto EC = HSR.readArray(HashValues, NumHashValues)) - return EC; - - HSR.setOffset(Header->IndexOffsetBuffer.Off); - uint32_t NumTypeIndexOffsets = - Header->IndexOffsetBuffer.Length / sizeof(TypeIndexOffset); - if (auto EC = HSR.readArray(TypeIndexOffsets, NumTypeIndexOffsets)) - return EC; - - HSR.setOffset(Header->HashAdjBuffer.Off); - uint32_t NumHashAdjustments = - Header->HashAdjBuffer.Length / sizeof(TypeIndexOffset); - if (auto EC = HSR.readArray(HashAdjustments, NumHashAdjustments)) - return EC; - - HashStream = std::move(*HS); - - // TPI hash table is a parallel array for the type records. - // Verify that the hash values match with type records. - if (auto EC = verifyHashValues()) - return EC; + if (Header->HashStreamIndex != kInvalidStreamIndex) { + if (Header->HashStreamIndex >= Pdb.getNumStreams()) + return make_error<RawError>(raw_error_code::corrupt_file, + "Invalid TPI hash stream index."); + + auto HS = MappedBlockStream::createIndexedStream( + Pdb.getMsfLayout(), Pdb.getMsfBuffer(), Header->HashStreamIndex); + StreamReader HSR(*HS); + + uint32_t NumHashValues = + Header->HashValueBuffer.Length / sizeof(ulittle32_t); + if (NumHashValues != NumTypeRecords()) + return make_error<RawError>( + raw_error_code::corrupt_file, + "TPI hash count does not match with the number of type records."); + HSR.setOffset(Header->HashValueBuffer.Off); + if (auto EC = HSR.readArray(HashValues, NumHashValues)) + return EC; + std::vector<ulittle32_t> HashValueList; + for (auto I : HashValues) + HashValueList.push_back(I); + + HSR.setOffset(Header->IndexOffsetBuffer.Off); + uint32_t NumTypeIndexOffsets = + Header->IndexOffsetBuffer.Length / sizeof(TypeIndexOffset); + if (auto EC = HSR.readArray(TypeIndexOffsets, NumTypeIndexOffsets)) + return EC; + + HSR.setOffset(Header->HashAdjBuffer.Off); + uint32_t NumHashAdjustments = + Header->HashAdjBuffer.Length / sizeof(TypeIndexOffset); + if (auto EC = HSR.readArray(HashAdjustments, NumHashAdjustments)) + return EC; + + HashStream = std::move(HS); + + // TPI hash table is a parallel array for the type records. + // Verify that the hash values match with type records. + if (auto EC = verifyHashValues()) + return EC; + } return Error::success(); } @@ -267,7 +171,7 @@ TpiStream::getHashAdjustments() const { iterator_range<CVTypeArray::Iterator> TpiStream::types(bool *HadError) const { - return llvm::make_range(TypeRecords.begin(HadError), TypeRecords.end()); + return make_range(TypeRecords.begin(HadError), TypeRecords.end()); } Error TpiStream::commit() { return Error::success(); } diff --git a/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStreamBuilder.cpp b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStreamBuilder.cpp new file mode 100644 index 0000000..c769321 --- /dev/null +++ b/contrib/llvm/lib/DebugInfo/PDB/Raw/TpiStreamBuilder.cpp @@ -0,0 +1,145 @@ +//===- TpiStreamBuilder.cpp - -------------------------------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#include "llvm/ADT/ArrayRef.h" +#include "llvm/ADT/STLExtras.h" +#include "llvm/DebugInfo/CodeView/TypeIndex.h" +#include "llvm/DebugInfo/CodeView/TypeRecord.h" +#include "llvm/DebugInfo/MSF/ByteStream.h" +#include "llvm/DebugInfo/MSF/MSFBuilder.h" +#include "llvm/DebugInfo/MSF/MappedBlockStream.h" +#include "llvm/DebugInfo/MSF/StreamArray.h" +#include "llvm/DebugInfo/MSF/StreamReader.h" +#include "llvm/DebugInfo/MSF/StreamWriter.h" +#include "llvm/DebugInfo/PDB/Raw/PDBFile.h" +#include "llvm/DebugInfo/PDB/Raw/RawError.h" +#include "llvm/DebugInfo/PDB/Raw/RawTypes.h" +#include "llvm/DebugInfo/PDB/Raw/TpiStream.h" +#include "llvm/DebugInfo/PDB/Raw/TpiStreamBuilder.h" +#include "llvm/Support/Allocator.h" +#include "llvm/Support/Endian.h" +#include "llvm/Support/Error.h" +#include <algorithm> +#include <cstdint> + +using namespace llvm; +using namespace llvm::msf; +using namespace llvm::pdb; +using namespace llvm::support; + +TpiStreamBuilder::TpiStreamBuilder(MSFBuilder &Msf, uint32_t StreamIdx) + : Msf(Msf), Allocator(Msf.getAllocator()), Header(nullptr), Idx(StreamIdx) { +} + +TpiStreamBuilder::~TpiStreamBuilder() = default; + +void TpiStreamBuilder::setVersionHeader(PdbRaw_TpiVer Version) { + VerHeader = Version; +} + +void TpiStreamBuilder::addTypeRecord(const codeview::CVType &Record) { + TypeRecords.push_back(Record); + TypeRecordStream.setItems(TypeRecords); +} + +Error TpiStreamBuilder::finalize() { + if (Header) + return Error::success(); + + TpiStreamHeader *H = Allocator.Allocate<TpiStreamHeader>(); + + uint32_t Count = TypeRecords.size(); + uint32_t HashBufferSize = calculateHashBufferSize(); + + H->Version = *VerHeader; + H->HeaderSize = sizeof(TpiStreamHeader); + H->TypeIndexBegin = codeview::TypeIndex::FirstNonSimpleIndex; + H->TypeIndexEnd = H->TypeIndexBegin + Count; + H->TypeRecordBytes = TypeRecordStream.getLength(); + + H->HashStreamIndex = HashStreamIndex; + H->HashAuxStreamIndex = kInvalidStreamIndex; + H->HashKeySize = sizeof(ulittle32_t); + H->NumHashBuckets = MinTpiHashBuckets; + + // Recall that hash values go into a completely different stream identified by + // the `HashStreamIndex` field of the `TpiStreamHeader`. Therefore, the data + // begins at offset 0 of this independent stream. + H->HashValueBuffer.Off = 0; + H->HashValueBuffer.Length = HashBufferSize; + H->HashAdjBuffer.Off = H->HashValueBuffer.Off + H->HashValueBuffer.Length; + H->HashAdjBuffer.Length = 0; + H->IndexOffsetBuffer.Off = H->HashAdjBuffer.Off + H->HashAdjBuffer.Length; + H->IndexOffsetBuffer.Length = 0; + + Header = H; + return Error::success(); +} + +uint32_t TpiStreamBuilder::calculateSerializedLength() const { + return sizeof(TpiStreamHeader) + TypeRecordStream.getLength(); +} + +uint32_t TpiStreamBuilder::calculateHashBufferSize() const { + if (TypeRecords.empty() || !TypeRecords[0].Hash.hasValue()) + return 0; + return TypeRecords.size() * sizeof(ulittle32_t); +} + +Error TpiStreamBuilder::finalizeMsfLayout() { + uint32_t Length = calculateSerializedLength(); + if (auto EC = Msf.setStreamSize(Idx, Length)) + return EC; + + uint32_t HashBufferSize = calculateHashBufferSize(); + + if (HashBufferSize == 0) + return Error::success(); + + auto ExpectedIndex = Msf.addStream(HashBufferSize); + if (!ExpectedIndex) + return ExpectedIndex.takeError(); + HashStreamIndex = *ExpectedIndex; + ulittle32_t *H = Allocator.Allocate<ulittle32_t>(TypeRecords.size()); + MutableArrayRef<ulittle32_t> HashBuffer(H, TypeRecords.size()); + for (uint32_t I = 0; I < TypeRecords.size(); ++I) { + HashBuffer[I] = *TypeRecords[I].Hash % MinTpiHashBuckets; + } + ArrayRef<uint8_t> Bytes(reinterpret_cast<const uint8_t *>(HashBuffer.data()), + HashBufferSize); + HashValueStream = llvm::make_unique<ByteStream>(Bytes); + return Error::success(); +} + +Error TpiStreamBuilder::commit(const msf::MSFLayout &Layout, + const msf::WritableStream &Buffer) { + if (auto EC = finalize()) + return EC; + + auto InfoS = + WritableMappedBlockStream::createIndexedStream(Layout, Buffer, Idx); + + StreamWriter Writer(*InfoS); + if (auto EC = Writer.writeObject(*Header)) + return EC; + + auto RecordArray = VarStreamArray<codeview::CVType>(TypeRecordStream); + if (auto EC = Writer.writeArray(RecordArray)) + return EC; + + if (HashStreamIndex != kInvalidStreamIndex) { + auto HVS = WritableMappedBlockStream::createIndexedStream(Layout, Buffer, + HashStreamIndex); + StreamWriter HW(*HVS); + if (auto EC = HW.writeStreamRef(*HashValueStream)) + return EC; + } + + return Error::success(); +} |
