diff options
| author | dim <dim@FreeBSD.org> | 2014-03-21 17:53:59 +0000 |
|---|---|---|
| committer | dim <dim@FreeBSD.org> | 2014-03-21 17:53:59 +0000 |
| commit | 9cedb8bb69b89b0f0c529937247a6a80cabdbaec (patch) | |
| tree | c978f0e9ec1ab92dc8123783f30b08a7fd1e2a39 /contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp | |
| parent | 03fdc2934eb61c44c049a02b02aa974cfdd8a0eb (diff) | |
| download | FreeBSD-src-9cedb8bb69b89b0f0c529937247a6a80cabdbaec.zip FreeBSD-src-9cedb8bb69b89b0f0c529937247a6a80cabdbaec.tar.gz | |
MFC 261991:
Upgrade our copy of llvm/clang to 3.4 release. This version supports
all of the features in the current working draft of the upcoming C++
standard, provisionally named C++1y.
The code generator's performance is greatly increased, and the loop
auto-vectorizer is now enabled at -Os and -O2 in addition to -O3. The
PowerPC backend has made several major improvements to code generation
quality and compile time, and the X86, SPARC, ARM32, Aarch64 and SystemZ
backends have all seen major feature work.
Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.4/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html>
MFC 262121 (by emaste):
Update lldb for clang/llvm 3.4 import
This commit largely restores the lldb source to the upstream r196259
snapshot with the addition of threaded inferior support and a few bug
fixes.
Specific upstream lldb revisions restored include:
SVN git
181387 779e6ac
181703 7bef4e2
182099 b31044e
182650 f2dcf35
182683 0d91b80
183862 15c1774
183929 99447a6
184177 0b2934b
184948 4dc3761
184954 007e7bc
186990 eebd175
Sponsored by: DARPA, AFRL
MFC 262186 (by emaste):
Fix mismerge in r262121
A break statement was lost in the merge. The error had no functional
impact, but restore it to reduce the diff against upstream.
MFC 262303:
Pull in r197521 from upstream clang trunk (by rdivacky):
Use the integrated assembler by default on FreeBSD/ppc and ppc64.
Requested by: jhibbits
MFC 262611:
Pull in r196874 from upstream llvm trunk:
Fix a crash that occurs when PWD is invalid.
MCJIT needs to be able to run in hostile environments, even when PWD
is invalid. There's no need to crash MCJIT in this case.
The obvious fix is to simply leave MCContext's CompilationDir empty
when PWD can't be determined. This way, MCJIT clients,
and other clients that link with LLVM don't need a valid working directory.
If we do want to guarantee valid CompilationDir, that should be done
only for clients of getCompilationDir(). This is as simple as checking
for an empty string.
The only current use of getCompilationDir is EmitGenDwarfInfo, which
won't conceivably run with an invalid working dir. However, in the
purely hypothetically and untestable case that this happens, the
AT_comp_dir will be omitted from the compilation_unit DIE.
This should help fix assertions occurring with ports-mgmt/tinderbox,
when it is using jails, and sometimes invalidates clang's current
working directory.
Reported by: decke
MFC 262809:
Pull in r203007 from upstream clang trunk:
Don't produce an alias between destructors with different calling conventions.
Fixes pr19007.
(Please note that is an LLVM PR identifier, not a FreeBSD one.)
This should fix Firefox and/or libxul crashes (due to problems with
regparm/stdcall calling conventions) on i386.
Reported by: multiple users on freebsd-current
PR: bin/187103
MFC 263048:
Repair recognition of "CC" as an alias for the C++ compiler, since it
was silently broken by upstream for a Windows-specific use-case.
Apparently some versions of CMake still rely on this archaic feature...
Reported by: rakuco
MFC 263049:
Garbage collect the old way of adding the libstdc++ include directories
in clang's InitHeaderSearch.cpp. This has been superseded by David
Chisnall's commit in r255321.
Moreover, if libc++ is used, the libstdc++ include directories should
not be in the search path at all. These directories are now only used
if you pass -stdlib=libstdc++.
Diffstat (limited to 'contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp')
| -rw-r--r-- | contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp | 862 |
1 files changed, 651 insertions, 211 deletions
diff --git a/contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp b/contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp index 722af5d..e0b090f 100644 --- a/contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp +++ b/contrib/llvm/tools/clang/lib/Format/UnwrappedLineParser.cpp @@ -16,12 +16,22 @@ #define DEBUG_TYPE "format-parser" #include "UnwrappedLineParser.h" -#include "clang/Basic/Diagnostic.h" #include "llvm/Support/Debug.h" namespace clang { namespace format { +class FormatTokenSource { +public: + virtual ~FormatTokenSource() {} + virtual FormatToken *getNextToken() = 0; + + virtual unsigned getPosition() = 0; + virtual FormatToken *setPosition(unsigned Position) = 0; +}; + +namespace { + class ScopedDeclarationState { public: ScopedDeclarationState(UnwrappedLine &Line, std::vector<bool> &Stack, @@ -37,6 +47,7 @@ public: else Line.MustBeDeclaration = true; } + private: UnwrappedLine &Line; std::vector<bool> &Stack; @@ -45,11 +56,11 @@ private: class ScopedMacroState : public FormatTokenSource { public: ScopedMacroState(UnwrappedLine &Line, FormatTokenSource *&TokenSource, - FormatToken &ResetToken, bool &StructuralError) + FormatToken *&ResetToken, bool &StructuralError) : Line(Line), TokenSource(TokenSource), ResetToken(ResetToken), PreviousLineLevel(Line.Level), PreviousTokenSource(TokenSource), StructuralError(StructuralError), - PreviousStructuralError(StructuralError) { + PreviousStructuralError(StructuralError), Token(NULL) { TokenSource = this; Line.Level = 0; Line.InPPDirective = true; @@ -63,44 +74,60 @@ public: StructuralError = PreviousStructuralError; } - virtual FormatToken getNextToken() { + virtual FormatToken *getNextToken() { // The \c UnwrappedLineParser guards against this by never calling // \c getNextToken() after it has encountered the first eof token. assert(!eof()); Token = PreviousTokenSource->getNextToken(); if (eof()) - return createEOF(); + return getFakeEOF(); return Token; } -private: - bool eof() { return Token.HasUnescapedNewline; } + virtual unsigned getPosition() { return PreviousTokenSource->getPosition(); } - FormatToken createEOF() { - FormatToken FormatTok; - FormatTok.Tok.startToken(); - FormatTok.Tok.setKind(tok::eof); - return FormatTok; + virtual FormatToken *setPosition(unsigned Position) { + Token = PreviousTokenSource->setPosition(Position); + return Token; + } + +private: + bool eof() { return Token && Token->HasUnescapedNewline; } + + FormatToken *getFakeEOF() { + static bool EOFInitialized = false; + static FormatToken FormatTok; + if (!EOFInitialized) { + FormatTok.Tok.startToken(); + FormatTok.Tok.setKind(tok::eof); + EOFInitialized = true; + } + return &FormatTok; } UnwrappedLine &Line; FormatTokenSource *&TokenSource; - FormatToken &ResetToken; + FormatToken *&ResetToken; unsigned PreviousLineLevel; FormatTokenSource *PreviousTokenSource; bool &StructuralError; bool PreviousStructuralError; - FormatToken Token; + FormatToken *Token; }; +} // end anonymous namespace + class ScopedLineState { public: ScopedLineState(UnwrappedLineParser &Parser, bool SwitchToPreprocessorLines = false) - : Parser(Parser), SwitchToPreprocessorLines(SwitchToPreprocessorLines) { + : Parser(Parser) { + OriginalLines = Parser.CurrentLines; if (SwitchToPreprocessorLines) Parser.CurrentLines = &Parser.PreprocessorDirectives; + else if (!Parser.Line->Tokens.empty()) + Parser.CurrentLines = &Parser.Line->Tokens.back().Children; PreBlockLine = Parser.Line.take(); Parser.Line.reset(new UnwrappedLine()); Parser.Line->Level = PreBlockLine->Level; @@ -113,37 +140,102 @@ public: } assert(Parser.Line->Tokens.empty()); Parser.Line.reset(PreBlockLine); - Parser.MustBreakBeforeNextToken = true; - if (SwitchToPreprocessorLines) - Parser.CurrentLines = &Parser.Lines; + if (Parser.CurrentLines == &Parser.PreprocessorDirectives) + Parser.MustBreakBeforeNextToken = true; + Parser.CurrentLines = OriginalLines; } private: UnwrappedLineParser &Parser; - const bool SwitchToPreprocessorLines; UnwrappedLine *PreBlockLine; + SmallVectorImpl<UnwrappedLine> *OriginalLines; }; -UnwrappedLineParser::UnwrappedLineParser( - clang::DiagnosticsEngine &Diag, const FormatStyle &Style, - FormatTokenSource &Tokens, UnwrappedLineConsumer &Callback) +namespace { + +class IndexedTokenSource : public FormatTokenSource { +public: + IndexedTokenSource(ArrayRef<FormatToken *> Tokens) + : Tokens(Tokens), Position(-1) {} + + virtual FormatToken *getNextToken() { + ++Position; + return Tokens[Position]; + } + + virtual unsigned getPosition() { + assert(Position >= 0); + return Position; + } + + virtual FormatToken *setPosition(unsigned P) { + Position = P; + return Tokens[Position]; + } + + void reset() { Position = -1; } + +private: + ArrayRef<FormatToken *> Tokens; + int Position; +}; + +} // end anonymous namespace + +UnwrappedLineParser::UnwrappedLineParser(const FormatStyle &Style, + ArrayRef<FormatToken *> Tokens, + UnwrappedLineConsumer &Callback) : Line(new UnwrappedLine), MustBreakBeforeNextToken(false), - CurrentLines(&Lines), StructuralError(false), Diag(Diag), Style(Style), - Tokens(&Tokens), Callback(Callback) {} + CurrentLines(&Lines), StructuralError(false), Style(Style), Tokens(NULL), + Callback(Callback), AllTokens(Tokens), PPBranchLevel(-1) {} + +void UnwrappedLineParser::reset() { + PPBranchLevel = -1; + Line.reset(new UnwrappedLine); + CommentsBeforeNextToken.clear(); + FormatTok = NULL; + MustBreakBeforeNextToken = false; + PreprocessorDirectives.clear(); + CurrentLines = &Lines; + DeclarationScopeStack.clear(); + StructuralError = false; + PPStack.clear(); +} bool UnwrappedLineParser::parse() { - DEBUG(llvm::dbgs() << "----\n"); - readToken(); - parseFile(); - for (std::vector<UnwrappedLine>::iterator I = Lines.begin(), E = Lines.end(); - I != E; ++I) { - Callback.consumeUnwrappedLine(*I); - } + IndexedTokenSource TokenSource(AllTokens); + do { + DEBUG(llvm::dbgs() << "----\n"); + reset(); + Tokens = &TokenSource; + TokenSource.reset(); + + readToken(); + parseFile(); + // Create line with eof token. + pushToken(FormatTok); + addUnwrappedLine(); + + for (SmallVectorImpl<UnwrappedLine>::iterator I = Lines.begin(), + E = Lines.end(); + I != E; ++I) { + Callback.consumeUnwrappedLine(*I); + } + Callback.finishRun(); + Lines.clear(); + while (!PPLevelBranchIndex.empty() && + PPLevelBranchIndex.back() + 1 >= PPLevelBranchCount.back()) { + PPLevelBranchIndex.resize(PPLevelBranchIndex.size() - 1); + PPLevelBranchCount.resize(PPLevelBranchCount.size() - 1); + } + if (!PPLevelBranchIndex.empty()) { + ++PPLevelBranchIndex.back(); + assert(PPLevelBranchIndex.size() == PPLevelBranchCount.size()); + assert(PPLevelBranchIndex.back() <= PPLevelBranchCount.back()); + } + } while (!PPLevelBranchIndex.empty()); - // Create line with eof token. - pushToken(FormatTok); - Callback.consumeUnwrappedLine(*Line); return StructuralError; } @@ -151,15 +243,16 @@ void UnwrappedLineParser::parseFile() { ScopedDeclarationState DeclarationState( *Line, DeclarationScopeStack, /*MustBeDeclaration=*/ !Line->InPPDirective); - parseLevel(/*HasOpeningBrace=*/ false); + parseLevel(/*HasOpeningBrace=*/false); // Make sure to format the remaining tokens. flushComments(true); addUnwrappedLine(); } void UnwrappedLineParser::parseLevel(bool HasOpeningBrace) { + bool SwitchLabelEncountered = false; do { - switch (FormatTok.Tok.getKind()) { + switch (FormatTok->Tok.getKind()) { case tok::comment: nextToken(); addUnwrappedLine(); @@ -167,19 +260,24 @@ void UnwrappedLineParser::parseLevel(bool HasOpeningBrace) { case tok::l_brace: // FIXME: Add parameter whether this can happen - if this happens, we must // be in a non-declaration context. - parseBlock(/*MustBeDeclaration=*/ false); + parseBlock(/*MustBeDeclaration=*/false); addUnwrappedLine(); break; case tok::r_brace: if (HasOpeningBrace) return; - Diag.Report(FormatTok.Tok.getLocation(), - Diag.getCustomDiagID(clang::DiagnosticsEngine::Error, - "unexpected '}'")); StructuralError = true; nextToken(); addUnwrappedLine(); break; + case tok::kw_default: + case tok::kw_case: + if (!SwitchLabelEncountered && + (Style.IndentCaseLabels || (Line->InPPDirective && Line->Level == 1))) + ++Line->Level; + SwitchLabelEncountered = true; + parseStructuralElement(); + break; default: parseStructuralElement(); break; @@ -187,41 +285,150 @@ void UnwrappedLineParser::parseLevel(bool HasOpeningBrace) { } while (!eof()); } -void UnwrappedLineParser::parseBlock(bool MustBeDeclaration, - unsigned AddLevels) { - assert(FormatTok.Tok.is(tok::l_brace) && "'{' expected"); +void UnwrappedLineParser::calculateBraceTypes() { + // We'll parse forward through the tokens until we hit + // a closing brace or eof - note that getNextToken() will + // parse macros, so this will magically work inside macro + // definitions, too. + unsigned StoredPosition = Tokens->getPosition(); + unsigned Position = StoredPosition; + FormatToken *Tok = FormatTok; + // Keep a stack of positions of lbrace tokens. We will + // update information about whether an lbrace starts a + // braced init list or a different block during the loop. + SmallVector<FormatToken *, 8> LBraceStack; + assert(Tok->Tok.is(tok::l_brace)); + do { + // Get next none-comment token. + FormatToken *NextTok; + unsigned ReadTokens = 0; + do { + NextTok = Tokens->getNextToken(); + ++ReadTokens; + } while (NextTok->is(tok::comment)); + + switch (Tok->Tok.getKind()) { + case tok::l_brace: + LBraceStack.push_back(Tok); + break; + case tok::r_brace: + if (!LBraceStack.empty()) { + if (LBraceStack.back()->BlockKind == BK_Unknown) { + // If there is a comma, semicolon or right paren after the closing + // brace, we assume this is a braced initializer list. Note that + // regardless how we mark inner braces here, we will overwrite the + // BlockKind later if we parse a braced list (where all blocks inside + // are by default braced lists), or when we explicitly detect blocks + // (for example while parsing lambdas). + // + // We exclude + and - as they can be ObjC visibility modifiers. + if (NextTok->isOneOf(tok::comma, tok::semi, tok::r_paren, + tok::r_square, tok::l_brace, tok::colon) || + (NextTok->isBinaryOperator() && + !NextTok->isOneOf(tok::plus, tok::minus))) { + Tok->BlockKind = BK_BracedInit; + LBraceStack.back()->BlockKind = BK_BracedInit; + } else { + Tok->BlockKind = BK_Block; + LBraceStack.back()->BlockKind = BK_Block; + } + } + LBraceStack.pop_back(); + } + break; + case tok::semi: + case tok::kw_if: + case tok::kw_while: + case tok::kw_for: + case tok::kw_switch: + case tok::kw_try: + if (!LBraceStack.empty()) + LBraceStack.back()->BlockKind = BK_Block; + break; + default: + break; + } + Tok = NextTok; + Position += ReadTokens; + } while (Tok->Tok.isNot(tok::eof) && !LBraceStack.empty()); + // Assume other blocks for all unclosed opening braces. + for (unsigned i = 0, e = LBraceStack.size(); i != e; ++i) { + if (LBraceStack[i]->BlockKind == BK_Unknown) + LBraceStack[i]->BlockKind = BK_Block; + } + + FormatTok = Tokens->setPosition(StoredPosition); +} + +void UnwrappedLineParser::parseBlock(bool MustBeDeclaration, bool AddLevel, + bool MunchSemi) { + assert(FormatTok->Tok.is(tok::l_brace) && "'{' expected"); + unsigned InitialLevel = Line->Level; nextToken(); addUnwrappedLine(); ScopedDeclarationState DeclarationState(*Line, DeclarationScopeStack, MustBeDeclaration); - Line->Level += AddLevels; - parseLevel(/*HasOpeningBrace=*/ true); + if (AddLevel) + ++Line->Level; + parseLevel(/*HasOpeningBrace=*/true); - if (!FormatTok.Tok.is(tok::r_brace)) { - Line->Level -= AddLevels; + if (!FormatTok->Tok.is(tok::r_brace)) { + Line->Level = InitialLevel; StructuralError = true; return; } nextToken(); // Munch the closing brace. - Line->Level -= AddLevels; + if (MunchSemi && FormatTok->Tok.is(tok::semi)) + nextToken(); + Line->Level = InitialLevel; +} + +void UnwrappedLineParser::parseChildBlock() { + FormatTok->BlockKind = BK_Block; + nextToken(); + { + ScopedLineState LineState(*this); + ScopedDeclarationState DeclarationState(*Line, DeclarationScopeStack, + /*MustBeDeclaration=*/false); + Line->Level += 1; + parseLevel(/*HasOpeningBrace=*/true); + Line->Level -= 1; + } + nextToken(); } void UnwrappedLineParser::parsePPDirective() { - assert(FormatTok.Tok.is(tok::hash) && "'#' expected"); + assert(FormatTok->Tok.is(tok::hash) && "'#' expected"); ScopedMacroState MacroState(*Line, Tokens, FormatTok, StructuralError); nextToken(); - if (FormatTok.Tok.getIdentifierInfo() == NULL) { + if (FormatTok->Tok.getIdentifierInfo() == NULL) { parsePPUnknown(); return; } - switch (FormatTok.Tok.getIdentifierInfo()->getPPKeywordID()) { + switch (FormatTok->Tok.getIdentifierInfo()->getPPKeywordID()) { case tok::pp_define: parsePPDefine(); + return; + case tok::pp_if: + parsePPIf(/*IfDef=*/false); + break; + case tok::pp_ifdef: + case tok::pp_ifndef: + parsePPIf(/*IfDef=*/true); + break; + case tok::pp_else: + parsePPElse(); + break; + case tok::pp_elif: + parsePPElIf(); + break; + case tok::pp_endif: + parsePPEndIf(); break; default: parsePPUnknown(); @@ -229,16 +436,77 @@ void UnwrappedLineParser::parsePPDirective() { } } +void UnwrappedLineParser::pushPPConditional() { + if (!PPStack.empty() && PPStack.back() == PP_Unreachable) + PPStack.push_back(PP_Unreachable); + else + PPStack.push_back(PP_Conditional); +} + +void UnwrappedLineParser::parsePPIf(bool IfDef) { + ++PPBranchLevel; + assert(PPBranchLevel >= 0 && PPBranchLevel <= (int)PPLevelBranchIndex.size()); + if (PPBranchLevel == (int)PPLevelBranchIndex.size()) { + PPLevelBranchIndex.push_back(0); + PPLevelBranchCount.push_back(0); + } + PPChainBranchIndex.push(0); + nextToken(); + bool IsLiteralFalse = (FormatTok->Tok.isLiteral() && + StringRef(FormatTok->Tok.getLiteralData(), + FormatTok->Tok.getLength()) == "0") || + FormatTok->Tok.is(tok::kw_false); + if ((!IfDef && IsLiteralFalse) || PPLevelBranchIndex[PPBranchLevel] > 0) { + PPStack.push_back(PP_Unreachable); + } else { + pushPPConditional(); + } + parsePPUnknown(); +} + +void UnwrappedLineParser::parsePPElse() { + if (!PPStack.empty()) + PPStack.pop_back(); + assert(PPBranchLevel < (int)PPLevelBranchIndex.size()); + if (!PPChainBranchIndex.empty()) + ++PPChainBranchIndex.top(); + if (PPBranchLevel >= 0 && !PPChainBranchIndex.empty() && + PPLevelBranchIndex[PPBranchLevel] != PPChainBranchIndex.top()) { + PPStack.push_back(PP_Unreachable); + } else { + pushPPConditional(); + } + parsePPUnknown(); +} + +void UnwrappedLineParser::parsePPElIf() { parsePPElse(); } + +void UnwrappedLineParser::parsePPEndIf() { + assert(PPBranchLevel < (int)PPLevelBranchIndex.size()); + if (PPBranchLevel >= 0 && !PPChainBranchIndex.empty()) { + if (PPChainBranchIndex.top() + 1 > PPLevelBranchCount[PPBranchLevel]) { + PPLevelBranchCount[PPBranchLevel] = PPChainBranchIndex.top() + 1; + } + } + --PPBranchLevel; + if (!PPChainBranchIndex.empty()) + PPChainBranchIndex.pop(); + if (!PPStack.empty()) + PPStack.pop_back(); + parsePPUnknown(); +} + void UnwrappedLineParser::parsePPDefine() { nextToken(); - if (FormatTok.Tok.getKind() != tok::identifier) { + if (FormatTok->Tok.getKind() != tok::identifier) { parsePPUnknown(); return; } nextToken(); - if (FormatTok.Tok.getKind() == tok::l_paren && - FormatTok.WhiteSpaceLength == 0) { + if (FormatTok->Tok.getKind() == tok::l_paren && + FormatTok->WhitespaceRange.getBegin() == + FormatTok->WhitespaceRange.getEnd()) { parseParens(); } addUnwrappedLine(); @@ -287,15 +555,15 @@ bool tokenCanStartNewLine(clang::Token Tok) { } void UnwrappedLineParser::parseStructuralElement() { - assert(!FormatTok.Tok.is(tok::l_brace)); - switch (FormatTok.Tok.getKind()) { + assert(!FormatTok->Tok.is(tok::l_brace)); + switch (FormatTok->Tok.getKind()) { case tok::at: nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { + if (FormatTok->Tok.is(tok::l_brace)) { parseBracedList(); break; } - switch (FormatTok.Tok.getObjCKeywordID()) { + switch (FormatTok->Tok.getObjCKeywordID()) { case tok::objc_public: case tok::objc_protected: case tok::objc_package: @@ -322,7 +590,7 @@ void UnwrappedLineParser::parseStructuralElement() { return; case tok::kw_inline: nextToken(); - if (FormatTok.Tok.is(tok::kw_namespace)) { + if (FormatTok->Tok.is(tok::kw_namespace)) { parseNamespace(); return; } @@ -357,10 +625,10 @@ void UnwrappedLineParser::parseStructuralElement() { return; case tok::kw_extern: nextToken(); - if (FormatTok.Tok.is(tok::string_literal)) { + if (FormatTok->Tok.is(tok::string_literal)) { nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ true, 0); + if (FormatTok->Tok.is(tok::l_brace)) { + parseBlock(/*MustBeDeclaration=*/true, /*AddLevel=*/false); addUnwrappedLine(); return; } @@ -371,10 +639,10 @@ void UnwrappedLineParser::parseStructuralElement() { break; } do { - switch (FormatTok.Tok.getKind()) { + switch (FormatTok->Tok.getKind()) { case tok::at: nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) + if (FormatTok->Tok.is(tok::l_brace)) parseBracedList(); break; case tok::kw_enum: @@ -397,38 +665,63 @@ void UnwrappedLineParser::parseStructuralElement() { case tok::l_paren: parseParens(); break; + case tok::caret: + nextToken(); + if (FormatTok->is(tok::l_brace)) { + parseChildBlock(); + } + break; case tok::l_brace: - // A block outside of parentheses must be the last part of a - // structural element. - // FIXME: Figure out cases where this is not true, and add projections for - // them (the one we know is missing are lambdas). - parseBlock(/*MustBeDeclaration=*/ false); - addUnwrappedLine(); - return; - case tok::identifier: + if (!tryToParseBracedList()) { + // A block outside of parentheses must be the last part of a + // structural element. + // FIXME: Figure out cases where this is not true, and add projections + // for them (the one we know is missing are lambdas). + if (Style.BreakBeforeBraces == FormatStyle::BS_Linux || + Style.BreakBeforeBraces == FormatStyle::BS_Stroustrup || + Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); + addUnwrappedLine(); + return; + } + // Otherwise this was a braced init list, and the structural + // element continues. + break; + case tok::identifier: { + StringRef Text = FormatTok->TokenText; nextToken(); if (Line->Tokens.size() == 1) { - if (FormatTok.Tok.is(tok::colon)) { + if (FormatTok->Tok.is(tok::colon)) { parseLabel(); return; } // Recognize function-like macro usages without trailing semicolon. - if (FormatTok.Tok.is(tok::l_paren)) { + if (FormatTok->Tok.is(tok::l_paren)) { parseParens(); - if (FormatTok.HasUnescapedNewline && - tokenCanStartNewLine(FormatTok.Tok)) { + if (FormatTok->HasUnescapedNewline && + tokenCanStartNewLine(FormatTok->Tok)) { addUnwrappedLine(); return; } + } else if (FormatTok->HasUnescapedNewline && Text.size() >= 5 && + Text == Text.upper()) { + // Recognize free-standing macros like Q_OBJECT. + addUnwrappedLine(); + return; } } break; + } case tok::equal: nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { + if (FormatTok->Tok.is(tok::l_brace)) { parseBracedList(); } break; + case tok::l_square: + tryToParseLambda(); + break; default: nextToken(); break; @@ -436,52 +729,146 @@ void UnwrappedLineParser::parseStructuralElement() { } while (!eof()); } -void UnwrappedLineParser::parseBracedList() { +void UnwrappedLineParser::tryToParseLambda() { + // FIXME: This is a dirty way to access the previous token. Find a better + // solution. + if (!Line->Tokens.empty() && + Line->Tokens.back().Tok->isOneOf(tok::identifier, tok::kw_operator)) { + nextToken(); + return; + } + assert(FormatTok->is(tok::l_square)); + FormatToken &LSquare = *FormatTok; + if (!tryToParseLambdaIntroducer()) + return; + + while (FormatTok->isNot(tok::l_brace)) { + switch (FormatTok->Tok.getKind()) { + case tok::l_brace: + break; + case tok::l_paren: + parseParens(); + break; + case tok::identifier: + case tok::kw_mutable: + nextToken(); + break; + default: + return; + } + } + LSquare.Type = TT_LambdaLSquare; + parseChildBlock(); +} + +bool UnwrappedLineParser::tryToParseLambdaIntroducer() { + nextToken(); + if (FormatTok->is(tok::equal)) { + nextToken(); + if (FormatTok->is(tok::r_square)) { + nextToken(); + return true; + } + if (FormatTok->isNot(tok::comma)) + return false; + nextToken(); + } else if (FormatTok->is(tok::amp)) { + nextToken(); + if (FormatTok->is(tok::r_square)) { + nextToken(); + return true; + } + if (!FormatTok->isOneOf(tok::comma, tok::identifier)) { + return false; + } + if (FormatTok->is(tok::comma)) + nextToken(); + } else if (FormatTok->is(tok::r_square)) { + nextToken(); + return true; + } + do { + if (FormatTok->is(tok::amp)) + nextToken(); + if (!FormatTok->isOneOf(tok::identifier, tok::kw_this)) + return false; + nextToken(); + if (FormatTok->is(tok::comma)) { + nextToken(); + } else if (FormatTok->is(tok::r_square)) { + nextToken(); + return true; + } else { + return false; + } + } while (!eof()); + return false; +} + +bool UnwrappedLineParser::tryToParseBracedList() { + if (FormatTok->BlockKind == BK_Unknown) + calculateBraceTypes(); + assert(FormatTok->BlockKind != BK_Unknown); + if (FormatTok->BlockKind == BK_Block) + return false; + parseBracedList(); + return true; +} + +bool UnwrappedLineParser::parseBracedList(bool ContinueOnSemicolons) { + bool HasError = false; nextToken(); // FIXME: Once we have an expression parser in the UnwrappedLineParser, // replace this by using parseAssigmentExpression() inside. - bool StartOfExpression = true; do { // FIXME: When we start to support lambdas, we'll want to parse them away // here, otherwise our bail-out scenarios below break. The better solution // might be to just implement a more or less complete expression parser. - switch (FormatTok.Tok.getKind()) { - case tok::l_brace: - if (!StartOfExpression) { - // Probably a missing closing brace. Bail out. - addUnwrappedLine(); - return; + switch (FormatTok->Tok.getKind()) { + case tok::caret: + nextToken(); + if (FormatTok->is(tok::l_brace)) { + parseChildBlock(); } + break; + case tok::l_square: + tryToParseLambda(); + break; + case tok::l_brace: + // Assume there are no blocks inside a braced init list apart + // from the ones we explicitly parse out (like lambdas). + FormatTok->BlockKind = BK_BracedInit; parseBracedList(); - StartOfExpression = false; break; case tok::r_brace: nextToken(); - return; + return !HasError; case tok::semi: - // Probably a missing closing brace. Bail out. - return; + HasError = true; + if (!ContinueOnSemicolons) + return !HasError; + nextToken(); + break; case tok::comma: nextToken(); - StartOfExpression = true; break; default: nextToken(); - StartOfExpression = false; break; } } while (!eof()); + return false; } void UnwrappedLineParser::parseReturn() { nextToken(); do { - switch (FormatTok.Tok.getKind()) { + switch (FormatTok->Tok.getKind()) { case tok::l_brace: parseBracedList(); - if (FormatTok.Tok.isNot(tok::semi)) { + if (FormatTok->Tok.isNot(tok::semi)) { // Assume missing ';'. addUnwrappedLine(); return; @@ -498,6 +885,9 @@ void UnwrappedLineParser::parseReturn() { nextToken(); addUnwrappedLine(); return; + case tok::l_square: + tryToParseLambda(); + break; default: nextToken(); break; @@ -506,29 +896,31 @@ void UnwrappedLineParser::parseReturn() { } void UnwrappedLineParser::parseParens() { - assert(FormatTok.Tok.is(tok::l_paren) && "'(' expected."); + assert(FormatTok->Tok.is(tok::l_paren) && "'(' expected."); nextToken(); do { - switch (FormatTok.Tok.getKind()) { + switch (FormatTok->Tok.getKind()) { case tok::l_paren: parseParens(); break; case tok::r_paren: nextToken(); return; + case tok::r_brace: + // A "}" inside parenthesis is an error if there wasn't a matching "{". + return; + case tok::l_square: + tryToParseLambda(); + break; case tok::l_brace: { - nextToken(); - ScopedLineState LineState(*this); - ScopedDeclarationState DeclarationState(*Line, DeclarationScopeStack, - /*MustBeDeclaration=*/ false); - Line->Level += 1; - parseLevel(/*HasOpeningBrace=*/ true); - Line->Level -= 1; + if (!tryToParseBracedList()) { + parseChildBlock(); + } break; } case tok::at: nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) + if (FormatTok->Tok.is(tok::l_brace)) parseBracedList(); break; default: @@ -539,26 +931,33 @@ void UnwrappedLineParser::parseParens() { } void UnwrappedLineParser::parseIfThenElse() { - assert(FormatTok.Tok.is(tok::kw_if) && "'if' expected"); + assert(FormatTok->Tok.is(tok::kw_if) && "'if' expected"); nextToken(); - if (FormatTok.Tok.is(tok::l_paren)) + if (FormatTok->Tok.is(tok::l_paren)) parseParens(); bool NeedsUnwrappedLine = false; - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false); - NeedsUnwrappedLine = true; + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + else + NeedsUnwrappedLine = true; } else { addUnwrappedLine(); ++Line->Level; parseStructuralElement(); --Line->Level; } - if (FormatTok.Tok.is(tok::kw_else)) { + if (FormatTok->Tok.is(tok::kw_else)) { nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); addUnwrappedLine(); - } else if (FormatTok.Tok.is(tok::kw_if)) { + } else if (FormatTok->Tok.is(tok::kw_if)) { parseIfThenElse(); } else { addUnwrappedLine(); @@ -572,15 +971,22 @@ void UnwrappedLineParser::parseIfThenElse() { } void UnwrappedLineParser::parseNamespace() { - assert(FormatTok.Tok.is(tok::kw_namespace) && "'namespace' expected"); + assert(FormatTok->Tok.is(tok::kw_namespace) && "'namespace' expected"); nextToken(); - if (FormatTok.Tok.is(tok::identifier)) + if (FormatTok->Tok.is(tok::identifier)) nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ true, 0); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Linux || + Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + + bool AddLevel = Style.NamespaceIndentation == FormatStyle::NI_All || + (Style.NamespaceIndentation == FormatStyle::NI_Inner && + DeclarationScopeStack.size() > 1); + parseBlock(/*MustBeDeclaration=*/true, AddLevel); // Munch the semicolon after a namespace. This is more common than one would // think. Puttin the semicolon into its own line is very ugly. - if (FormatTok.Tok.is(tok::semi)) + if (FormatTok->Tok.is(tok::semi)) nextToken(); addUnwrappedLine(); } @@ -588,13 +994,15 @@ void UnwrappedLineParser::parseNamespace() { } void UnwrappedLineParser::parseForOrWhileLoop() { - assert((FormatTok.Tok.is(tok::kw_for) || FormatTok.Tok.is(tok::kw_while)) && + assert((FormatTok->Tok.is(tok::kw_for) || FormatTok->Tok.is(tok::kw_while)) && "'for' or 'while' expected"); nextToken(); - if (FormatTok.Tok.is(tok::l_paren)) + if (FormatTok->Tok.is(tok::l_paren)) parseParens(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); addUnwrappedLine(); } else { addUnwrappedLine(); @@ -605,10 +1013,12 @@ void UnwrappedLineParser::parseForOrWhileLoop() { } void UnwrappedLineParser::parseDoWhile() { - assert(FormatTok.Tok.is(tok::kw_do) && "'do' expected"); + assert(FormatTok->Tok.is(tok::kw_do) && "'do' expected"); nextToken(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); } else { addUnwrappedLine(); ++Line->Level; @@ -617,7 +1027,7 @@ void UnwrappedLineParser::parseDoWhile() { } // FIXME: Add error handling. - if (!FormatTok.Tok.is(tok::kw_while)) { + if (!FormatTok->Tok.is(tok::kw_while)) { addUnwrappedLine(); return; } @@ -627,90 +1037,84 @@ void UnwrappedLineParser::parseDoWhile() { } void UnwrappedLineParser::parseLabel() { - if (FormatTok.Tok.isNot(tok::colon)) - return; nextToken(); unsigned OldLineLevel = Line->Level; if (Line->Level > 1 || (!Line->InPPDirective && Line->Level > 0)) --Line->Level; - if (CommentsBeforeNextToken.empty() && FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false); - if (FormatTok.Tok.is(tok::kw_break)) - parseStructuralElement(); // "break;" after "}" goes on the same line. + if (CommentsBeforeNextToken.empty() && FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); + if (FormatTok->Tok.is(tok::kw_break)) { + // "break;" after "}" on its own line only for BS_Allman + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseStructuralElement(); + } } addUnwrappedLine(); Line->Level = OldLineLevel; } void UnwrappedLineParser::parseCaseLabel() { - assert(FormatTok.Tok.is(tok::kw_case) && "'case' expected"); + assert(FormatTok->Tok.is(tok::kw_case) && "'case' expected"); // FIXME: fix handling of complex expressions here. do { nextToken(); - } while (!eof() && !FormatTok.Tok.is(tok::colon)); + } while (!eof() && !FormatTok->Tok.is(tok::colon)); parseLabel(); } void UnwrappedLineParser::parseSwitch() { - assert(FormatTok.Tok.is(tok::kw_switch) && "'switch' expected"); + assert(FormatTok->Tok.is(tok::kw_switch) && "'switch' expected"); nextToken(); - if (FormatTok.Tok.is(tok::l_paren)) + if (FormatTok->Tok.is(tok::l_paren)) parseParens(); - if (FormatTok.Tok.is(tok::l_brace)) { - parseBlock(/*MustBeDeclaration=*/ false, Style.IndentCaseLabels ? 2 : 1); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + parseBlock(/*MustBeDeclaration=*/false); addUnwrappedLine(); } else { addUnwrappedLine(); - Line->Level += (Style.IndentCaseLabels ? 2 : 1); + ++Line->Level; parseStructuralElement(); - Line->Level -= (Style.IndentCaseLabels ? 2 : 1); + --Line->Level; } } void UnwrappedLineParser::parseAccessSpecifier() { nextToken(); // Otherwise, we don't know what it is, and we'd better keep the next token. - if (FormatTok.Tok.is(tok::colon)) + if (FormatTok->Tok.is(tok::colon)) nextToken(); addUnwrappedLine(); } void UnwrappedLineParser::parseEnum() { nextToken(); - if (FormatTok.Tok.is(tok::identifier) || - FormatTok.Tok.is(tok::kw___attribute) || - FormatTok.Tok.is(tok::kw___declspec)) { + // Eat up enum class ... + if (FormatTok->Tok.is(tok::kw_class) || + FormatTok->Tok.is(tok::kw_struct)) + nextToken(); + while (FormatTok->Tok.getIdentifierInfo() || + FormatTok->isOneOf(tok::colon, tok::coloncolon)) { nextToken(); // We can have macros or attributes in between 'enum' and the enum name. - if (FormatTok.Tok.is(tok::l_paren)) { + if (FormatTok->Tok.is(tok::l_paren)) { parseParens(); } - if (FormatTok.Tok.is(tok::identifier)) + if (FormatTok->Tok.is(tok::identifier)) nextToken(); } - if (FormatTok.Tok.is(tok::l_brace)) { - nextToken(); - addUnwrappedLine(); - ++Line->Level; - do { - switch (FormatTok.Tok.getKind()) { - case tok::l_paren: - parseParens(); - break; - case tok::r_brace: - addUnwrappedLine(); - nextToken(); - --Line->Level; - return; - case tok::comma: + if (FormatTok->Tok.is(tok::l_brace)) { + FormatTok->BlockKind = BK_Block; + bool HasError = !parseBracedList(/*ContinueOnSemicolons=*/true); + if (HasError) { + if (FormatTok->is(tok::semi)) nextToken(); - addUnwrappedLine(); - break; - default: - nextToken(); - break; - } - } while (!eof()); + addUnwrappedLine(); + } } // We fall through to parsing a structural element afterwards, so that in // enum A {} n, m; @@ -719,18 +1123,20 @@ void UnwrappedLineParser::parseEnum() { void UnwrappedLineParser::parseRecord() { nextToken(); - if (FormatTok.Tok.is(tok::identifier) || - FormatTok.Tok.is(tok::kw___attribute) || - FormatTok.Tok.is(tok::kw___declspec)) { + if (FormatTok->Tok.is(tok::identifier) || + FormatTok->Tok.is(tok::kw___attribute) || + FormatTok->Tok.is(tok::kw___declspec) || + FormatTok->Tok.is(tok::kw_alignas)) { nextToken(); // We can have macros or attributes in between 'class' and the class name. - if (FormatTok.Tok.is(tok::l_paren)) { + if (FormatTok->Tok.is(tok::l_paren)) { parseParens(); } // The actual identifier can be a nested name specifier, and in macros // it is often token-pasted. - while (FormatTok.Tok.is(tok::identifier) || - FormatTok.Tok.is(tok::coloncolon) || FormatTok.Tok.is(tok::hashhash)) + while (FormatTok->Tok.is(tok::identifier) || + FormatTok->Tok.is(tok::coloncolon) || + FormatTok->Tok.is(tok::hashhash)) nextToken(); // Note that parsing away template declarations here leads to incorrectly @@ -743,37 +1149,49 @@ void UnwrappedLineParser::parseRecord() { // and thus rule out the record production in case there is no template // (this would still leave us with an ambiguity between template function // and class declarations). - if (FormatTok.Tok.is(tok::colon) || FormatTok.Tok.is(tok::less)) { - while (!eof() && FormatTok.Tok.isNot(tok::l_brace)) { - if (FormatTok.Tok.is(tok::semi)) + if (FormatTok->Tok.is(tok::colon) || FormatTok->Tok.is(tok::less)) { + while (!eof() && FormatTok->Tok.isNot(tok::l_brace)) { + if (FormatTok->Tok.is(tok::semi)) return; nextToken(); } } } - if (FormatTok.Tok.is(tok::l_brace)) - parseBlock(/*MustBeDeclaration=*/ true); + if (FormatTok->Tok.is(tok::l_brace)) { + if (Style.BreakBeforeBraces == FormatStyle::BS_Linux || + Style.BreakBeforeBraces == FormatStyle::BS_Allman) + addUnwrappedLine(); + + parseBlock(/*MustBeDeclaration=*/true, /*Addlevel=*/true, + /*MunchSemi=*/false); + } // We fall through to parsing a structural element afterwards, so // class A {} n, m; // will end up in one unwrapped line. } void UnwrappedLineParser::parseObjCProtocolList() { - assert(FormatTok.Tok.is(tok::less) && "'<' expected."); + assert(FormatTok->Tok.is(tok::less) && "'<' expected."); do nextToken(); - while (!eof() && FormatTok.Tok.isNot(tok::greater)); + while (!eof() && FormatTok->Tok.isNot(tok::greater)); nextToken(); // Skip '>'. } void UnwrappedLineParser::parseObjCUntilAtEnd() { do { - if (FormatTok.Tok.isObjCAtKeyword(tok::objc_end)) { + if (FormatTok->Tok.isObjCAtKeyword(tok::objc_end)) { nextToken(); addUnwrappedLine(); break; } - parseStructuralElement(); + if (FormatTok->is(tok::l_brace)) { + parseBlock(/*MustBeDeclaration=*/false); + // In ObjC interfaces, nothing should be following the "}". + addUnwrappedLine(); + } else { + parseStructuralElement(); + } } while (!eof()); } @@ -782,19 +1200,19 @@ void UnwrappedLineParser::parseObjCInterfaceOrImplementation() { nextToken(); // interface name // @interface can be followed by either a base class, or a category. - if (FormatTok.Tok.is(tok::colon)) { + if (FormatTok->Tok.is(tok::colon)) { nextToken(); nextToken(); // base class name - } else if (FormatTok.Tok.is(tok::l_paren)) + } else if (FormatTok->Tok.is(tok::l_paren)) // Skip category, if present. parseParens(); - if (FormatTok.Tok.is(tok::less)) + if (FormatTok->Tok.is(tok::less)) parseObjCProtocolList(); // If instance variables are present, keep the '{' on the first line too. - if (FormatTok.Tok.is(tok::l_brace)) - parseBlock(/*MustBeDeclaration=*/ true); + if (FormatTok->Tok.is(tok::l_brace)) + parseBlock(/*MustBeDeclaration=*/true); // With instance variables, this puts '}' on its own line. Without instance // variables, this ends the @interface line. @@ -807,11 +1225,11 @@ void UnwrappedLineParser::parseObjCProtocol() { nextToken(); nextToken(); // protocol name - if (FormatTok.Tok.is(tok::less)) + if (FormatTok->Tok.is(tok::less)) parseObjCProtocolList(); // Check for protocol declaration. - if (FormatTok.Tok.is(tok::semi)) { + if (FormatTok->Tok.is(tok::semi)) { nextToken(); return addUnwrappedLine(); } @@ -820,24 +1238,40 @@ void UnwrappedLineParser::parseObjCProtocol() { parseObjCUntilAtEnd(); } +LLVM_ATTRIBUTE_UNUSED static void printDebugInfo(const UnwrappedLine &Line, + StringRef Prefix = "") { + llvm::dbgs() << Prefix << "Line(" << Line.Level << ")" + << (Line.InPPDirective ? " MACRO" : "") << ": "; + for (std::list<UnwrappedLineNode>::const_iterator I = Line.Tokens.begin(), + E = Line.Tokens.end(); + I != E; ++I) { + llvm::dbgs() << I->Tok->Tok.getName() << "[" << I->Tok->Type << "] "; + } + for (std::list<UnwrappedLineNode>::const_iterator I = Line.Tokens.begin(), + E = Line.Tokens.end(); + I != E; ++I) { + const UnwrappedLineNode &Node = *I; + for (SmallVectorImpl<UnwrappedLine>::const_iterator + I = Node.Children.begin(), + E = Node.Children.end(); + I != E; ++I) { + printDebugInfo(*I, "\nChild: "); + } + } + llvm::dbgs() << "\n"; +} + void UnwrappedLineParser::addUnwrappedLine() { if (Line->Tokens.empty()) return; DEBUG({ - llvm::dbgs() << "Line(" << Line->Level << ")" - << (Line->InPPDirective ? " MACRO" : "") << ": "; - for (std::list<FormatToken>::iterator I = Line->Tokens.begin(), - E = Line->Tokens.end(); - I != E; ++I) { - llvm::dbgs() << I->Tok.getName() << " "; - - } - llvm::dbgs() << "\n"; + if (CurrentLines == &Lines) + printDebugInfo(*Line); }); CurrentLines->push_back(*Line); Line->Tokens.clear(); if (CurrentLines == &Lines && !PreprocessorDirectives.empty()) { - for (std::vector<UnwrappedLine>::iterator + for (SmallVectorImpl<UnwrappedLine>::iterator I = PreprocessorDirectives.begin(), E = PreprocessorDirectives.end(); I != E; ++I) { @@ -847,15 +1281,15 @@ void UnwrappedLineParser::addUnwrappedLine() { } } -bool UnwrappedLineParser::eof() const { return FormatTok.Tok.is(tok::eof); } +bool UnwrappedLineParser::eof() const { return FormatTok->Tok.is(tok::eof); } void UnwrappedLineParser::flushComments(bool NewlineBeforeNext) { bool JustComments = Line->Tokens.empty(); - for (SmallVectorImpl<FormatToken>::const_iterator + for (SmallVectorImpl<FormatToken *>::const_iterator I = CommentsBeforeNextToken.begin(), E = CommentsBeforeNextToken.end(); I != E; ++I) { - if (I->NewlinesBefore && JustComments) { + if ((*I)->NewlinesBefore && JustComments) { addUnwrappedLine(); } pushToken(*I); @@ -869,7 +1303,7 @@ void UnwrappedLineParser::flushComments(bool NewlineBeforeNext) { void UnwrappedLineParser::nextToken() { if (eof()) return; - flushComments(FormatTok.NewlinesBefore > 0); + flushComments(FormatTok->NewlinesBefore > 0); pushToken(FormatTok); readToken(); } @@ -878,8 +1312,8 @@ void UnwrappedLineParser::readToken() { bool CommentsInCurrentLine = true; do { FormatTok = Tokens->getNextToken(); - while (!Line->InPPDirective && FormatTok.Tok.is(tok::hash) && - (FormatTok.HasUnescapedNewline || FormatTok.IsFirst)) { + while (!Line->InPPDirective && FormatTok->Tok.is(tok::hash) && + (FormatTok->HasUnescapedNewline || FormatTok->IsFirst)) { // If there is an unfinished unwrapped line, we flush the preprocessor // directives only after that unwrapped line was finished later. bool SwitchToPreprocessorLines = @@ -888,12 +1322,18 @@ void UnwrappedLineParser::readToken() { // Comments stored before the preprocessor directive need to be output // before the preprocessor directive, at the same level as the // preprocessor directive, as we consider them to apply to the directive. - flushComments(FormatTok.NewlinesBefore > 0); + flushComments(FormatTok->NewlinesBefore > 0); parsePPDirective(); } - if (!FormatTok.Tok.is(tok::comment)) + + if (!PPStack.empty() && (PPStack.back() == PP_Unreachable) && + !Line->InPPDirective) { + continue; + } + + if (!FormatTok->Tok.is(tok::comment)) return; - if (FormatTok.NewlinesBefore > 0 || FormatTok.IsFirst) { + if (FormatTok->NewlinesBefore > 0 || FormatTok->IsFirst) { CommentsInCurrentLine = false; } if (CommentsInCurrentLine) { @@ -904,10 +1344,10 @@ void UnwrappedLineParser::readToken() { } while (!eof()); } -void UnwrappedLineParser::pushToken(const FormatToken &Tok) { - Line->Tokens.push_back(Tok); +void UnwrappedLineParser::pushToken(FormatToken *Tok) { + Line->Tokens.push_back(UnwrappedLineNode(Tok)); if (MustBreakBeforeNextToken) { - Line->Tokens.back().MustBreakBefore = true; + Line->Tokens.back().Tok->MustBreakBefore = true; MustBreakBeforeNextToken = false; } } |
