[clang-scan-deps] add skip excluded conditional preprocessor block preprocessing optimization

This commit adds an optimization to clang-scan-deps and clang's preprocessor that skips excluded preprocessor
blocks by bumping the lexer pointer, and not lexing the tokens until reaching appropriate #else/#endif directive.
The skip positions and lexer offsets are computed when the file is minimized, directly from the minimized tokens.

On an 18-core iMacPro with macOS Catalina Beta I got 10-15% speed-up from this optimization when running clang-scan-deps on
the compilation database for a recent LLVM and Clang (3511 files).

Differential Revision: https://reviews.llvm.org/D67127

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@371656 91177308-0d34-0410-b5e6-96231b3b80d8

1 files changed, 18 insertions, 0 deletions

diff --git a/include/clang/Lex/DependencyDirectivesSourceMinimizer.h b/include/clang/Lex/DependencyDirectivesSourceMinimizer.h
index cc7be3c5b6..d832df6b61 100644
--- a/include/clang/Lex/DependencyDirectivesSourceMinimizer.h
+++ b/include/clang/Lex/DependencyDirectivesSourceMinimizer.h
@@ -66,6 +66,24 @@ struct Token {
   Token(TokenKind K, int Offset) : K(K), Offset(Offset) {}
 };
 
+/// Simplified token range to track the range of a potentially skippable PP
+/// directive.
+struct SkippedRange {
+  /// Offset into the output byte stream of where the skipped directive begins.
+  int Offset;
+
+  /// The number of bytes that can be skipped before the preprocessing must
+  /// resume.
+  int Length;
+};
+
+/// Computes the potential source ranges that can be skipped by the preprocessor
+/// when skipping a directive like #if, #ifdef or #elsif.
+///
+/// \returns false on success, true on error.
+bool computeSkippedRanges(ArrayRef<Token> Input,
+                          llvm::SmallVectorImpl<SkippedRange> &Range);
+
 } // end namespace minimize_source_to_dependency_directives
 
 /// Minimize the input down to the preprocessor directives that might have