Improve behavior in the case of stack exhaustion.

Summary:
Clang performs various recursive operations (such as template instantiation),
and may use non-trivial amounts of stack space in each recursive step (for
instance, due to recursive AST walks). While we try to keep the stack space
used by such steps to a minimum and we have explicit limits on the number of
such steps we perform, it's impractical to guarantee that we won't blow out the
stack on deeply recursive template instantiations on complex ASTs, even with
only a moderately high instantiation depth limit.

The user experience in these cases is generally terrible: we crash with
no hint of what went wrong. Under this patch, we attempt to do better:

 * Detect when the stack is nearly exhausted, and produce a warning with a
   nice template instantiation backtrace, telling the user that we might
   run slowly or crash.

 * For cases where we're forced to trigger recursive template
   instantiation in arbitrarily-deeply-nested contexts, check whether
   we're nearly out of stack space and allocate a new stack (by spawning
   a new thread) after producing the warning.

Reviewers: rnk, aaron.ballman

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66361

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@369940 91177308-0d34-0410-b5e6-96231b3b80d8

2 files changed, 72 insertions, 0 deletions

diff --git a/lib/Basic/CMakeLists.txt b/lib/Basic/CMakeLists.txt
index b4d6ced4d5..be739c7046 100644
--- a/lib/Basic/CMakeLists.txt
+++ b/lib/Basic/CMakeLists.txt
@@ -60,6 +60,7 @@ add_clang_library(clangBasic
   Sanitizers.cpp
   SourceLocation.cpp
   SourceManager.cpp
+  Stack.cpp
   TargetInfo.cpp
   Targets.cpp
   Targets/AArch64.cpp
diff --git a/lib/Basic/Stack.cpp b/lib/Basic/Stack.cpp
new file mode 100644
index 0000000000..667db5c51f
--- /dev/null
+++ b/lib/Basic/Stack.cpp
@@ -0,0 +1,71 @@
+//===--- Stack.h - Utilities for dealing with stack space -------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Defines utilities for dealing with stack allocation and stack space.
+///
+//===----------------------------------------------------------------------===//
+
+#include "clang/Basic/Stack.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/Support/CrashRecoveryContext.h"
+
+static LLVM_THREAD_LOCAL void *BottomOfStack = nullptr;
+
+static void *getStackPointer() {
+#if __GNUC__ || __has_builtin(__builtin_frame_address)
+  return __builtin_frame_address(0);
+#elif defined(_MSC_VER)
+  return _AddressOfReturnAddress();
+#else
+  char CharOnStack = 0;
+  // The volatile store here is intended to escape the local variable, to
+  // prevent the compiler from optimizing CharOnStack into anything other
+  // than a char on the stack.
+  //
+  // Tested on: MSVC 2015 - 2019, GCC 4.9 - 9, Clang 3.2 - 9, ICC 13 - 19.
+  char *volatile Ptr = &CharOnStack;
+  return Ptr;
+#endif
+}
+
+void clang::noteBottomOfStack() {
+  if (!BottomOfStack)
+    BottomOfStack = getStackPointer();
+}
+
+bool clang::isStackNearlyExhausted() {
+  // We consider 256 KiB to be sufficient for any code that runs between checks
+  // for stack size.
+  constexpr size_t SufficientStack = 256 << 10;
+
+  // If we don't know where the bottom of the stack is, hope for the best.
+  if (!BottomOfStack)
+    return false;
+
+  intptr_t StackDiff = (intptr_t)getStackPointer() - (intptr_t)BottomOfStack;
+  size_t StackUsage = (size_t)std::abs(StackDiff);
+
+  // If the stack pointer has a surprising value, we do not understand this
+  // stack usage scheme. (Perhaps the target allocates new stack regions on
+  // demand for us.) Don't try to guess what's going on.
+  if (StackUsage > DesiredStackSize)
+    return false;
+
+  return StackUsage >= DesiredStackSize - SufficientStack;
+}
+
+void clang::runWithSufficientStackSpaceSlow(llvm::function_ref<void()> Diag,
+                                            llvm::function_ref<void()> Fn) {
+  llvm::CrashRecoveryContext CRC;
+  CRC.RunSafelyOnThread([&] {
+    noteBottomOfStack();
+    Diag();
+    Fn();
+  }, DesiredStackSize);
+}