[analyzer] Rework both constraint managers to handle mixed-type comparisons.

This involves keeping track of three separate types: the symbol type, the
adjustment type, and the comparison type. For example, in "$x + 5 > 0ULL",
if the type of $x is 'signed char', the adjustment type is 'int' and the
comparison type is 'unsigned long long'. Most of the time these three types
will be the same, but we should still do the right thing when the
comparison value is out of range, and wraparound should be calculated in
the adjustment type.

This also re-disables an out-of-bounds test; we were extracting the symbol
from non-additive SymIntExprs, but then throwing away the integer.

Sorry for the large patch; both the basic and range constraint managers needed
to be updated together, since they share code in SimpleConstraintManager.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@156361 91177308-0d34-0410-b5e6-96231b3b80d8

1 files changed, 38 insertions, 0 deletions

diff --git a/lib/StaticAnalyzer/Core/APSIntType.cpp b/lib/StaticAnalyzer/Core/APSIntType.cpp
new file mode 100644
index 0000000000..884b0faa9e
--- /dev/null
+++ b/lib/StaticAnalyzer/Core/APSIntType.cpp
@@ -0,0 +1,38 @@
+//===--- APSIntType.cpp - Simple record of the type of APSInts ------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/StaticAnalyzer/Core/PathSensitive/APSIntType.h"
+
+using namespace clang;
+using namespace ento;
+
+APSIntType::RangeTestResultKind
+APSIntType::testInRange(const llvm::APSInt &Value) const {
+  // Negative numbers cannot be losslessly converted to unsigned type.
+  if (IsUnsigned && Value.isSigned() && Value.isNegative())
+    return RTR_Below;
+
+  // Signed integers can be converted to signed integers of the same width
+  // or (if positive) unsigned integers with one fewer bit.
+  // Unsigned integers can be converted to unsigned integers of the same width
+  // or signed integers with one more bit.
+  unsigned MinBits;
+  if (Value.isSigned())
+    MinBits = Value.getMinSignedBits() - IsUnsigned;
+  else
+    MinBits = Value.getActiveBits() + !IsUnsigned;
+
+  if (MinBits <= BitWidth)
+    return RTR_Within;
+
+  if (Value.isSigned() && Value.isNegative())
+    return RTR_Below;
+  else
+    return RTR_Above;
+}