qUn/Compress: reject negative lengths

In qCompress, we've been calculating postive len values out of them,
only to fail at random points later, possibly running into UB. Fail
early instead.

In qUncompress, we've been catching negative values, and reported them
indiscriminately as "invalid data". Use a better warning message
instead.

By rights, nbytes ≥ 0 would be a precondition of both functions (which
we would Q_ASSERT() on), but seeing we're picking this back into LTS
branches, I found it prudent to use a non-fatal way to signal the
precondition violation.

If and when we keep these functions for Qt 7, it will be as an
overload that takes QByteArrayView, in which case nbytes ≥ 0 enters as
a hard precondition via the QByteArrayView constructor, so there
appears to be no need to pre-program a Q_ASSERT() for Qt 7.0.

Task-number: QTBUG-104972
Task-number: QTBUG-106542
Change-Id: I6a1b25fe12d31e3d4c845033cad320832976f83c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
(cherry picked from commit bbd1f576f70fb52187185b79636e6591cd17e9b5)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>

1 files changed, 12 insertions, 0 deletions

diff --git a/src/corelib/text/qbytearray.cpp b/src/corelib/text/qbytearray.cpp
index e461167ff3..7d30cc39f0 100644
--- a/src/corelib/text/qbytearray.cpp
+++ b/src/corelib/text/qbytearray.cpp
@@ -559,6 +559,12 @@ static QByteArray dataIsNull(ZLibOp op)
 }
 
 Q_DECL_COLD_FUNCTION
+static QByteArray lengthIsNegative(ZLibOp op)
+{
+    return zlibError(op, "Input length is negative");
+}
+
+Q_DECL_COLD_FUNCTION
 static QByteArray tooMuchData(ZLibOp op)
 {
     return zlibError(op, "Not enough memory");
@@ -579,6 +585,9 @@ QByteArray qCompress(const uchar* data, qsizetype nbytes, int compressionLevel)
     if (!data)
         return dataIsNull(ZLibOp::Compression);
 
+    if (nbytes < 0)
+        return lengthIsNegative(ZLibOp::Compression);
+
     if (compressionLevel < -1 || compressionLevel > 9)
         compressionLevel = -1;
 
@@ -657,6 +666,9 @@ QByteArray qUncompress(const uchar* data, qsizetype nbytes)
     if (!data)
         return dataIsNull(ZLibOp::Decompression);
 
+    if (nbytes < 0)
+        return lengthIsNegative(ZLibOp::Decompression);
+
     constexpr qsizetype HeaderSize = sizeof(CompressSizeHint_t);
     if (nbytes < HeaderSize)
         return invalidCompressedData();