diff options
author | Joseph Myers <joseph@codesourcery.com> | 2020-02-12 23:31:56 +0000 |
---|---|---|
committer | Joseph Myers <joseph@codesourcery.com> | 2020-02-12 23:31:56 +0000 |
commit | 9333498794cde1d5cca518badf79533a24114b6f (patch) | |
tree | 19e5d0d11573074dd2a21f5bf87fc31f94bbced3 /sysdeps/ieee754/ldbl-96/e_rem_pio2l.c | |
parent | 4fbba6fe904d0094ddc4284066b3860d119cbd4a (diff) | |
download | glibc-9333498794cde1d5cca518badf79533a24114b6f.tar.gz |
Avoid ldbl-96 stack corruption from range reduction of pseudo-zero (bug 25487).
Bug 25487 reports stack corruption in ldbl-96 sinl on a pseudo-zero
argument (an representation where all the significand bits, including
the explicit high bit, are zero, but the exponent is not zero, which
is not a valid representation for the long double type).
Although this is not a valid long double representation, existing
practice in this area (see bug 4586, originally marked invalid but
subsequently fixed) is that we still seek to avoid invalid memory
accesses as a result, in case of programs that treat arbitrary binary
data as long double representations, although the invalid
representations of the ldbl-96 format do not need to be consistently
handled the same as any particular valid representation.
This patch makes the range reduction detect pseudo-zero and unnormal
representations that would otherwise go to __kernel_rem_pio2, and
returns a NaN for them instead of continuing with the range reduction
process. (Pseudo-zero and unnormal representations whose unbiased
exponent is less than -1 have already been safely returned from the
function before this point without going through the rest of range
reduction.) Pseudo-zero representations would previously result in
the value passed to __kernel_rem_pio2 being all-zero, which is
definitely unsafe; unnormal representations would previously result in
a value passed whose high bit is zero, which might well be unsafe
since that is not a form of input expected by __kernel_rem_pio2.
Tested for x86_64.
Diffstat (limited to 'sysdeps/ieee754/ldbl-96/e_rem_pio2l.c')
-rw-r--r-- | sysdeps/ieee754/ldbl-96/e_rem_pio2l.c | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c b/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c index 5f742321ae..bcdf20179f 100644 --- a/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c +++ b/sysdeps/ieee754/ldbl-96/e_rem_pio2l.c @@ -210,6 +210,18 @@ __ieee754_rem_pio2l (long double x, long double *y) return 0; } + if ((i0 & 0x80000000) == 0) + { + /* Pseudo-zero and unnormal representations are not valid + representations of long double. We need to avoid stack + corruption in __kernel_rem_pio2, which expects input in a + particular normal form, but those representations do not need + to be consistently handled like any particular floating-point + value. */ + y[1] = y[0] = __builtin_nanl (""); + return 0; + } + /* Split the 64 bits of the mantissa into three 24-bit integers stored in a double array. */ exp = j0 - 23; |