From 9316d536321af0c44f75a1e62d910f6444f02318 Mon Sep 17 00:00:00 2001 From: Jim Meyering Date: Sat, 14 Dec 2013 09:34:08 -0800 Subject: grep: handle lines longer than INT_MAX on more systems When trying to exercize some long-line-handling code, I ran these commands: $ dd bs=1 seek=2G of=big < /dev/null; grep -l x big; echo $? grep: big: Invalid argument 2 grep should not have issued that diagnostic, and it should have exited with status 1, not 2. What happened? grep read the 2GiB of NULs, doubled its buffer size, copied the 2GiB into the new 4GiB buffer, and proceeded to call "read" with a byte-count argument of 2^32. On at least Darwin 12.5.0, that makes read fail with EINVAL. The solution is to use gnulib's safe_read wrapper. * src/main.c: Include "safe-read.h" (fillbuf): Use safe_read, rather than bare read. The latter cannot handle a read size of 2^32 on some systems. * bootstrap.conf (gnulib_modules): Add safe-read. * tests/long-line-vs-2GiB-read: New file. * tests/Makefile.am (TESTS): Add it. * NEWS (Bug fixes): Mention it. --- NEWS | 2 ++ bootstrap.conf | 1 + src/main.c | 3 ++- tests/Makefile.am | 1 + tests/long-line-vs-2GiB-read | 22 ++++++++++++++++++++++ 5 files changed, 28 insertions(+), 1 deletion(-) create mode 100755 tests/long-line-vs-2GiB-read diff --git a/NEWS b/NEWS index e974f29d..80a064bf 100644 --- a/NEWS +++ b/NEWS @@ -20,6 +20,8 @@ GNU grep NEWS -*- outline -*- printf '\x82'|LC_ALL=en_US.UTF-8 grep -P y Now grep handles arbitrary PCRE errors. [bug introduced in grep-2.15] + Handle very long lines (2GiB and longer) on systems with a deficient + read system call. * Noteworthy changes in release 2.15 (2013-10-26) [stable] diff --git a/bootstrap.conf b/bootstrap.conf index 8f6f5a3d..580f01c2 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -68,6 +68,7 @@ quote readme-release realloc-gnu regex +safe-read same-inode ssize_t stddef diff --git a/src/main.c b/src/main.c index 7ecf00a5..6ee1632b 100644 --- a/src/main.c +++ b/src/main.c @@ -44,6 +44,7 @@ #include "progname.h" #include "propername.h" #include "quote.h" +#include "safe-read.h" #include "version-etc.h" #include "xalloc.h" #include "xstrtol.h" @@ -641,7 +642,7 @@ fillbuf (size_t save, struct stat const *st) readsize = buffer + bufalloc - readbuf; readsize -= readsize % pagesize; - fillsize = read (bufdesc, readbuf, readsize); + fillsize = safe_read (bufdesc, readbuf, readsize); if (fillsize < 0) fillsize = cc = 0; bufoffset += fillsize; diff --git a/tests/Makefile.am b/tests/Makefile.am index 76b8c52b..72af1556 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -71,6 +71,7 @@ TESTS = \ inconsistent-range \ invalid-multibyte-infloop \ khadafy \ + long-line-vs-2GiB-read \ max-count-vs-context \ multibyte-white-space \ empty-line-mb \ diff --git a/tests/long-line-vs-2GiB-read b/tests/long-line-vs-2GiB-read new file mode 100755 index 00000000..773c542d --- /dev/null +++ b/tests/long-line-vs-2GiB-read @@ -0,0 +1,22 @@ +#!/bin/sh +# Ensure that grep can handle lines 2GiB long. +# Before grep-2.16, a line of length 2^31 or greater would provoke +# an "Invalid argument" (EINVAL) failure from the read syscall on +# systems like OS/X 10.8.5. + +. "${srcdir=.}/init.sh"; path_prepend_ ../src + +# Searching 2GiB takes a while. +expensive_ + +echo big > exp || framework_failure_ + +MiB=1048576 +dd bs=$MiB seek=2048 of=big < /dev/null || framework_failure_ +echo x >> big || framework_failure_ +grep -l x big > out 2> err || fail=1 + +compare exp out || fail=1 +compare /dev/null err || fail=1 + +Exit $fail -- cgit v1.2.1