sed: improve integer overflow checking

Fix some some longstanding but unlikely integer overflows.
Internally, 'sed' now more often prefers signed integer arithmetic,
which can be checked automatically via 'gcc -fsanitize=undefined'.
* basicdefs.h (countT): Remove.  All uses replaced
with a more-specific signed type, e.g., idx_t.
Similarly, change uses of types like size_t to
signed types like idx_t when appropriate.
(REALLOC): Remove; no longer used.  We now use xpalloc
because that detects integer overflow in size calculations.
Also, we no longer use XCALLOC since the code never relies
on the storage being zero, and leaving it uninitialized is
more likely to catch errors when debugging implementations
are used.  We use XNMALLOC instead, or xpalloc.
* bootstrap.conf (gnulib_modules): Add stdckdint, strtoimax.
* lib/.gitignore, m4/.gitignore: Update for new Gnulib modules.
* sed/compile.c: Include stdckdint.h.
(VECTOR_ALLOC_INCREMENT): Remove; no longer used.
(in_integer): Return maximal value if integer overflow.
All callers changed to expect this.
(next_cmd_entry): Use xpalloc instead of reallocating by hand,
which might suffer integer overflow.
(normalize_text): Don’t rely on system-defined conversion
of out-of-range size_t to int.
(next_cmd_entry): Arg is now pointer, not pointer-to-pointer.
All uses changed.
* sed/debug.c (debug_print_function): Don’t attempt to
fwrite a null pointer with a zero size.
* sed/execute.c: Include <stdckdint.h>, "minmax.h".
(resize_line): LEN arg is now increment, not total length,
to avoid overflow when calculating total length.
All uses changed.  Do not assume lb->alloc * 2 cannot overflow.
(resize_line, line_copy): Use xpalloc instead of doing realloc by
hand, which might suffer integer overflow.
(str_append_modified): Do not add n to to->length until
after it's known this cannot overflow.
(read_file_line): Don’t assume ssize_t fits in long.
(get_backup_file_name): Don’t assume string length fits in int.
Do not assume PTR-1+1 works; behavior is undefined if PTR
is at buffer start.  Check for integer overflow in buffer
size calculation.
(read_pattern_space): Check for line number overflow.
(match_address_p): Check for address overflow.
(debug_print_line): Omit unnecessary test for in->active being null.
(execute_program): Check for Q overflow.
* sed/regexp.c: Include <stdckdint.h>.
(match_regex): Don’t assume TYPE_MAXIMUM (regoff_t) == INT_MAX.
* sed/sed.c: Include inttypes.h, for strtoimax.
(main): Use strtoimax, not atoi.
* sed/utils.c (init_buffer): Use xmalloc and xpalloc
instead of guessing sizes ourselves, and unnecessarily
initializing.
(resize_buffer): Remove; all callers changed to use xpalloc.
(free_buffer): Don’t call free (NULL), since we already
test whether the pointer is null.

1 files changed, 2 insertions, 0 deletions

diff --git a/bootstrap.conf b/bootstrap.conf
index d491028..8db6f04 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -59,7 +59,9 @@ selinux-h
 ssize_t
 stat-macros
 stdbool
+stdckdint
 strerror
+strtoimax
 strverscmp
 unlocked-io
 update-copyright