sed: improve integer overflow checking

Fix some some longstanding but unlikely integer overflows.
Internally, 'sed' now more often prefers signed integer arithmetic,
which can be checked automatically via 'gcc -fsanitize=undefined'.
* basicdefs.h (countT): Remove.  All uses replaced
with a more-specific signed type, e.g., idx_t.
Similarly, change uses of types like size_t to
signed types like idx_t when appropriate.
(REALLOC): Remove; no longer used.  We now use xpalloc
because that detects integer overflow in size calculations.
Also, we no longer use XCALLOC since the code never relies
on the storage being zero, and leaving it uninitialized is
more likely to catch errors when debugging implementations
are used.  We use XNMALLOC instead, or xpalloc.
* bootstrap.conf (gnulib_modules): Add stdckdint, strtoimax.
* lib/.gitignore, m4/.gitignore: Update for new Gnulib modules.
* sed/compile.c: Include stdckdint.h.
(VECTOR_ALLOC_INCREMENT): Remove; no longer used.
(in_integer): Return maximal value if integer overflow.
All callers changed to expect this.
(next_cmd_entry): Use xpalloc instead of reallocating by hand,
which might suffer integer overflow.
(normalize_text): Don’t rely on system-defined conversion
of out-of-range size_t to int.
(next_cmd_entry): Arg is now pointer, not pointer-to-pointer.
All uses changed.
* sed/debug.c (debug_print_function): Don’t attempt to
fwrite a null pointer with a zero size.
* sed/execute.c: Include <stdckdint.h>, "minmax.h".
(resize_line): LEN arg is now increment, not total length,
to avoid overflow when calculating total length.
All uses changed.  Do not assume lb->alloc * 2 cannot overflow.
(resize_line, line_copy): Use xpalloc instead of doing realloc by
hand, which might suffer integer overflow.
(str_append_modified): Do not add n to to->length until
after it's known this cannot overflow.
(read_file_line): Don’t assume ssize_t fits in long.
(get_backup_file_name): Don’t assume string length fits in int.
Do not assume PTR-1+1 works; behavior is undefined if PTR
is at buffer start.  Check for integer overflow in buffer
size calculation.
(read_pattern_space): Check for line number overflow.
(match_address_p): Check for address overflow.
(debug_print_line): Omit unnecessary test for in->active being null.
(execute_program): Check for Q overflow.
* sed/regexp.c: Include <stdckdint.h>.
(match_regex): Don’t assume TYPE_MAXIMUM (regoff_t) == INT_MAX.
* sed/sed.c: Include inttypes.h, for strtoimax.
(main): Use strtoimax, not atoi.
* sed/utils.c (init_buffer): Use xmalloc and xpalloc
instead of guessing sizes ourselves, and unnecessarily
initializing.
(resize_buffer): Remove; all callers changed to use xpalloc.
(free_buffer): Don’t call free (NULL), since we already
test whether the pointer is null.

1 files changed, 7 insertions, 10 deletions

diff --git a/sed/utils.h b/sed/utils.h
index cac8a05..1713307 100644
--- a/sed/utils.h
+++ b/sed/utils.h
@@ -31,26 +31,23 @@ _Noreturn void panic (const char *str, ...)
 
 FILE *ck_fopen (const char *name, const char *mode, int fail);
 FILE *ck_fdopen (int fd, const char *name, const char *mode, int fail);
-void ck_fwrite (const void *ptr, size_t size, size_t nmemb, FILE *stream);
-size_t ck_fread (void *ptr, size_t size, size_t nmemb, FILE *stream);
+void ck_fwrite (const void *ptr, idx_t size, idx_t nmemb, FILE *stream);
+idx_t ck_fread (void *ptr, idx_t size, idx_t nmemb, FILE *stream);
 void ck_fflush (FILE *stream);
 void ck_fclose (FILE *stream);
 const char *follow_symlink (const char *path);
-size_t ck_getdelim (char **text, size_t *buflen, char buffer_delimiter,
-                    FILE *stream);
+ssize_t ck_getdelim (char **text, size_t *buflen, char buffer_delimiter,
+                     FILE *stream);
 FILE * ck_mkstemp (char **p_filename, const char *tmpdir, const char *base,
                    const char *mode) _GL_ARG_NONNULL ((1, 2, 3, 4));
 void ck_rename (const char *from, const char *to);
 
-void *ck_malloc (size_t size);
-void *ck_realloc (void *ptr, size_t size);
-
 void cancel_cleanup (void);
 void remove_cleanup_file (void);
 
-struct buffer *init_buffer (void);
+struct buffer *init_buffer (void) _GL_ATTRIBUTE_MALLOC;
 char *get_buffer (struct buffer const *b) _GL_ATTRIBUTE_PURE;
-size_t size_buffer (struct buffer const *b) _GL_ATTRIBUTE_PURE;
-char *add_buffer (struct buffer *b, const char *p, size_t n);
+idx_t size_buffer (struct buffer const *b) _GL_ATTRIBUTE_PURE;
+char *add_buffer (struct buffer *b, const char *p, idx_t n);
 char *add1_buffer (struct buffer *b, int ch);
 void free_buffer (struct buffer *b);