| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix some some longstanding but unlikely integer overflows.
Internally, 'sed' now more often prefers signed integer arithmetic,
which can be checked automatically via 'gcc -fsanitize=undefined'.
* basicdefs.h (countT): Remove. All uses replaced
with a more-specific signed type, e.g., idx_t.
Similarly, change uses of types like size_t to
signed types like idx_t when appropriate.
(REALLOC): Remove; no longer used. We now use xpalloc
because that detects integer overflow in size calculations.
Also, we no longer use XCALLOC since the code never relies
on the storage being zero, and leaving it uninitialized is
more likely to catch errors when debugging implementations
are used. We use XNMALLOC instead, or xpalloc.
* bootstrap.conf (gnulib_modules): Add stdckdint, strtoimax.
* lib/.gitignore, m4/.gitignore: Update for new Gnulib modules.
* sed/compile.c: Include stdckdint.h.
(VECTOR_ALLOC_INCREMENT): Remove; no longer used.
(in_integer): Return maximal value if integer overflow.
All callers changed to expect this.
(next_cmd_entry): Use xpalloc instead of reallocating by hand,
which might suffer integer overflow.
(normalize_text): Don’t rely on system-defined conversion
of out-of-range size_t to int.
(next_cmd_entry): Arg is now pointer, not pointer-to-pointer.
All uses changed.
* sed/debug.c (debug_print_function): Don’t attempt to
fwrite a null pointer with a zero size.
* sed/execute.c: Include <stdckdint.h>, "minmax.h".
(resize_line): LEN arg is now increment, not total length,
to avoid overflow when calculating total length.
All uses changed. Do not assume lb->alloc * 2 cannot overflow.
(resize_line, line_copy): Use xpalloc instead of doing realloc by
hand, which might suffer integer overflow.
(str_append_modified): Do not add n to to->length until
after it's known this cannot overflow.
(read_file_line): Don’t assume ssize_t fits in long.
(get_backup_file_name): Don’t assume string length fits in int.
Do not assume PTR-1+1 works; behavior is undefined if PTR
is at buffer start. Check for integer overflow in buffer
size calculation.
(read_pattern_space): Check for line number overflow.
(match_address_p): Check for address overflow.
(debug_print_line): Omit unnecessary test for in->active being null.
(execute_program): Check for Q overflow.
* sed/regexp.c: Include <stdckdint.h>.
(match_regex): Don’t assume TYPE_MAXIMUM (regoff_t) == INT_MAX.
* sed/sed.c: Include inttypes.h, for strtoimax.
(main): Use strtoimax, not atoi.
* sed/utils.c (init_buffer): Use xmalloc and xpalloc
instead of guessing sizes ourselves, and unnecessarily
initializing.
(resize_buffer): Remove; all callers changed to use xpalloc.
(free_buffer): Don’t call free (NULL), since we already
test whether the pointer is null.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* basicdefs.h, sed/regexp.c (N_): Remove; no longer used.
* bootstrap.conf (XGETTEXT_OPTIONS):
Treat bad_prog as a function that translates its first arg.
Omit unnecessary flags.
* sed/compile.c, sed/regexp.c (errors): Remove this array of
concatenated strings, and associated macros. All users of these
macros now simply call bad_prog ("diagnostic") instead of bad_prog
(_(DIAGNOSTIC_MACRO)). This is easier to read and maintain, and
made it easy to catch two diagnostics that were never properly
translated before: "cannot specify modifiers on empty regexp",
"missing filename in r/R/w/W commands".
* sed/compile.c (bad_command): Remove; no longer used.
(vbad_prog): New function, with most of the old bad_prog contents.
(bad_prog): Now variadic, like printf. Always translate WHY.
All callers changed.
(bad_prog_notranslate): New function.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* bootstrap.conf (gnulib_modules): Remove getopt module, depracated in
https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=0abc38bd2a1398f0
* po/POTFILES.in: Remove getopt.c
* sed/utils.h (panic): Rename _GL_ATTRIBUTE_FORMAT_PRINTF to
_GL_ATTRIBUTE_FORMAT_PRINTF_STANDARD following gnulib's change:
https://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=387d654cabd7bc15
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
|
|
|
|
|
|
|
| |
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
|
|
|
|
| |
* all files: Run "make update-copyright".
|
|
|
|
|
|
|
|
| |
Perl-regexp syntax (PCRE) in GNU Sed is shelved indefinitely.
See https://bugs.gnu.org/22801 , https://bugs.gnu.org/22647 .
Remove all (unused) REG_PERL related code.
* sed/sed.c, sed/sed.h, sed/regexp.c, sed/compile.c: Remove REG_PERL code.
|
|
|
|
|
|
|
| |
* basicdefs.h, sed/compile.c, sed/execute.c, sed/mbcs.c, sed/regexp.c,
sed/sed.c, sed/sed.h, sed/utils.c, sed/utils.h, testsuite/get-mb-cur-max.c,
testsuite/test-mbrtowc.c (GPL License notice): Replace FSF's office
address with 'https://www.gnu.org/licenses/'.
|
|
|
|
|
| |
* sed/regexp.c (match_regex): Free the previously allocated regex struct
before re-building the regex during program execution.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Under normal operation there is no need for explicit freeing,
as all memory will be released when sed terminates.
During development (and specifically, valgrind and address-sanitizing)
enabling lint prevents false-positive warnings about memory
leaks.
Lint mode can be enabled with CFLAGS="-Dlint". It is also automatically
enabled by default when building from git (as opposed to tarball).
For consistency all instances of "#ifdef DEBUG_LEAKS" are replaced
with "#ifdef lint".
* sed/sed.h (struct subst): Add member variable to keep the address of
allocated buffer in compile.c:setup_replacement().
(release_regex): Add declaration.
(finish_program): Function now takes an argument: the sed program
vector.
* sed/sed.c (main): Adjust call to finish_program.
* sed/compile.c (finish_program): Release program allocations.
(setup_replacement): Remember the allocated buffer address.
(compile_program): Free temporary array in 'y' command.
* sed/execute.c (execute_program): Free allocated but unused buffer in
'R' command when reaching EOF.
* sed/regexp.c (release_regex): Remove 'static', free the allocated
dfa struct.
* sed/utils.c (panic): Free open files linked-list elements.
|
|
|
|
|
|
| |
* cfg.mk (sc_prohibit-form-feed): Copied from coreutils' cfg.mk.
* sed/compile.c, sed/execute.c, sed/regexp.c, sed/sed.h, sed/utils.c:
Remove form feed characters.
|
|
|
|
|
|
| |
* cfg.mk (sc_space_before_open_paren): Copy target from coreutils' cfg.mk.
* basicdefs.h, sed/compile.c, sed/execute.c, sed/regexp.c,
sed/sed.c, sed/utils.c: Add space before parentheses.
|
|
|
|
|
|
|
|
|
|
| |
ck_malloc zero'd out the allocated buffer, thus replaced with
xcalloc/xzalloc (not xmalloc).
* basicdefs.h (MALLOC): Remove.
* utils.c (ck_malloc): Remove.
* sed/compile.c, sed/execute.c, sed/regexp.c, sed/sed.c: Replace
MALLOC with XCALLOC; ck_malloc with xzalloc.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sed would access invalid memory when matching EOF combined with
s///n flag:
$ yes 0 | fmt -w 40 | head -n2 | valgrind sed 'N;s/$//2m'
==13131== Conditional jump or move depends on uninitialised value(s)
==13131== at 0x4C3002B: memchr (vg_replace_strmem.c:883)
==13131== by 0x1120BD: match_regex (regexp.c:286)
==13131== by 0x110736: do_subst (execute.c:1101)
==13131== by 0x1115D3: execute_program (execute.c:1591)
==13131== by 0x111A4C: process_files (execute.c:1774)
==13131== by 0x112E1C: main (sed.c:405)
==13131==
==13131== Invalid read of size 1
==13131== at 0x4C30027: memchr (vg_replace_strmem.c:883)
==13131== by 0x1120BD: match_regex (regexp.c:286)
==13131== by 0x110736: do_subst (execute.c:1101)
==13131== by 0x1115D3: execute_program (execute.c:1591)
==13131== by 0x111A4C: process_files (execute.c:1774)
==13131== by 0x112E1C: main (sed.c:405)
==13131== Address 0x55ec765 is 0 bytes after a block of size 101 alloc'd
==13131== at 0x4C2DDCF: realloc (vg_replace_malloc.c:785)
==13131== by 0x113BA2: ck_realloc (utils.c:418)
==13131== by 0x10E682: resize_line (execute.c:154)
==13131== by 0x10E6F0: str_append (execute.c:165)
==13131== by 0x110779: do_subst (execute.c:1106)
==13131== by 0x1115D3: execute_program (execute.c:1591)
==13131== by 0x111A4C: process_files (execute.c:1774)
==13131== by 0x112E1C: main (sed.c:405)
==13131==
The ^/$ optimization code added in v4.2.2-161-g6dea75e called memchr()
using 'buflen', ignoring the value of 'buf_start_offset' (which, if not
zero, reduces the number of bytes available for the search).
Reported by bugs@feusi.co (bug#32271) in
https://lists.gnu.org/r/bug-sed/2018-07/msg00018.html .
* NEWS: Mention the fix.
* sed/regexp.c (match_regex): Use correct buffer length in memchr().
* testsuite/bug-32271-2.sh: Test using valgrind.
* testsuite/local.mk (T): Add new test.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Sed uses 'size_t' internally, but gnulib's re_search uses 'signed int'.
If the buffer is larger than INT_MAX, reject it (panic, exit code 4).
The test is marked 'very expensive' and skipped by default (due to
creating a 2GB input file). To run it, use:
make check SUBDIRS=. RUN_VERY_EXPENSIVE_TESTS=yes \
TESTS=testsuite/regex-max-int.sh
Reported by YushiOMOTE in https://bugs.gnu.org/30520 .
* NEWS: Mention this.
* Makefile.am (check-expensive, check-very-expensive): New targets.
* init.cfg (expensive_, very_expensive_): Copied from coreutils.
* sed/regexp.c (match_regex): Check buffer length and panic if too large.
* testsuite/regex-max-int.sh: New test.
* testsuite/local.mk (T): Add new test.
|
|
|
|
|
| |
* gnulib: Update to latest.
* all files: Run "make update-copyright".
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
in sed-4.2 and later, '--posix' option wrongly disabled the special meaning
of '+' even in ERE mode (--posix should only disable it in BRE mode,
since it is a GNU extension in BRE). Bug introduced in the original
implementation of --posix in v4.1a-5-gba68fb4.
Reported by Jordan Torbiak in https://bugs.gnu.org/26409 .
* NEWS: Mention bug fix.
* sed/regexp.c (compile_regex_1): In --posix mode, set RE_LIMITED_OPS
only in BRE mode.
* testsuite/posix-mode-ERE.sh: New test.
* testsuite/local.mk: Add new test.
|
|
|
|
|
| |
* gnulib: Update to latest.
* all files: Run "make update-copyright".
|
|
|
|
| |
* sed/regexp.c (release_regex) [DEBUG_LEAKS]: Modernize declaration.
|
|
|
|
|
|
|
|
|
| |
* sed/regexp.c (compile_regex_1): Remove #ifdef RE_ICASE guard,
since dfa.c uses it unconditionally. Suggested by Norihiro Tanaka.
Also remove the nearby "#ifdef RE_NO_SUB". Those macros are guaranteed
to be defined by virtue of configure-time tests that ensure we're using
either a new-enough native glibc (RE_NO_SUB was added in 2004), or the
included replacement.
|
|
|
|
|
|
|
|
| |
* sed/regexp.c (compile_regex_1): Do away with use of DFA_CASE_FOLD.
This gnulib update pulls in a dfa module in which that symbol no
longer exists. Instead, it uses RE_ICASE in syntax bits.
* m4/.gitignore: Add files created by running bootstrap.
* lib/.gitignore: Likewise.
|
|
|
|
|
|
| |
* cfg.mk (exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF):
Also exempt zero-anchor.{inp,good}.
* sed/regexp.c (match_regex): s/can not/cannot/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Sed incorrectly matches the anchor '^' to a newline character
(ASCII 10) despite using NUL line terminators (-z).
See https://bugs.gnu.org/24615 .
Previous (incorrect) behavior:
$ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a
X a nul b nul
With this fix:
$ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a
X a nul X b nul
Current regex engine can not handle NUL multiline anchors.
Implement such search manually (by breaking the pattern into separate
NUL-terminated lines) and searching each one individually.
* NEWS: Mention it.
* sed/regexp.c (compile_regex_1): Don't use newline_anchor of regex, if
the buffer delimiter is not newline character.
(match_regex): Do above case line-by-line.
* testsuite/zero-anchor.good, testsuite/zero-anchor.inp,
testsuite/zero-anchor.sed: New test.
* testsuite/local.mk: Add the test.
* testsuite/Makefile.tests: Add the test.
* bootstrap.conf: Add memrchr.
* lib/.gitignore, m4/.gitignore: Ignore memrchr files.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Optimize /^/ and /$/ patterns and avoid invoking the regular expression
engine if possible.
See https://bugs.gnu.org/24615 .
* sed/regex.c (compile_regex_1): Mark the patterns which consist of ^ or $.
(match_regex): Handle the patterns which consist of ^ or $ manually.
* sed/sed.h (struct regex): New members 'begline' and 'endline'.
* testsuite/newline-anchor.good, testsuite/newline-anchor.sed,
testsuite/newline-anchor.sed: New test.
* testsuite/Makefile.tests: Add the test.
* testsuite/local.mk: Add the test.
|
|
|
|
|
|
|
|
| |
* sed/regexp.c (compile_regex_1): Avoid building fastmap for case-
insensitive matching.
* testsuite/title-case.sh: New file, to test for this fix.
* testsuite/local.mk (T): Add the new file name and sort alphabetically.
* init.cfg (require_el_iso88597_locale_): Define.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* bootstrap.conf (gnulib_modules): Add dfa.
* sed/dfa.c: Remove file.
* sed/dfa.h: Likewise.
* sed/local.mk (sed_sed_SOURCES): Remove dfa.c.
(NOINST_HEADERS): Remove dfa.h.
* sed/regexp.c (compile_regex_1): Use new dfasyntax API.
* sed/sed.c (localeinfo): New global.
(main): Call init_localeinfo to initialize it.
* sed/sed.h: Include localeinfo.h and declare the new global.
* lib/.gitignore: Ignore new gnulib-imported files.
* m4/.gitignore: Likewise.
* po/POTFILES.in: s,sed/dfa.c,lib/dfa.c,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When possible, sed now uses grep's DFA matcher to match regular
expression, often resulting in a 10x speed-up.
* NEWS (Improvements): Mention it. Move this section to the top.
* sed/local.mk (sed_sed_SOURCES): Add dfa.c, and sort file names.
(noinst_HEADERS): Add dfa.h, and sort file names.
* sed/regexp.c (dfaerror, dfawarn): New functions.
(compile_regex_1): Compile pattern with dfa.
(match_regex): Use dfa.
* sed/sed.h: Include dfa.h.
(struct regex): New member dfa.
* sed/dfa.c: New file, copied from grep.
* sed/dfa.h: New file, copied from grep.
* po/POTFILES.in: Add sed/dfa.c.
|
|
|
|
|
|
|
|
| |
Run "make update-copyright" and then...
* gnulib: Update to latest.
* tests/init.sh: Update from gnulib.
* bootstrap: Likewise.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Convert many function definitions from K&R to ANSI style.
When possible, make a function static and remove the immediately-
preceding declaration of that same function.
Also, add the const to a few declarations.
* sed/compile.c: As above.
(special_files): Likewise.
* sed/execute.c: Likewise.
* sed/fmt.c: Likewise.
* sed/mbcs.c: Likewise.
* sed/regexp.c: Likewise.
* sed/sed.c: Likewise.
* sed/sed.h: Likewise.
* sed/utils.c: Likewise.
|
|
|
|
| |
* sed/regexp.c (END_ERRORS): Remove definition of unused macro.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* cfg.mk (local-checks-to-skip): Remove its exemption.
Exempt all files under testsuite/, for now, and a few others.
* .gitmodules: Change each leading TAB to 8 spaces spaces.
* BUGS: Likewise.
* NEWS: Likewise.
* configure.ac: Likewise.
* sed/compile.c: Likewise.
* sed/execute.c: Likewise.
* sed/fmt.c: Likewise.
* sed/regexp.c: Likewise.
* sed/sed.c: Likewise.
* sed/sed.h: Likewise.
* sed/utils.c: Likewise.
* sed/utils.h: Likewise.
* COPYING: Update from coreutils.
|
|
|
|
|
| |
* cfg.mk (local-checks-to-skip): Remove its exemption.
* sed/regexp.c (match_regex): Do not cast return value of alloca.
|
|
|
|
|
|
| |
* bootstrap.conf (gnulib_modules): Add update-copyright.
Then, run "make update-copyright".
* Makefile.am: Add copyright header.
|
|
|
|
|
|
|
|
|
|
|
| |
2012-03-16 Paolo Bonzini <bonzini@gnu.org>
* basicdefs.h (FREE): Remove.
* sed/compile.c: Adjust.
* sed/execute.c: Adjust.
* sed/regexp.c: Adjust.
* sed/utils.c: Adjust, remove ck_free.
* sed/utils.h: Adjust, remove ck_free.
|
|
|
|
|
|
|
|
|
|
|
| |
2012-03-16 Paolo Bonzini <bonzini@gnu.org>
* basicdefs.h (CAST, VCAST, VOID, MEMCPY, MEMMOVE): Rip out.
* sed/compile.c: Adjust.
* sed/execute.c: Adjust.
* sed/regexp.c: Adjust.
* sed/utils.c: Adjust.
* sed/utils.h: Adjust.
|
|
|
|
|
|
|
|
|
|
|
|
| |
2012-03-16 Paolo Bonzini <bonzini@gnu.org>
* basicdefs.h: Enjoy gnulib's headers.
* sed/compile.c: Likewise.
* sed/execute.c: Likewise.
* sed/regexp.c: Likewise.
* sed/sed.c: Likewise.
* sed/sed.h: Likewise.
* sed/utils.c: Likewise.
|
|
|
|
| |
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-94
|
|
|
|
|
|
|
|
|
|
| |
2006-12-21 Paolo Bonzini <bonzini@gnu.org>
* sed/mbcs.c: Include string.h (reported by Henning Nielsen Lund).
* sed/fmt.c: Likewise.
* sed/regexp.c: Likewise.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-89
|
|
|
|
|
|
|
|
| |
2006-12-15 Paolo Bonzini <bonzini@gnu.org>
* sed/regexp.c: Disable all extensions on --posix.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-84
|
|
|
|
|
|
|
|
|
| |
2006-08-21 Paolo Bonzini <bonzini@gnu.org>
* sed/regexp.c: Accept NUL bytes for `.'. Accept 'a\(b' in
POSIXLY_CORRECT/POSIXLY_BASIC posixicity.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-73
|
|
|
|
| |
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-50
|
|
|
|
| |
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-49
|
|
|
|
|
|
|
|
|
| |
2005-02-10 Paolo Bonzini <bonzini@gnu.org>
* sed/regexp.c: Fix off-by-one error in the "invalid reference
to subexpression" message. Debian bug 294339.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-42
|
|
|
|
|
|
|
|
| |
2005-01-25 Paolo Bonzini <bonzini@gnu.org>
* sed/regexp.c: Fix building on GCC 2.95 and earlier.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-35
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
2004-12-26 Paolo Bonzini <bonzini@gnu.org>
Do not use leftmost-longest matching for addresses.
* NEWS: Add a note about this.
* testsuite/recall2.good, testsuite/recall2.inp,
testsuite/recall2.sed: New test.
* testsuite/Makefile.am, testsuite/Makefile.tests: Add the
recall2 test.
* sed/sed.h (struct regex): New.
(struct addr, struct subst, compile_regex, match_regex,
release_regex): Use it instead of regex_t.
* sed/compile.c (compile_program): Update for new meaning of
the third parameter of compile_regex.
* sed/execute.c (do_subst): Pass less conservative value to
the regsize parameter of match_regex.
* sed/regexp.c (compile_regex_1): New, extracted out of
compile_regex. The third parameter, needed_sub, now includes
\0 (so 10 means that \0 .. \9 are needed). Pass RE_NO_SUB
if needed_sub is zero.
(compile_regex): Accept a struct regex instead of a regex_t.
Save the regular expression's text.
(match_regex): Accept a struct regex instead of a regex_t.
Recompile the pattern if it was compiled with RE_NO_SUB.
(release_regex): Accept a struct regex instead of a regex_t.
* doc/Makefile.am: Generate sed.texi correctly when
building outside srcdir.
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--patch-33
|
|
(automatically generated log message)
git-archimport-id: bonzini@gnu.org--2004b/sed--stable--4.1--base-0
|