| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Gnulib, Emacs, etc. we are changing ftp: and http: URLs to use
https:, to discourage man-in-the-middle attacks when downloading
software. The attached patch propagates these changes upstream to
Automake. This patch does not affect files that Automake is
downstream of, which I'll patch separately.
Althouth the resources are not secret, plain HTTP is vulnerable to
malicious routers that tamper with responses from GNU servers,
and this sort of thing is all too common when people in some other
countries browse US-based websites. See, for example:
Aceto G, Botta A, Pescapé A, Awan MF, Ahmad T, Qaisar
S. Analyzing internet censorship in Pakistan. RTSI
2016. https://dx.doi.org/10.1109/RTSI.2016.7740626
HTTPS is not a complete solution here, but it can be a significant
help. The GNU project regularly serves up code to users, so we should
take some care here.
|
|
|
|
|
|
|
|
| |
* all files: Run this command, using update-copyright from gnulib:
UPDATE_COPYRIGHT_FORCE=1 \
UPDATE_COPYRIGHT_USE_INTERVALS=2 \
UPDATE_COPYRIGHT_MAX_LINE_LENGTH=79 \
update-copyright $(git ls-files)
|
|
|
|
|
| |
Set time-stamp-time-zone to "UTC0", not to "UTC", as POSIX defines
TZ="UTC0" not TZ="UTC".
|
|
|
|
| |
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
|
|
|
| |
We've been in 2014 already for few months now...
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The exit status of a test should be reported in the test logs, so
that one can see at a glance whether the test has succeeded or failed,
without having to look also into the corresponding .trs file.
This fixes automake bug#11814.
* lib/test-driver: Also report the test script exit status in the
test log (as the last line).
* t/check-exit-status-reported.sh: Test this new behaviour.
* t/list-of-tests.mk: Add the new test.
* t/ax/test-lib.sh( am_exit_trap): No longer log the test exit status;
this has been made redundant by the change to 'test-driver'. While at
it, fix an imperfect quoting.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes automake bug#14840.
* lib/test-driver: Catch and report usage errors where the caller has
forgotten to specify one of the mandatory options (--test-name,
--log-file, --trs-file) or has not passed any non-option argument.
Also, be sure to work correctly even when no '--' special argument
is passed to separate option from non-options arguments.
* THANKS: Update.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
|
| |
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|\
| |
| |
| |
| |
| |
| |
| | |
* maint:
parallel-tests: "recheck" behaves better in case of compilation failures
scripts: quote 'like this', not `like this'
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
| |
| |
| |
| |
| | |
* lib/test-driver.sh: Here.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The old serial testsuite driver is still supported, and can be activated
by the 'serial-tests' option (introduced by commit 'v1.11-1989-gdeb7773'
of 2012-02-20, "automake: new option 'serial-tests'").
* lib/Automake/Options.pm: Now, the 'parallel-tests' option is a no-op
(but still explicitly recognized, for backward compatibility), while the
'serial-tests' option is recognized and registered.
* automake.in: Adjust to ensure that the parallel test harness is used by
default; mostly, this amounts at using "! option('serial-tests')" where
"option('parallel-tests')" was used before.
* NEWS, doc/automake.texi: Update.
* configure.ac (AM_INIT_AUTOMAKE): Drop 'parallel-tests' option, is
redundant now.
* defs: In the generated AM_INIT_AUTOMAKE call, use the 'serial-tests'
when the serial testsuite harness is to be used, and no option otherwise.
* lib/am/check.am: Use (static) conditional '%SERIAL_TESTS%' instead of
conditional '%PARALLEL_TESTS%'.
* lib/test-driver: Adjust heading comments.
* t/ax/extract-testsuite-summary.pl: Likewise.
* t/check-exported-srcdir.sh: Likewise.
* t/check-subst.sh: Likewise.
* t/java-compile-run-nested.sh: Likewise.
* t/parallel-tests-dry-run-1.sh: Likewise.
* t/parallel-tests-dry-run-2.sh: Likewise.
* t/parallel-tests-interrupt.sh: Likewise.
* t/parallel-tests-many.sh: Likewise.
* t/test-metadata-global-log.sh: Likewise.
* t/tests-environment-backcompat.sh: Likewise.
* m4/init.m4 (AM_INIT_AUTOMAKE): Adjust a comment.
* t/check8.sh: Likewise.
* t/check-tests-in-builddir.sh: Likewise.
* t/java-compile-run-flat.sh: Adjust a botched comment.
* t/ax/tap-summary-aux.sh: Remove explicit use of the 'parallel-tests'
option.
* t/ax/testsuite-summary-checks.sh: Likewise.
* t/aclocal-path-precedence.sh: Likewise.
* t/dist-aux-many-subdirs.sh: Likewise.
* t/parallel-tests-no-color-in-log.sh: Likewise.
* t/parallel-tests-reset-term.sh: Likewise.
* t/parallel-tests-subdir.sh: Likewise.
* t/check-no-test-driver.sh: Fix setup to adjust to the changed
semantics. Update heading comments.
* t/parallel-tests-driver-install.sh: Adjust to the new semantic.
* t/test-driver-is-distributed.sh: Simplified according to the
new semantics.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
| |
* lib/tap-driver.pl ($USAGE): The `--trs-file' option is mandatory
too.
* lib/tap-driver.sh (print_usage): Likewise.
* lib/test-driver (print_usage): Likewise.
($scriptversion): Update.
|
|
|
|
|
|
| |
* lib/test-driver: Add comment to the code initializing ANSI color
escapes, telling to keep it in sync with the similar initialization
in `lib/am/check.am'.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lib/tap-driver (yn): New subroutine, converts a boolean value to
a "yes" or "no" string.
(write_test_results): Use it, and related reformatting.
* lib/test-driver (trap): Also remove the `.trs' file on signals,
for extra safety.
(fatal): Remove this function, it's never used.
($logfile, $trsfile): Renamed ...
($log_file, $trs_file): ... to these, for clarity, and in order to
be more consistent with the `tap-driver' script.
Improve a couple of comments.
* automake.in (handle_tests): Don't define anymore the now-obsolete
make macro `$(TEST_LOGS_TMP)', nor add it to the list of files to
be removed upon "make mostlyclean".
* lib/am/check.am ($(TEST_SUITE_LOG)): New shell function `f_ok',
tells whether a path refers to an existing, regular, readable file.
Use it throughout.
(recheck): Be safer w.r.t. make implementation that run recipes
with `errexit' shell flag active.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lib/am/check.am ($(TEST_SUITE_LOG)): When producing the
global test log, take into account the new metadata field
`:global-test-result:, to write nicely formatted RST section
titles, instead of leaving this chore to the individual test
drivers.
(am_rst_section): Re-introduce this variable, as removed in
commit `v1.11-870-ga27c9c4'.
* lib/test-driver, lib/tap-driver: Write the global test result
as metadata in the `.trs' file, not as (part of) a formatted RST
subsection title in the `.log' file. Related simplifications;
in particular, get rid of temporary files usage.
* doc/automake.texi (Log files generation and test results
recording): Document the new metadata.
* tests/check12.test: Update.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/parallel-tests-log-override-2.test: Likewise.
* tests/parallel-tests-log-override-recheck.test: Likewise.
* tests/parallel-tests-unreadable.test: Likewise.
* tests/tap-global-result.test: Likewise.
* tests/test-metadata-results.test: Likewise.
* tests/test-log.test: Likewise, and another minor unrelated fix.
* tests/test-metadata-global-result.test: New test.
* tests/Makefile.am (TESTS): Update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this change, the test harness will keep test metadata in
dedicated `.trs' files, instead of having them embedded into the
`.log' files. This allows for easier forward-compatibility and
extension of test metadata, and for more flexibility in the
format of the `.log' files. Note that this change makes the
`:end-metadata:' field obsolete.
* doc/automake.texi (Parallel Test Harness, Log files generation
and test results recording): Document the new APIs and behaviour;
some related minor rewordings and fixlets.
* NEWS: Update.
* automake.in (handle_tests): When bringing in the content of
`check2.am', substitute %BASE% with the basename of the `.log'
file being created by a rule. Add the generated `.trs' files
to the list of files to be cleaned by "make mostlyclean".
* lib/am/check.am (am__test_driver_flags): Rename ...
(am__common_driver_flags): ... to this, and remove the flags
`--test-name' and `--log-file' from it: they are now define in
the proper rules in `check2.am'.
(am__TEST_BASES): New internal variable, holding the names of
the tests, with any registered extension removed.
(am__stealth_MAKE): New internal variable, can be used instead of
$(MAKE) in recipes requiring a recursive call to make, but which
are not intended to be executed by "make -n".
(.log.trs): New suffix rule, to recover from deletion of `.trs'
files.
($(TEST_SUITE_LOG)): Almost completely rewritten to follow the
new API of "test logs in `.log' files, test metadata in `.trs'
files". It goes to some length to work correctly in face of
unreadable or missing `.log' and `.trs' files, and to error out
with proper error messages when this is not possible.
[%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale"
`.trs' files (in addition to `.log files) before remaking the
$(TEST_SUITE_LOG).
(recheck, recheck-html): Look for the `:recheck:' field in the
`.trs' files, not in the `.log' files.
* lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust
the call to the test driver, in particularly passing the new
option `--trs-file'.
[%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise.
* lib/tap-driver ($USAGE): Adjust the help screen.
(Getopt::Long::GetOptions): Handle the `--trs-file' option,
through the use of ...
($trs_file): ... this new global variable.
(finish): Write metadata for the test run to `$trs_file' rather
then to `$log_file', through the use of ...
(write_test_results): ... this new function.
* lib/test-driver (print_usage): Update the help screen.
(Option parsing): Handle the `--trs-file' option, through the
use of ...
($resfile): ... this new global variable.
(Main code): Write metadata for the test run to `$trsfile' rather
than to `$logfile'.
Minor related adjustments to comments.
* tests/.gitignore: Ignore `*.trs' files.
* tests/parallel-tests-unreadable-log.test: Moved ...
* tests/parallel-tests-unreadable.test: ... to this, and extended
to also check the semantics for unreadable `.trs' files.
* tests/test-driver-end-metadata.test: Deleted as obsolete.
* tests/test-driver-metadata-no-leading-space.test: Likewise.
* tests/test-driver-global-log.test: Renamed ...
* tests/test-metadata-global-log.test: ... to this, and modified
as to verify the new APIs and semantics.
* tests/test-driver-recheck: Renamed ...
* tests/test-metadata-recheck.test: ... to this, and modified
likewise.
* tests/parallel-tests-once.test: New test.
* tests/parallel-tests-make-n.test: Likewise.
* test-metadata-results.test: Likewise.
* test-missing.test: Likewise.
* test-missing2.test: Likewise.
* test-trs-basic.test: Likewise.
* test-trs-recover.test: Likewise.
* test-trs-recover2.test: Likewise.
* tests/Makefile.am (TESTS): Update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change reworks and improves the parallel test harness to use
more specialized reStructuredText fields in the log files (instead
of relying on specially-placed of "magic lines" and more indirect
semantical formatting); the new fields are the following:
- ":recheck:": tell whether the associated test will have to be
re-run by "make recheck";
- ":copy-in-global-log:": tell whether the content of the log
file should be copied in the "global log" `test-suite.log';
- ":end-metadata:", which inhibits the scanning of the rest of
the log file (for what concerns test metadata).
Also, the special `:test-result:' value "END" has been removed,
superseded by the new `:end-metadata:' field.
* doc/automake.texi (Log files generation and test results
recording): Document the new API and semantics. Remove or fix
some obsolete comments.
* lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html):
Adjust comments and code.
* lib/tap-driver (finish): Adjust, with the help of ...
(must_recheck, copy_in_global_log): ... these new functions.
* lib/test-driver (Main code): Adjust, with the help of ...
($recheck, $gcopy): ... these new variables.
* tests/trivial-test-driver: Update to obey the new APIs.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/tap-passthrough.test: Relax the test, avoiding to check
what is written in `test-suite.log'; such check has been moved ...
* tests/tap-global-log.test: ... in this new test, and extended.
* tests/test-driver-metadata-no-leading-space.test: New test.
* tests/test-driver-end-test-results.test: Removed, it checked
the old APIs; superseded by ...
* tests/test-driver-end-metadata.test: ... this new test.
* tests/tap-log.test: Improve syncing with ...
* tests/test-log.test: ... this new test.
* tests/parallel-tests.test: Remove some duplication w.r.t. this
last new test. Updated heading comments.
* tests/Makefile.am (TESTS): Update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
`ERROR'. Use it when encountering unreadable test logs (previously
a simple `FAIL' was used in this situations).
* lib/test-driver: Set the global test result to `ERROR' when the
test exit status is 99. When doing colorized output, color `ERROR'
results in magenta.
* doc/automake.texi (Log files generation and test results
recording): Update by also listing `ERROR' among the list of valid
`:test-results:' arguments.
* NEWS: Update.
* tests/trivial-test-driver: Update.
* tests/parallel-tests.test: Likewise.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-no-spurious-summary.test: Likewise.
* tests/test-driver-global-log.test: Likewise.
* tests/test-driver-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-no-html.test: Likewise.
* tests/test-driver-end-test-results.test: Likewise.
* tests/color.test: Likewise. Also, make stricter, and also test
from VPATH.
* tests/color2.test: Likewise, and improve syncing with color.test.
* tests/parallel-tests-exit-statuses.test: New test.
* tests/parallel-tests-console-output.test: Likewise.
* tests/Makefile.am (TESTS): Update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new code for parsing the testsuite-generated `.log' files,
as introduced in commit `v1.11-872-gc96b881', considers each
`:test-result:' field anywhere in a `.log' file as a declaration
of a test result, and accounts for it as such in the testsuite
summary. Unfortunately this could easily cause spurious test
failures being reported in the testsuite summary. This happened
in practice with the Automake's own testsuite; for example:
$ make check TESTS='check12-p.test'; echo exit: $?
...
PASS: check12-p.test
=====================================
4 of 5 tests failed
See tests/test-suite.log
Please report to bug-automake@gnu.org
=====================================
make[2]: *** [test-suite.log] Error 1
make: *** [check-am] Error 2
exit: 2
This change introduces a new special `:test-result:' "END", that,
when seen, prevents the rest of the log file from being parsed.
For more information, refer to the thread:
<http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
* lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
file as soon as the special ":test-result:END" directive is seen.
Related changes and enhancements.
* lib/test-driver: Protect the rest of the log after the result
lined with a ":test-result:END" directive.
* tests/parallel-tests-no-spurious-summary.test: New test.
* tests/test-driver-end-test-results.test: Likewise.
* tests/Makefile.am (TESTS): Update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this change, we improve the code creating the `test-suite.log'
global log and the console testsuite summary to make it able to
grasp multiple results per test script. This is required in order
to introduce the planned support for test protocols, like TAP and
SubUnit, which can indeed run multiple tests per test script, each
with its individual result.
The implementation makes use of a custom reStructuredText field
`:test-result:'.
Note that no new documentation is added by this change; that is
be left for follow-up changes.
* lib/check.am ($(TEST_SUITE_LOG)): When processing .log files,
recognize a report of a test's result only if it is declared with
the custom `:test-result:' reStructuredText field placed at the
beginning of a line. Extend and add explanatory comments.
(recheck, recheck-html): Add explanatory comments.
* lib/test-driver: Write an appropriate reStructuredText field
`:test-result:' in the generated log file. Use a reStructuredText
transition to better separate the test outcome report from the
test script's registered output. Improve comments.
* tests/test-driver-custom-xfail-tests.test: Adjust.
* tests/parallel-tests7.test: Adjust.
* tests/parallel-tests-empty-testlogs.test: New test.
* tests/parallel-tests-recheck-override.test: Likewise.
* tests/parallel-tests2.test: Extend and keep more in-sync with ...
* tests/test-driver-custom-html.test: ... this new related test.
* tests/test-driver-custom-no-html.test: New test.
* tests/test-driver-custom-multitest.test: Likewise.
* tests/test-driver-custom-multitest-recheck.test: Likewise.
* tests/test-driver-custom-multitest-recheck2.test: Likewise.
* tests/trivial-test-driver: New file, used by the last four tests
above.
* tests/Makefile.am (TESTS): Update.
(EXTRA_DIST): Distribute `trivial-test-driver'.
(test-driver-custom-multitest.log): Depend on `trivial-test-driver'.
(test-driver-custom-multitest-recheck.log): Likewise.
(test-driver-custom-multitest-recheck2.log): Likewise.
(test-driver-custom-html.log): Likewise.
|
|
This refactoring should cause no API of functionality change,
and is meant only to simplify the future implementation of TAP
and SubUnit testsuite drivers. More precisely, our roadmap is
to move most of the "testsuite driving" features out of the
Automake-generated Makefiles, and into external scripts with
well-defined interfaces. This will allow the user to define
its own personalized testsuite drivers, and will also offer us
a framework upon which to implement our new TAP and SubUnit
drivers, all in a very unobtrusive way and retaining an high
degree of code reuse and backward-compatibility.
* lib/test-driver: New auxiliary script.
* lib/Makefile.am (dist_SCRIPT_DATA): Add it.
* automake.in (handle_tests): Require the new auxiliary script
`test-driver', and define a new internal makefile variable
`$(am__test_driver)', used to call it. Perform new substitution
on `DRIVER' when processing the `check2.am' file.
* lib/check.am (am__tty_colors): Define new shell variable
`$am__color_tests'.
(am__rst_section): Removed, its role taken over by the new
`test-driver' script.
(am__test_driver_flags): New variable, contains the command
line options passed to `test-driver'.
(am__check_pre): Do not deal with temporary files and exit
traps anymore, as the `test-driver' script takes care of that
now. Define shell variable `$am__enable_hard_errors', used by
`$(am__test_driver_flags)'. Reorder so that we don't need to
save and restore the value of the `TERM' environment variable
anymore.
Other related adjustments.
(am__check_post): Remove, as its role has been completely taken
over by the `test-driver' script.
* am/check2.am (?GENERIC?%EXT%$(EXEEXT).log, ?GENERIC?%EXT%.log,
?!GENERIC?%OBJ%): Call the test script through the Automake
substituted `%DRIVER%', and honor the command-line options
in `$(am__test_driver_flags)'. Do not call the obsoleted
`$(am__check_post)' anymore.
* doc/automake.texi (Auxiliary Programs): Mention the new
`test-driver' script.
(Optional): Mention `test-driver' in AC_CONFIG_AUX_DIR.
Since we are at it, break the list of auxiliary scripts by
placing one per line, to simplify potential future additions
of new scripts.
* tests/check.test: Adjust.
* tests/check2.test : Likewise.
* tests/check3.test : Likewise.
* tests/check4.test : Likewise.
* tests/check10.test: Likewise.
* tests/color.test: Likewise.
* tests/color2.test: Likewise.
* tests/comment9.test: Likewise.
* tests/dejagnu.test: Likewise.
* tests/exeext4.test: Likewise.
* tests/maken3.test: Likewise.
* tests/maken4.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/posixsubst-tests.test: Likewise.
* tests/repeated-options.test: Likewise.
* tests/check-no-test-driver.test: New test.
* tests/parallel-test-driver-install.test: Likewise.
* tests/Makefile.am (TESTS): Update.
* NEWS: Update.
|