summaryrefslogtreecommitdiff
path: root/t/test-driver-fail.sh
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'master' into ng/masterStefano Lattarini2012-06-301-2/+2
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * master: texi: clean after Texinfo manuals in $(SUBDIRS) directories correctly coverage: regression in texinfo "make mostlyclean" with $(SUBDIRS) tests: fix a spurious failure configure: fix detection of POSIX shell to work in a VPATH build tests: prefer "test ! -e FILE" to check that a file doesn't exist configure: move a misplaced "section" comment configure: clump check for ${var%...} and ${var#...} expansion together cosmetics: improve wording of a couple of configure messages configure: the testsuite shell must support 'test -e' and 'test ! -e' tests: remove stale workarounds for Solaris /bin/sh Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
| * tests: prefer "test ! -e FILE" to check that a file doesn't existStefano Lattarini2012-06-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Once, for the sake of (at least) Solaris 10 /bin/sh, we had to use "test ! -f FILE" or "test ! -r FILE" or "test ! -d FILE" instead, because the that shell's 'test' built-in didn't grok the '-e' option. Note however that we still can't use "test ! -e" in the Makefile recipes used in the test cases; that is because those recipes are run with the shell detected by 'configure', and Autoconf-generated configure scripts do no guarantee to find or provide a POSIX-compatible shell. * Several tests: Adjust. * t/yacc-clean-cxx: Adjust, and remove a couple of useless commands. * t/parallel-tests-dry-run-2.sh: Adjust, and add invocation to "make -n" forgotten in previous versions of the test. * t/txinfo26.sh: Adjust, and don't bother to skip the test when it's run in a directory whose absolute path contain whitespace: that setup is not supported anyway. * t/maken3.sh: Adjust, and fix a typo that could cause a minor false negative. * t/test-trs-recover2.sh: Prefer using 'skip_' with a suitable error message over a bare 'exit 77'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
* | Merge branch 'master' into ng/masterStefano Lattarini2012-06-281-3/+3
|\ \ | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * master: cosmetics: remove trailing whitespace in some files coverage: enhance tests in 'missing' script a little coverage: tests AM_MISSING_PROG usage docs: document AM_MISSING_PROG tests: fix a spurious failure with Solaris make tests: simpler workaround for shells losing the exit status in exit trap tests: avoid several spurious failures on Solaris tests: avoid a spurious failure on NetBSD + Extra non-trivial edits: * Several tests: Adjusted to use 'exit' rather than 'Exit'. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
| * tests: simpler workaround for shells losing the exit status in exit trapStefano Lattarini2012-06-251-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we can assume our tests are run by a decent POSIX shell, we can simplify our workaround aimed at having the exit status propagated correctly to the code in the exit trap. Unfortunately, we cannot dispense with such a workaround altogether, because it's still required by some shells we need to support (at least Solaris 10 /bin/ksh and /usr/xpg4/bin/sh). For more information about the need of that workaround, see the entry about 'trap' in the section "Limitations of Shell Builtins" in the Autoconf manual: <http://www.gnu.org/software/autoconf/manual/autoconf.html#trap> The new workaround has been tested successfully with the following shells: - Bash 4.1 - Bash 3.2 - Bash 3.0 - Bash 2.05b - dash 0.5.5.1 - dash 0.5.2 - AT&T Ksh 93u (from official Debian package) - MirBSD Korn Shell 40.2 (from official Debian package) - Solaris 9, 10 and 11 /bin/ksh - Solaris 9, 10 and 11 /usr/xpg4/bin/sh - NetBSD 5.1 /bin/sh - NetBSD 5.1 /bin/ksh * t/ax/test-init.sh (Exit): Rename ... (_am_exit): ... like this. (exit): New alias for '_am_exit'. We cannot simply redefine 'exit' as a shell function, because some shells (dash 0.5.5.1, Solaris 10 /bin/ksh and /usr/xpg4/bin/sh) do not allow it. (_am_exit, trap): Add extra escaping for 'exit' calls, to ensure we really invoke the 'exit' builtin and not our alias with the same name. * configure.ac: Check that the shell selected to run our testsuite supports aliases named like shell builtins. * t/REAMDE: Adjust. * All tests: Adjust, by simply using 'exit' instead of 'Exit'. * t/self-check-explicit-skips.sh: Adjust: the first usage of 'exit' after it has been redefined as an alias must be on a new line w.r.t. that where the alias is defined, in order for the redefinition to be honored. * syntax-checks.mk (sc_tests_Exit_not_exit): Delete. (sc_tests_exit_not_Exit): New. (syntax_check_rules): Adjust. (sc_tests_automake_fails): Simplify the recipe a little. * Several tests: Remove now useless spurious quoting once required to placate the 'sc_tests_Exit_not_exit' maintainer check. * gen-testsuite-part: Likewise. Also, avoid uses of 'Exit' in the generated scripts. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
* | [ng] check: unconditionally distribute test-driverStefano Lattarini2012-05-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This new behaviour is suboptimal, but will allow us to move yet more logic and processing from the automake scripts into the generated Makefiles. * automake.in (handle_tests): Distribute 'test-driver' unconditionally if the variable TESTS is defined and the 'serial-tests' option is not used. (handle_per_suffix_test): Don't distribute 'test-driver' anymore. * Several tests: Adjust. * t/test-driver-custom-no-extra-driver.sh: Remove as obsolete. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
* | Merge branch 'master' into ng/masterStefano Lattarini2012-05-181-1/+0
|\ \ | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * master: tests: use 'parallel-tests' Automake option by default doc: fix location of Automake's own test-suite.log file yacc tests: avoid a spurious failure with Sun C++ 5.9 self tests: cater for /bin/ksh symlinked to Zsh tests: fix spurious failures due to dpkg install-info + Non-trivial extra edits: * t/check7.sh: look at the contents of '$am_serial_tests', not of '$am_parallel_tests', to decide which kind of testsuite harness is being used. * t/parallel-tests-concatenated-suffix.sh: Don't explicitly set '$am_parallel_tests' to "yes" anymore, is not required and would cause a 'maintainer-check' failure. * t/parallel-tests-dynamic.sh: Likewise. * t/parallel-tests-longest-stem.sh: Likewise. * t/parallel-tests-no-repeat.sh: Likewise. * t/parallel-tests-per-suffix-deps.sh: Likewise. * t/parallel-tests-srcdir-in-test-name.sh: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
| * tests: use 'parallel-tests' Automake option by defaultStefano Lattarini2012-05-181-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will help our transition from 'serial-tests' to 'parallel-tests' as the default test suite driver enabled by a TESTS assignment in the input Makefile.am. Note that that change of default will only take place in master, though. * defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE invocation in the created 'configure.ac' stub, unless the variable 'am_serial_tests' is set to "yes". Don't pay attention anymore to the 'am_parallel_tests' variable, that's obsolete now. * defs-static.in: Warn if the 'am_serial_tests' variable is set in the environment; conversely, don't warn anymore about 'am_parallel_tests' being set in the environment. * Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests' variable instead of the now-obsolete 'am_parallel_tests' one. * syntax-checks.mk (sc_tests_obsolete_variables): Also warn against uses of 'am_parallel_tests', which is now deprecated in favor of 'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen, suggest using 'am_serial_tests' instead, not 'am_parallel_tests'. * gen-testsuite-part: Now that we use the 'parallel-tests' by default in our tests, we need to completely change the logic and semantics of generation of sibling tests for those tests that check the Automake generated testsuite harness itself. Do that, and give a complete explanation of the new logic and semantics in the relevant comments. * t/README: Update. * Lots of test cases: Adjust. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
* | [ng] parallel-tests: simplify automake-time preprocessingStefano Lattarini2012-04-231-3/+5
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our new pattern rules, introduced in recent commit 'v1.11b-124-gaeaba3f', are smart enough to allow us to ditch most of our cumbersome automake-time analysis and rewriting of the $(TESTS) variable when the parallel-tests driver is used. This also makes other related simplifications possible. This change causes a little regression in that we won't be able anymore to diagnose at automake time test cases that start with the '$(srcdir)' or '$(top_srcdir)' component, but that's acceptable, since a later commit will make the former case actually work, and the second case isn't important enough to worry to much about. This change also causes a small behavioural change in automake, in that it will also need the 'LOG_DRIVER' variable to be defined in each Makefile.am defining the 'TESTS' variable; and if it's not defined, automake will define it itself, and require the presence of the 'test-driver' auxiliary script in the process (or bing it in if the "--add-missing" option is in use). See the changes to 'NG-NEWS' for more details, and for workarounds. * automake.in (handle_tests): Don't walk nor rewrite $(TESTS) when the 'parallel-tests' option is active. Other related minor simplifications. * t/exeext4.sh: Define '$am_parallel_tests' to "no", so that the test won't be run with the 'parallel-tests' option enabled (as the semantics in that case have been changed in a way that would make this test fail). * t/check7.sh: When '$am_parallel_tests' is "yes", skip checks that don't work anymore with the 'parallel-tests' option is enabled. * t/test-driver-custom-no-extra-driver.sh: Adjust to avoid spurious failures to to the "LOG_DRIVER must be defined somehow" semantic change. * t/tap-common-setup.sh: Likewise. * t/tap-recheck.sh: Likewise. * t/tap-basic.sh: Likewise. * t/tap-diagnostic-custom.sh: Likewise. * t/tap-more.sh: Likewise. * t/tap-recheck.sh: Likewise. * t/tap-signal.tap: Likewise. * t/test-driver-custom-multitest.sh: Likewise. * t/test-driver-custom-multitest-recheck.sh: Likewise. * t/test-driver-custom-multitest-recheck2.sh: Likewise. * t/test-driver-custom-xfail-tests.sh: Likewise. * t/test-driver-fail.sh: Likewise. * t/test-driver-strip-vpath.sh: Likewise. * t/test-metadata-global-log.sh: Likewise. * t/test-metadata-global-results.sh: Likewise, and enhanced a little since we are at it. * t/test-metadata-results.sh: Likewise. * t/tap-bad-prog.tap: Likewise. * t/parallel-tests8.sh: Move the now-failing checks verifying that automake diagnoses test cases that start with the '$(srcdir)' or '$(top_srcdir)'... * t/parallel-tests-srcdir-in-test-name.sh: ... this new xfailing test. * t/spy-pattern-rules.sh: New "spy" test, verify our new assumptions on the precedence of pattern rules truly hold. * t/parallel-tests-longest-stem.sh: New test. * t/list-of-tests.mk: Update. * doc/automake.texi: Document that $(TESTS) is rewritten for $(EXEEXT) appending only when the 'parallel-tests' option is *not* in use. Add a "FIXME" comment about the fact that automake does not diagnose test cases that start with the '$(srcdir)' or '$(top_srcdir)' anymore. * NG-NEWS: Update. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
* tests: rename 'tests/' => 't/', '*.test' => '*.sh'Stefano Lattarini2012-04-061-0/+65
When we (soon) convert the Automake testsuite to a non-recursive make setup, we'll have to fix the entries of $(TESTS) to be prepended with the subdirectory they are in; this will increase the length of $(TESTS), and thus increase the possibility of exceeding the command-line length limits on some systems (most notably, MinGW/MSYS). See automake bug#7868 for more information. Thus we rename the 'tests/' subdirectory to 't/', and each 'x.test' script in there to 'x.sh'; this way, the $(TESTS) entry 'foo.test' will become 't/foo.sh', which have the same number of characters. * tests/: Rename ... * t/: ... to this. * t/*.test: Rename ... * t/*.sh: ... to this. * t/.gitignore: Removed as obsolete. * t/defs: Adjust. * t/gen-testsuite-part: Likewise. * t/list-of-tests.mk: Likewise. * t/ccnoco.sh: Likewise. * t/ccnoco3.sh: Likewise. * t/self-check-cleanup.tap: Likewise. * t/self-check-dir.tap: Likewise. * t/self-check-me.tap: Likewise. * t/self-check-reexec.tap: Likewise. * README: Likewise. * bootstrap: Likewise * configure.ac: Likewise. * Makefile.am: Likewise. * .gitignore: Likewise. * syntax-check.mk: Likewise. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>