diff options
| -rw-r--r-- | README-hacking.md | 38 | ||||
| -rw-r--r-- | examples/local.mk | 3 | ||||
| -rw-r--r-- | tests/README.md | 9 | ||||
| -rw-r--r-- | tests/local.mk | 3 |
4 files changed, 29 insertions, 24 deletions
diff --git a/README-hacking.md b/README-hacking.md index 61c3a47c..a50e18b2 100644 --- a/README-hacking.md +++ b/README-hacking.md @@ -388,6 +388,11 @@ In examples/, there is a number of ready-to-use examples (see suites run by `make check`. The test results are in local `*.log` files (e.g., `$build/examples/c/calc/calc.log`). +To check only the examples, run `make check-examples`. To check just one +example, + + make check-examples TESTS='examples/c/bistromathic/bistromathic.test' + ### The Main Test Suite The main test suite, in tests/, is written on top of GNU Autotest, which is part of Autoconf. Run `info autoconf 'Using Autotest'` to read the @@ -401,28 +406,23 @@ contains lots of details that should help diagnosing issues, including build issues. The per-test logs are more convenient when working locally. #### TESTSUITEFLAGS -To run just the main test suite, run `make check-local`. +To run just the main test suite, run `make check-tests`. -The default is for make check-local to run all tests sequentially. This can -be very time consuming when checking repeatedly or on slower setups. This -can be sped up in two ways: +The default is for `make check-tests` to run all tests sequentially. This +can be very time consuming when checking repeatedly or on slower setups. +This can be sped up in two ways. -Using -j, in a make-like fashion, for example: +1. Using -j, in a make-like fashion, for example: - $ make check-local TESTSUITEFLAGS='-j8' + $ make check-tests TESTSUITEFLAGS='-j8' -Actually, when using GNU Make, TESTSUITEFLAGS defaults to the -jN passed to -it, so you may simply run +When using GNU Make, TESTSUITEFLAGS defaults to the -jN passed to it, so you +may simply run - $ make check-local -j8 + $ make check-tests -j8 -Running only the tests of a certain category, as specified in the AT files -with AT_KEYWORDS([[category]]). Categories include: -- c++, for c++ parsers -- deprec, for tests concerning deprecated constructs. -- glr, for glr parsers -- java, for java parsers -- report, for automaton dumps +2. Running only the tests of a certain category. See +[tests/README.md](tests/README.md#keywords) for the list of categories. To get a list of all the tests (and their keywords for -k), run @@ -430,11 +430,11 @@ To get a list of all the tests (and their keywords for -k), run To run a specific set of tests, use -k (for "keyword"). For example: - $ make check-local TESTSUITEFLAGS='-k c++' + $ make check-tests TESTSUITEFLAGS='-k c++' Both can be combined. - $ make check-local TESTSUITEFLAGS='-j8 -k c++' + $ make check-tests TESTSUITEFLAGS='-j8 -k c++' To rerun the tests that failed: @@ -488,7 +488,7 @@ useful. Here's one way to set them up with GCC 10 on Mac Ports 4. Run the tests with leak detection enabled (`ASAN_OPTIONS=detect_leaks=1`). E.g. for counterexamples: ``` - $ make check-local TESTSUITEFLAGS='-j5 -k cex' ASAN_OPTIONS=detect_leaks=1 + $ make check-tests TESTSUITEFLAGS='-j5 -k cex' ASAN_OPTIONS=detect_leaks=1 ``` 5. You might need a suppression file. See diff --git a/examples/local.mk b/examples/local.mk index 09e1747b..318df0dc 100644 --- a/examples/local.mk +++ b/examples/local.mk @@ -93,6 +93,9 @@ dist_examples_DATA = %D%/README.md CLEANDIRS += %D%/*.dSYM +.PHONY: check-examples +check-examples: check-TESTS + include %D%/c/local.mk include %D%/c++/local.mk include %D%/d/local.mk diff --git a/tests/README.md b/tests/README.md index 53d1a2bf..aa409a54 100644 --- a/tests/README.md +++ b/tests/README.md @@ -7,11 +7,12 @@ define a number of other macros to: - what skeleton is used # Keywords -The following keywords are used to label tests. Please follow them -and avoid creating synonyms. +The following keywords are used to label tests (with +AT_KEYWORDS([[category]])). Please follow them and avoid creating +synonyms. - action - api.value.type -- c++ +- c++: c++ language - cex: counterexamples. - d: d language - deprec: deprecated features. @@ -23,7 +24,7 @@ and avoid creating synonyms. - %merge - multistart - push -- report +- report: for automaton dumps - %union - variant diff --git a/tests/local.mk b/tests/local.mk index 37c542ad..9896d912 100644 --- a/tests/local.mk +++ b/tests/local.mk @@ -114,7 +114,8 @@ recheck: $(RUN_TESTSUITE_deps) -e 'eof && /^(\d+).*: FAILED/ && print "$$1 "' \ %D%/testsuite.dir/*/testsuite.log) -check-local: $(RUN_TESTSUITE_deps) +.PHONY: check-tests +check-local check-tests: $(RUN_TESTSUITE_deps) $(RUN_TESTSUITE) # Run the test suite on the *installed* tree. |