summaryrefslogtreecommitdiff
path: root/testsuite/driver/testlib.py
Commit message (Collapse)AuthorAgeFilesLines
* Replace deprecated string.join(x,y) with y.join(x)Krzysztof Gogolewski2014-09-211-10/+9
|
* Replace several Python 2 only idioms with Python 2/3 compatible versionsKrzysztof Gogolewski2014-09-211-52/+51
| | | | Mostly done by manually checking 2to3 output
* Improve testsuite driver compatibility with Python 3 (Trac #9184)Krzysztof Gogolewski2014-09-211-43/+44
| | | | Thanks to aspidites <emarshall85@gmail.com> who provided the patch.
* testsuite: normalise integer library name for T8958Sergei Trofimovich2014-08-301-4/+2
| | | | Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
* testsuite: add signal_exit_code function to the driverKarel Gardas2014-08-071-0/+11
| | | | | | | | | | | | | | | | | | | Summary: New function signal_exit_code hides differences between target platforms handling of fatal error signals and the applications' exit codes. E.g. on Linux the application exit code which receives fatal error signal is encoded as 128 + signal value. On the other hand on Solaris the application exit code is signal value alone. Test Plan: validated on Linux and tested on Solaris Reviewers: austin, simonmar Reviewed By: simonmar Subscribers: phaskell, simonmar, relrod, ezyang, carter Differential Revision: https://phabricator.haskell.org/D122
* Correctly round when calculating the deviationJoachim Breitner2014-07-171-1/+1
|
* Include test case name in performance resultJoachim Breitner2014-07-171-9/+11
| | | | | | | With THREADS=n, for n > 1, it becomes impossible to match the performance numbers to the test case name. Hence include it in the output. This also makes grepping through a bunch of logs for a specific test case much easier, and outweighs the extra verbosity.
* Give performance benchmark deviation also in percentsJoachim Breitner2014-07-141-2/+6
| | | | | this makes it easier to spot a “just over the mark” change (e.g. +5.1%), compared to a more radical jump (e.g. +15%).
* New testsuite verbosity level 4Joachim Breitner2014-07-141-1/+1
| | | | | which makes it print performance numbers even when the test succeeds (good for historic analysis)
* Remove external coreAustin Seipp2014-05-031-95/+0
| | | | Signed-off-by: Austin Seipp <austin@well-typed.com>
* Fix #5435, adding new test config check_stdout.Edward Z. Yang2014-04-091-1/+15
| | | | | | | | | | | check_stdout(f) allows you to override the test framework's diff based output checking with another mechanism. f is a function which takes two arguments: the first is the filename containing the observed stdout, the second is the normaliser that would have been applied (in case you want to read, normalise, and then do something.) Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
* Fix some Python brainos in testlib (except e is not valid form).Edward Z. Yang2014-02-061-2/+2
| | | | Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>
* Explain skipping compiler-perf tests when debugging is onJoachim Breitner2013-11-281-0/+2
|
* Summarily skip compiler_perf tests when DEBUG is onJoachim Breitner2013-11-281-0/+3
| | | | by ensuring this in the test runner.
* Flag to test suite: SKIP_PERF_TESTSJoachim Breitner2013-10-051-0/+6
| | | | | | More often than not the output of the performance tests is in the way, rather than helping. This allows the use of `make SKIP_PERF_TESTS=YES` to skip these tests. Fixes #8413
* Driver: Fix % operator typoJoachim Breitner2013-09-231-1/+1
|
* Test driver: Implement different verbositiesJoachim Breitner2013-09-201-18/+20
| | | | | | | | | Select verbosity with "make VERBOSE=n". Options so far: n=0: No per-test output n=1: Only failing test results n=2: As above, plus progress information (names of all tests) n=3: As aobve, plus commands called. Default currently is n=3, although n=2 might be a nicer default.
* Test driver: Print total time spent at the endJoachim Breitner2013-09-201-1/+6
|
* Make test cleaning a little more consistentIan Lynagh2013-05-171-4/+6
|
* Add a multi_cpu_race helperIan Lynagh2013-04-171-0/+5
| | | | | This makes the test run alone, to increase the chance of a multi-CPU race happening.
* Fix tests for dynamic ghcIan Lynagh2013-03-151-1/+1
|
* Remove composeIan Lynagh2013-02-171-3/+0
|
* Remove uses of compose(s) in tests, and change how composition is handledIan Lynagh2013-02-141-13/+10
| | | | The driver now also supports nested lists of setup functions
* Reorder some functions to group themIan Lynagh2013-02-131-18/+17
| | | | Predicates now match the order they are documented on the wiki
* Tweak the brokens list to include the directory the test is inIan Lynagh2013-02-111-4/+4
|
* More conversionsIan Lynagh2013-02-111-6/+3
|
* More helper conversionsIan Lynagh2013-02-111-6/+3
|
* Convert more helpersIan Lynagh2013-02-111-11/+2
|
* Convert more helper functionsIan Lynagh2013-02-111-61/+22
|
* Convert more helper functions to the new schemeIan Lynagh2013-02-111-33/+6
|
* Convert more helpers to the new schemeIan Lynagh2013-02-111-21/+6
|
* Change '{if,unless}_arch' to 'arch'Ian Lynagh2013-02-111-11/+2
|
* Handle duplicates in the brokens listIan Lynagh2013-02-111-4/+8
|
* expect_broken_for should also be added to the list of brokensIan Lynagh2013-02-111-8/+10
|
* Replace 'if_os' and 'unless_os' with 'opsys'Ian Lynagh2013-02-111-11/+2
| | | | The name 'os' clashes with a python library
* Add 'make list_brokens'Ian Lynagh2013-02-111-1/+5
| | | | | | Gives a list of tickets that the testsuite thinks are broken, and what bug it thinks is the reason. This can then be pasted into trac and 'previewed', which will show any closed tickets with strikeout.
* Rearrange the summary outputIan Lynagh2013-02-101-16/+18
| | | | | | In particular, the 3 values you most need to care about (framework failures, unexpected passes, unexpected failures) are now together in a block.
* Remove unused helper functionsIan Lynagh2013-02-081-6/+0
|
* Replace 'if_platform' and 'unless_platform' with 'platform'Ian Lynagh2013-02-081-11/+2
|
* Allow a simpler form of stats_num_field where all platforms use the same valueIan Lynagh2013-02-071-5/+9
|
* Add a different sort of stats_num_field helper functionIan Lynagh2013-02-071-7/+29
| | | | | | | | | | | | | | | | Uses look like stats_num_field('bytes allocated', [(wordsize(32), 45648, 5), (wordsize(64), 49400, 5)]) where the first matching triple will be used. e.g. we could override the Win32 expected values with: [(platform('i386-unknown-mingw32'), 41000, 5), (wordsize(32), 45648, 5), (wordsize(64), 49400, 5)]) with other 32-bit platforms falling through to the wordsize(32) case. This makes it easier to give different values for different platforms, while being sure that all platforms are covered.
* Pass the test name to the test optionsIan Lynagh2013-02-071-93/+90
| | | | | | | | This allows them to give framework failures. I also had to change how setTestOpts works. Now, rather than applying the options to the directory's "default options", it just stores the options to be applied for each test (i.e. once we know the test name).
* Change how the testsuite driver handles copying the test optionsIan Lynagh2013-02-071-5/+3
| | | | | We now deepcopy the options for each test, which means that the test helpers don't need to worry about sharing.
* Rename some argumentsIan Lynagh2013-02-071-4/+4
|
* Define 'when' and 'unless' helpersIan Lynagh2013-02-071-11/+11
| | | | This will reduce the number of helper functions that we need
* Simplify driver code slightlyIan Lynagh2013-02-071-2/+1
|
* Add a TEST="..." line to testsuite outputIan Lynagh2013-02-071-0/+13
| | | | | This gives hte list of tests with unexpected results, so that you can easily run them again.
* Make failing perf test output easier to readIan Lynagh2013-02-061-10/+23
|
* Eliminate *_num_fieldIan Lynagh2013-02-061-49/+11
| | | | We now use *_range_field everywhere instead
* Fix cabal01 on WindowsIan Lynagh2013-02-051-1/+1
| | | | | | | We were getting a Creating library file: dist\build\libHStest-1.0-ghc7.7.20130205.dll.a message on stderr. We can't turn it off, so now the test just normalises it away.
View Lorry Controller Status