summaryrefslogtreecommitdiff
path: root/testsuite/driver/runtests.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Flag to test suite: SKIP_PERF_TESTSJoachim Breitner2013-10-051-0/+4
| | | | | | More often than not the output of the performance tests is in the way, rather than helping. This allows the use of `make SKIP_PERF_TESTS=YES` to skip these tests. Fixes #8413
* Test driver: Implement different verbositiesJoachim Breitner2013-09-201-1/+9
| | | | | | | | | Select verbosity with "make VERBOSE=n". Options so far: n=0: No per-test output n=1: Only failing test results n=2: As above, plus progress information (names of all tests) n=3: As aobve, plus commands called. Default currently is n=3, although n=2 might be a nicer default.
* Test driver: Print total time spent at the endJoachim Breitner2013-09-201-5/+2
|
* Tweak the brokens list to include the directory the test is inIan Lynagh2013-02-111-1/+1
|
* Print a warning if we get framework failures when listing brokensIan Lynagh2013-02-111-0/+6
|
* Add 'make list_brokens'Ian Lynagh2013-02-111-20/+25
| | | | | | Gives a list of tickets that the testsuite thinks are broken, and what bug it thinks is the reason. This can then be pasted into trac and 'previewed', which will show any closed tickets with strikeout.
* Pass the test name to the test optionsIan Lynagh2013-02-071-3/+0
| | | | | | | | This allows them to give framework failures. I also had to change how setTestOpts works. Now, rather than applying the options to the directory's "default options", it just stores the options to be applied for each test (i.e. once we know the test name).
* When the testsuite is ^Ced, print the summary anywayIan Lynagh2013-02-021-0/+4
|
* Revert "fix runtests to set LD_LIBRARY_PATH environment variable."Ian Lynagh2013-01-251-26/+22
| | | | | | This reverts commit d262089127c54bfe81963628ac70a309f8133492. We shouldn't need to do this on Linux.
* fix runtests to set LD_LIBRARY_PATH environment variable.David Terei2013-01-241-22/+26
| | | | Patch from Karel Gardas <karel.gardas@centrum.cz>.
* Run all the 'alone' tests at the end of the testsuite runIan Lynagh2012-10-111-1/+4
| | | | | This is more efficient than having to wait for all threads to finish before each 'alone' test.
* Fix a testsuite driver failure on cygwinIan Lynagh2012-05-221-0/+11
|
* Change how we detect whether the terminal is cygwin or msysIan Lynagh2012-05-011-9/+9
| | | | sys.platform now always returns 'win32' for me.
* Disable threads with python 2.7.2.Paolo Capriotti2012-03-061-0/+6
|
* cleaning of testsuiteDavid Terei2011-11-151-4/+0
|
* Driver: define config.libdir for tests to useIan Lynagh2011-10-051-1/+1
| | | | | We also use it internally, which saves us running "ghc --print-libdir" on Windows.
* Add support for checking whether files are written by more than one testIan Lynagh2011-08-071-0/+5
| | | | | | | | | | | | | | | | | | | | Work in progress, but largely works. Known issues: * only supported when using the timeout program * 'test.strace' files aren't cleaned, as they end up in the root directory rather than the test's directory * Doesn't yet track what the current directory is, so finds several files like "A.o" being written by multiple tests (and conversely, may be missing writes to the same file from different directories) * Lots of tests write to $HOME/.ghc/ghci_history. We should probably be passing ghci a flag to stop this from happening. * Some .strace lines aren't understood yet, causing framework failures * One .strace file can cause muiltiple framework failures, if it contains lots of lines that aren't understood Performance: Threads fast testsuite time fast testsuite time with checks 1 16:36.14 25:16.07 5 5:33.95 8:04.05
* Expect 4006 to fail on msysIan Lynagh2011-06-231-0/+8
| | | | | | | In the 65001 codepage, we can't even cat the expected output on msys: $ cat 4006.stdout It works here cat: write error: Permission denied
* speedup testsuite driver startup on Windows/OSXSimon Marlow2010-09-151-25/+27
| | | | | | Instead of calling "ghc-pkg list" and "ghc-pkg field" for each package, call "ghc-pkg dump" and grep the output. Saves a few seconds on Windows for 'make TEST=foo'.
* Move the ctypes import in the driverIan Lynagh2010-05-181-1/+1
| | | | | The import is failing on sparky, and we only use it on Windows anyway, so move it inside an "if windows".
* Fix running dyn tests on OS XIan Lynagh2010-05-031-17/+18
|
* Make the dynlib tests work on WindowsIan Lynagh2010-05-031-1/+33
|
* Fix running the testsuite on msysIan Lynagh2010-04-121-2/+10
| | | | | I'm not entirely sure if the cygwin code is actually right (i.e. I'm not sure what calling convention it uses), but it seems to work.
* Use the UTF8 codepage when running on WindowsIan Lynagh2010-03-191-22/+33
| | | | Fixes openTempFile001 for some system codepages, most notably 437 (US).
* Gather all tests at once, rather than doing them directory by directoryIan Lynagh2009-11-281-22/+13
| | | | | This increases the parallelism possible, and allows us to track what progress we are making.
* Fix setting of utf8 locale on Mac OS XManuel M T Chakravarty2009-11-121-1/+1
|
* Don't use threads on WindowsIan Lynagh2009-10-281-17/+22
| | | | It seems to cause some sort of deadlock
* wibbles to setting LC_ALL, trying to fix buildbot test failuresSimon Marlow2009-09-291-2/+3
|
* Fix runtests.py for Python 2.6.1Manuel M T Chakravarty2009-09-171-3/+5
| | | | - This is the version of Python that comes with Snow Leopard
* Look at whether we already have a UTF8 locale, before trying to find oneIan Lynagh2009-06-111-9/+14
| | | | | | This works around a problem on (old?) Fedora systems, where "locale -a" lists every locale that /might/ exist, rather than only those that /do/ exist.
* Try and find a utf8 locale to useIan Lynagh2009-06-101-0/+16
| | | | In other locales we get some test failures for some ghci unicode tests
* Fix trac #3091: the driver was choking on python versions containing lettersIan Lynagh2009-03-201-3/+7
|
* Simplify, and correct, the code for determining whether we should use threadsIan Lynagh2008-06-121-0/+1
|
* Refuse to use threads unless python version >= 2.5.2Ian Lynagh2008-06-111-1/+12
| | | | | According to trac #1558, 2.5.2 should work. It's possible a lower bound, e.g. 2.5, would suffice.
* Be more consistent in printing about framework failuresIan Lynagh2007-11-231-1/+1
|
* Set os.environ['TERM'] = 'vt100' before running testsIan Lynagh2007-09-061-0/+5
| | | | | Readline sometimes spews out ANSI escapes for some values of TERM, which result in test failures.
* count a syntax error in a .T file as a framework failureSimon Marlow2007-08-231-0/+1
|
* clean up the extra_ways machinerySimon Marlow2007-06-271-2/+4
|
* Allow tests to be skipped based on whether or not the compiler has a tagIan Lynagh2007-04-131-0/+3
| | | | | | When working on a new foo extension, you can now put your tests in the testsuite, set ProjectTags=-foo in mk/build.mk and skip_unless_tag('foo') in the tests.
* Calibrate the testsuite timeout if a value of -1 is givenIan Lynagh2007-04-051-0/+6
|
* Time how long is spent on each .T fileIan Lynagh2007-03-311-0/+15
|
* Look for .T files in packages tooSimon Marlow2007-02-211-2/+5
| | | | | | | | This means we can put package-specific tests in the repository for the package, rather than putting them in the testsuite. There should be a .T file to go with the tests, in the same way as for other tests in the testsuite (but this could be in addition to a standalone test driver that works with Cabal's 'setup test').
* Option to skip ways when running the testsuiteIan Lynagh2007-02-061-0/+10
|
* Allow threading to be completely disabled with USETHREADS=0Ian Lynagh2006-08-291-14/+26
| | | | | | | I had to pull the global classes and instances out into their own module as there was a catch-22: testlib needed to know if threading was enabled, but we don't know that until we have gone through the argument, but going through the arguments required changing things like config in testlib.
* Driver: Add THREADS-supportEsa Ilari Vuokko2006-08-221-3/+13
|
* [project @ 2005-02-04 10:59:55 by simonmar]simonmar2005-02-041-1/+0
| | | | | Add a timeout to test runs, using a wrapper program (written in Haskell, using System.Process of course!).
* [project @ 2003-08-19 21:51:53 by krc]krc2003-08-191-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Added support for testing generation and compilation of External Core code. There are two new ways, which are not automatically enabled but can be invoked from the command line: extcore and optextcore. Invoking either way will test that ghc is able to generate External Core code for a given test, read the code back in, and compile it to an executable that produces the expected output for the test. The External Core facility has a few limitations which result in certain tests failing for the "extcore" way. - External Core can't represent foreign calls other than static C calls - External Core can't correctly represent literals resulting from a "foreign label" declaration - External Core can't represent declarations of datatypes with no constructors The first of these was already known, and GHC panics if you tried to generate External Core for a program containing such a call. The second two cases were not handled properly before now; in another commit, I've changed the code that emits External Core to panic if either of them arises. Previously, GHC would happily generate External Core in either case, but would not be able to compile the resulting code. There are several tests that exhibit these limitations of External Core, so they've had to be made "expected failures" when compiling in the extcore or optextcore ways.
* [project @ 2002-09-11 10:47:57 by simonmar]simonmar2002-09-111-0/+9
| | | | | | | | | | - Move some of the way-selection logic into the configuration file; the build system now just passes in variables saying whether the compiler supports profiling and native code generation, and the configuration file adds the appropriate ways. - Add a new option to the test driver, --way=<way> to select just a single way.
* [project @ 2002-07-31 14:24:18 by simonmar]simonmar2002-07-311-0/+86
Revamp the testsuite framework. The previous framework was an experiment that got a little out of control - a whole new language with an interpreter written in Haskell was rather heavyweight and left us with a maintenance problem. So the new test driver is written in Python. The downside is that you need Python to run the testsuite, but we don't think that's too big a problem since it only affects developers and Python installs pretty easily onto everything these days. Highlights: - 790 lines of Python, vs. 5300 lines of Haskell + 720 lines of <strange made-up language>. - the framework supports running tests in various "ways", which should catch more bugs. By default, each test is run in three ways: normal, -O, and -O -fasm. Additionally, if profiling libraries have been built, another way (-O -prof -auto-all) is added. I plan to also add a 'GHCi' way. Running tests multiple ways has already shown up some new bugs! - documentation is in the README file and is somewhat improved. - the framework is rather less GHC-specific, and could without much difficulty be coaxed into using other compilers. Most of the GHC-specificness is in a separate configuration file (config/ghc). Things may need a while to settle down. Expect some unexpected failures.
View Lorry Controller Status