summaryrefslogtreecommitdiff
path: root/testsuite/driver/runtests.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Skip all performance tests if not in a git repo.David Eichmann2018-11-301-6/+26
| | | | | | | | | | | | Reviewers: bgamari, tdammers, osa1 Reviewed By: tdammers Subscribers: osa1, tdammers, rwbarton, carter GHC Trac Issues: #15923 Differential Revision: https://phabricator.haskell.org/D5367
* testsuite: Save performance metrics in git notes.David Eichmann2018-11-071-11/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes the following improvement: - Automatically records test metrics (per test environment) so that the programmer need not supply nor update expected values in *.T files. - On expected metric changes, the programmer need only indicate the direction of change in the git commit message. - Provides a simple python tool "perf_notes.py" to compare metrics over time. Issues: - Using just the previous commit allows performance to drift with each commit. - Currently we allow drift as we have a preference for minimizing false positives. - Some possible alternatives include: - Use metrics from a fixed commit per test: the last commit that allowed a change in performance (else the oldest metric) - Or use some sort of aggregate since the last commit that allowed a change in performance (else all available metrics) - These alternatives may result in a performance issue (with the test driver) having to heavily search git commits/notes. - Run locally, performance tests will trivially pass unless the tests were run locally on the previous commit. This is often not the case e.g. after pulling recent changes. Previously, *.T files contain statements such as: ``` stats_num_field('peak_megabytes_allocated', (2, 1)) compiler_stats_num_field('bytes allocated', [(wordsize(64), 165890392, 10)]) ``` This required the programmer to give the expected values and a tolerance deviation (percentage). With this patch, the above statements are replaced with: ``` collect_stats('peak_megabytes_allocated', 5) collect_compiler_stats('bytes allocated', 10) ``` So that programmer must only enter which metrics to test and a tolerance deviation. No expected value is required. CircleCI will then run the tests per test environment and record the metrics to a git note for that commit and push them to the git.haskell.org ghc repo. Metrics will be compared to the previous commit. If they are different by the tolerance deviation from the *.T file, then the corresponding test will fail. By adding to the git commit message e.g. ``` # Metric (In|De)crease <metric(s)> <options>: <tests> Metric Increase ['bytes allocated', 'peak_megabytes_allocated'] \ (test_env='linux_x86', way='default'): Test012, Test345 Metric Decrease 'bytes allocated': Test678 Metric Increase: Test711 ``` This will allow the noted changes (letting the test pass). Note that by omitting metrics or options, the change will apply to all possible metrics/options (i.e. in the above, an increase for all metrics in all test environments is allowed for Test711) phabricator will use the message in the description Reviewers: bgamari, hvr Reviewed By: bgamari Subscribers: rwbarton, carter GHC Trac Issues: #12758 Differential Revision: https://phabricator.haskell.org/D5059
* Simplify testsuite driver, part 2Krzysztof Gogolewski2018-08-121-4/+4
| | | | | | | | | | | | | | | | | | Summary: - Avoid import *; this helps tools such as pyflakes. The last occurrence in runtests.py is not easy to remove as it's used by .T files. - Use False/True instead of 0/1. Test Plan: validate Reviewers: bgamari, thomie, simonmar Reviewed By: thomie Subscribers: rwbarton, carter Differential Revision: https://phabricator.haskell.org/D5062
* Simplify testsuite driverKrzysztof Gogolewski2018-08-111-1/+1
| | | | | | | | | | | | | | | | | | | Summary: - remove clean_cmd - framework_failures was undefined - times_file was not used - if_verbose_dump was called only when verbose >= 1; remove the check - simplify normalise_whitespace Test Plan: validate Reviewers: bgamari, thomie Reviewed By: thomie Subscribers: rwbarton, carter Differential Revision: https://phabricator.haskell.org/D5061
* Testsuite driver: fix encoding issue when calling ghc-pkgKrzysztof Gogolewski2018-08-061-1/+1
| | | | | | | | | | | | | | | | Summary: In Python 3, subprocess.communicate() returns a pair of bytes, which need to be decoded. In runtests.py, we were just calling str() instead, which converts b'x' to "b'x'". As a result, the loop that was checking pkginfo for lines starting with 'library-dirs' couldn't work. Reviewers: bgamari, thomie, Phyx Reviewed By: thomie Subscribers: Phyx, rwbarton, carter Differential Revision: https://phabricator.haskell.org/D5046
* Fix typosKrzysztof Gogolewski2018-08-051-1/+1
|
* Remove dead code in testsuite driverKrzysztof Gogolewski2018-07-271-4/+0
| | | | | | | | | | | | Test Plan: validate Reviewers: bgamari, O7 GHC - Testsuite Reviewed By: bgamari Subscribers: rwbarton, thomie, carter Differential Revision: https://phabricator.haskell.org/D4972
* #15387 Fix setting testsuite verbose to zeroAntti Siponen2018-07-161-1/+1
|
* testsuite: Print summary even if interruptedBen Gamari2018-06-141-15/+18
| | | | | | | | | | | | | | Fixes #15265. Reviewers: osa1 Reviewed By: osa1 Subscribers: rwbarton, thomie, carter GHC Trac Issues: #15265 Differential Revision: https://phabricator.haskell.org/D4841
* Windows: fix all failing tests.Tamar Christina2018-01-021-10/+23
| | | | | | | | | | | | | | | | | | | | | | This makes the testsuite pass clean on Windows again. It also fixes the `libstdc++-6.dll` error harbormaster was showing. I'm marking some tests as isolated tests to reduce their flakiness (mostly concurrency tests) when the test system is under heavy load. Updates process submodule. Test Plan: ./validate Reviewers: hvr, bgamari, erikd, simonmar Reviewed By: bgamari Subscribers: rwbarton, thomie, carter Differential Revision: https://phabricator.haskell.org/D4277
* testsuite: Exit with non-zero exit code when tests failBen Gamari2017-12-181-1/+8
|
* Only look for locales of the form LL.VVGabor Greif2017-12-111-1/+1
| | | | | | Because in recent RHEL7 suddenly locales like `bokmål` pop up, which screw up reading-in of ASCII strings a line later. This additional criterion reliably eliminates those unicode characters.
* testsuite: Fix validation of waysBen Gamari2017-09-051-9/+18
| | | | | | | | | | | | Test Plan: Run `make test WAY=prof` Reviewers: angerman, austin Subscribers: rwbarton, thomie GHC Trac Issues: #14181 Differential Revision: https://phabricator.haskell.org/D3917
* testsuite: Don't pass allow_abbrevBen Gamari2017-07-281-2/+1
| | | | | | | | | | | | | This is only supported by Python 3.5 and later, which is too new for us to rely on. Reviewers: austin Subscribers: rwbarton, thomie, RyanGlScott GHC Trac Issues: #14050 Differential Revision: https://phabricator.haskell.org/D3803
* testsuite: Produce JUnit outputBen Gamari2017-07-281-0/+5
| | | | | | | | | | | | Test Plan: Validate, try ingesting into Jenkins. Reviewers: austin Subscribers: rwbarton, thomie GHC Trac Issues: #13716 Differential Revision: https://phabricator.haskell.org/D3796
* Switched out optparse for argparse in runtests.pyJared Weakly2017-07-281-77/+57
| | | | | | | | | | | | Tangentially related to my prior work on trac ticket #12758. Signed-off-by: Jared Weakly <jweakly@pdx.edu> Reviewers: austin, bgamari Subscribers: rwbarton, thomie Differential Revision: https://phabricator.haskell.org/D3792
* testsuite: Move echoing commands in make invocations to VERBOSE=5Reid Barton2017-03-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | D2894 added a new verbosity level VERBOSE=4 to strip -s/--silent flags from make invocations in test commands. This will probably cause the test to fail of course, but is useful for seeing what a test that's already failing is doing. However there was already an undocumented meaning of VERBOSE=4, added in commit cfeededf, that causes the results of performance tests to be printed unconditionally (even when they are within the expected range). nomeata's ghc builder uses these figures to collect historical data on performance test figures. The new meaning of VERBOSE=4 added in D2894 means that any test that uses make now fails on the builder. This commit moves the new behavior of D2894 to the level VERBOSE=5 so that nomeata's ghc builder again produces useful results on failing tests. It also adds documentation for both settings. Test Plan: did some manual testing Reviewers: austin, bgamari, Phyx, nomeata Reviewed By: bgamari, Phyx Subscribers: nomeata, thomie, Phyx Differential Revision: https://phabricator.haskell.org/D3141
* testsuite: Remove old python version testsReid Barton2017-02-231-38/+4
| | | | | | | | | | | | Test Plan: harbormaster Reviewers: austin, bgamari Reviewed By: bgamari Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D3140
* Revert "Suppress duplicate .T files"Gabor Greif2016-12-221-1/+1
| | | | | | | | This reverts commit 9a29b65bda8aed4c5fdbff25866ddf2dd1583210. It turns out that while not harmful, that commit is unnecessary, and a `make clean` resolved it. See: https://phabricator.haskell.org/rGHC9a29b65bda8aed4c5fdbff25866ddf2dd1583210
* Suppress duplicate .T filesGabor Greif2016-12-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As per http://stackoverflow.com/questions/7961363/removing-duplicates-in-lists use the set() function to zap duplicates from the obtained list of .T files. I am using $ python3 --version Python 3.5.1 and strangely findTFiles() returns some .T files twice: -- BEFORE Found 376 .T files... ... ====> Scanning ../../libraries/array/tests/all.T ====> Scanning ../../libraries/array/tests/all.T *** framework failure for T2120(duplicate) There are multiple tests with this name *** framework failure for largeArray(duplicate) There are multiple tests with this name *** framework failure for array001(duplicate) There are multiple tests with this name *** framework failure for T9220(duplicate) There are multiple tests with this name *** framework failure for T229(duplicate) There are multiple tests with this name ... -- AFTER Found 365 .T files... ... ====> Scanning ../../libraries/array/tests/all.T ... Even more strangely 'find' begs to differ: $ find libraries testsuite/tests -name "*.T" | sort | uniq | wc -l 368
* testsuite: Use python3 by defaultBen Gamari2016-11-291-4/+3
| | | | | | | | | | | | | | | | | | | | Summary: It turns out that Phyx's fix for #12554 (D2684) still fails with mingw-w64 python 2.7. However, Python 3 (both msys2 and mingw-w64) work fine. Given that supporting Python 2 has already become rather tiresome (as @thomie warned it would), let's just move to python3 by default. Test Plan: Validate Reviewers: austin, Phyx Reviewed By: Phyx Subscribers: Phyx, thomie Differential Revision: https://phabricator.haskell.org/D2766 GHC Trac Issues: #12554
* Fix testsuite threading, timeout, encoding and performance issues on WindowsTamar Christina2016-11-291-19/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In a land far far away, a project called Cygwin was born. Cygwin used newlib as it's standard C library implementation. But Cygwin wanted to emulate POSIX systems as closely as possible. So it implemented `execv` using the Windows function `spawnve`. Specifically ``` spawnve (_P_OVERLAY, path, argv, cur_environ ()) ``` `_P_OVERLAY` is crucial, as it makes the function behave *sort of* like execv on linux. the child process replaces the original process. With one major difference because of the difference in process models on Windows: the original process signals the caller that it's done. this is why the file is still locked. because it's still running, control was returned because the parent process was destroyed, but the child is still running. I think it's just pure dumb luck, that the older runtimes are slow enough to give the process time to terminate before we tried deleting the file. Which explains why you do have sporadic failures even on older runtimes like 2.5.0, of a test or two (like T7307). So this patch fixes a couple of things. I leverage the existing `timeout.exe` to implement a workaround for this issue. a) The old timeout used to start the process then assign it to the job. This is slightly faulty since child processes are only assigned to a job is their parent were assigned at the time they started. So this was a race condition. I now create the process suspended, assign it to the job and then resume it. Which means all child processes are not running under the same job. b) First things, Is to prevent dangling child processes. I mark the job with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` so when the last process in the job is done, it insures all processes under the job are killed. c) Secondly, I change the way we wait for results. Instead of waiting for the parent process to terminate, I wait for the job itself to terminate. There's a slight subtlety there, we can't wait on the job itself. Instead we have to create an I/O Completion port and wait for signals on it. See https://blogs.msdn.microsoft.com/oldnewthing/20130405-00/?p=4743 This fixes the issues on all runtimes for me and makes T7307 pass consistenly. The threading was also simplified by hiding all the locking in a single semaphore and a completion class. Futhermore some additional error reporting was added. For encoding the testsuite now no longer passes a file handle to the subprocess since on windows, sh.exe seems to acquire a lock on the file that is not released in a timely fashion. I suspect this because cygwin seems to emulate console handles by creating file handles and using those for std handles. So when we give it an existing file handle it just locks the file. I what's happening is that it's not releasing the handle until all shared cygwin processes are dead. Which explains why it worked in single threaded mode. So now instead we pass a pipe and do not interpret the resulting data. Any bytes written to stdin or read out of stdout/stderr are done so in binary mode and we do not interpret the data. The reason for this is that we have encoding tests in GHC which pass invalid utf-8. If we try to handle the data as text then python will throw an exception instead of a test comparison failing. Also I have fixed the ability to override `PYTHON` when calling `make tests`. This now works the same as with `.\validate`. Finally, after cleaning up the locks I was able to make the abort behavior work correctly as I believe it was intended: when you press Ctrl+C and send an interrupt signal, the testsuite finishes the active tests and then gracefully exits showing you a report of the progress it did make. So using Ctrl+C will not just *die* as it did before. These changes lift the restriction on which python version you use (msys/mingw) or which runtime or python 3 or python 2. All combinations should now be supported. Test Plan: PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH && PYTHON=/usr/bin/python THREADS=9 make test THREADS=9 make test PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH && PYTHON=/usr/bin/python ./validate --quiet --testsuite-only Reviewers: erikd, RyanGlScott, bgamari, austin Subscribers: jrtc27, mpickering, thomie, #ghc_windows_task_force Differential Revision: https://phabricator.haskell.org/D2684 GHC Trac Issues: #12725, #12554, #12661, #12004
* testsuite: Simplify kernel32 glue logicBen Gamari2016-11-021-5/+6
| | | | | | | | | | | | | | | | | | | | On Windows the testsuite driver calls kernel32 to set the current terminal codepage. The previous implementation of this was significantly more complex than necessary, and was wrong in the case of MSYS2, which requires that we explicitly load the library using the name of its DLL, including its file extension. Test Plan: Validate on Windows Reviewers: austin, RyanGlScott, Phyx Reviewed By: RyanGlScott, Phyx Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2641 GHC Trac Issues: #12661
* testsuite/driver: Allow threading on WindowsBen Gamari2016-10-171-3/+0
| | | | | | | | | | | | | | | | It seems that threading now works fine. The only caveat here is that it makes some race conditions more likely (e.g. #12554), although these also appear to affect single-threaded runs. Test Plan: Validate on Windows Reviewers: austin, Phyx Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2600 GHC Trac Issues: #10510
* testsuite/driver: More Unicode awarenessBen Gamari2016-10-171-2/+7
| | | | | | | | | | | | | Explicitly specify utf8 encoding in a few spots which were failing on Windows with Python 3. Test Plan: Validate Reviewers: austin, thomie Differential Revision: https://phabricator.haskell.org/D2602 GHC Trac Issues: #9184
* Testsuite: framework failure improvements (#11165)Thomas Miedema2016-06-281-7/+13
| | | | | | * add framework failures to unexpected results list * report errors in .T files as framework failures (show in summary) * don't report missing tests when framework failures in .T files
* Testsuite: cleanup printing of summaryThomas Miedema2016-06-281-2/+2
| | | | | | Just use a simple list of tuples, instead of a nested map. -90 lines of code.
* Testsuite: never pick up .T files in .run directoriesThomas Miedema2016-06-271-1/+1
| | | | | And use os.walk instead of calling os.listdir many times. The testsuite driver should be able to handle backward slashes on Windows now.
* Testsuite: write "\n" instead of "\r\n" when using mingw PythonThomas Miedema2016-06-181-1/+1
| | | | | | | | | | | | | | | Mingw style Python uses '\r\n' by default for newlines. This is annoying, because it means that when a GHC developer on Windows uses mingw Python to `make accept` a test, every single line of the .stderr file is touched. This makes it difficult to spot the real changes, and it leads to unnecessary git history bloat. Prevent this from happening by using io.open instead of open. See `Note [Universal newlines]` Reviewed by: Phyx Differential Revision: https://phabricator.haskell.org/D2342
* Testsuite: run tests in <testdir>.run instead of /tmpThomas Miedema2016-06-181-2/+48
| | | | | | | | | | | | | | | | | | | | As discussed in Phab:D1187, this approach makes it a bit easier to inspect the test directory while working on a new test. The only tests that needed changes are the ones that refer to files in ancestor directories. Those files are now copied directly into the test directory. validate still runs the tests in a temporary directory in /tmp, see `Note [Running tests in /tmp]` in testsuite/driver/runtests.py. Update submodule hpc. Reviewed by: simonmar Differential Revision: https://phabricator.haskell.org/D2333 GHC Trac Issues: #11980
* Testsuite driver: always quote opts.testdirThomas Miedema2016-06-071-1/+1
| | | | | | | This makes sure the testsuite keeps working when testdir contains backward slashes. Differential Revision: https://phabricator.haskell.org/D2314
* Testsuite: run tests in /tmp/ghctest-xxx instead of /tmp/ghctest/xxxThomas Miedema2016-06-021-15/+1
| | | | | | | This fixes a bug of not being able to create files or directories in /tmp/ghctest if it was created by a different user. Trac ticket #11980.
* Testsuite: delete check_files_writtenThomas Miedema2016-05-241-3/+0
| | | | | | | | | The CHECK_FILES_WRITTEN feature is no longer necessary, since tests don't write to the source directory anymore (#11980). Reviewed by: bgamari Differential Revision: https://phabricator.haskell.org/D2162
* Testsuite: run tests in /tmp after copying required filesThomas Miedema2016-05-171-4/+26
| | | | | | | | | | | | | | | | | | | | | | | Major change to the testsuite driver. For each TEST: * create a directory `<testdir>` inside `/tmp`. * link/copy all source files that the test needs into `<testdir>`. * run the test inside `<testdir>`. * delete `<testdir>` Extra files are (temporarily) tracked in `testsuite/driver/extra_files.py`, but can also be specified using the `extra_files` setup function. Differential Revision: https://phabricator.haskell.org/D1187 Reviewed by: Rufflewind, bgamari Trac: #11980
* Testsuite: report and error out on unfound testsThomas Miedema2015-10-291-1/+7
| | | | | | | | | | | | | | | | | | | Users are sometimes confused why their test doesn't run. It is usually because of a misspelled testname, for example using 'TEST=1234' instead of 'TEST=T1234'. After this patch it is hopefully more clear what the problem is, showing: ERROR: tests not found: ['1234'] Instead of: 0 total tests, which gave rise to 0 test cases, of which 0 were skipped Reviewed by: austin, bgamari Differential Revision: https://phabricator.haskell.org/D1388
* Testsuite Windows: don't use forward slashes in topdir pathThomas Miedema2015-10-201-1/+4
| | | | | | | | | | | | | | Changing backwards slashes to forward slashes apparently confuses msys2/mingw magic path handling. I don't quite understand why, but this fixes it. Test Plan: on Windows, make sure PATH does not contain 'inplace/mingw/bin' (let the testsuite driver add it), then run: make TEST='ghcilink003 ghcilink006'. Before this patch, it would fail. Reviewed by: Phyx, bgamari, austin Differential Revision: https://phabricator.haskell.org/D1343
* Make Windows linker more robust to unknown sectionsTamar Christina2015-10-031-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Windows Linker has 3 main parts that this patch changes. 1) Identification and classification of sections 2) Adding of symbols to the symbols tables 3) Reallocation of sections 1. Previously section identification used to be done on a whitelisted basis. It was also exclusively being done based on the names of the sections. This meant that there was a bit of a cat and mouse game between `GCC` and `GHC`. Every time `GCC` added new sections there was a good chance `GHC` would break. Luckily this hasn't happened much in the past because the `GCC` versions `GHC` used were largely unchanged. The new code instead treats all new section as `CODE` or `DATA` sections, and changes the classifications based on the `Characteristics` flag in the PE header. By doing so we no longer have the fragility of changing section names. The one exception to this is the `.ctors` section, which has no differentiating flag in the PE header, but we know we need to treat it as initialization data. The check to see if the sections are aligned by `4` has been removed. The reason is that debug sections often time are `1 aligned` but do have relocation symbols. In order to support relocations of `.debug` sections this check needs to be gone. Crucially this assumption doesn't seem to be in the rest of the code. We only check if there are at least 4 bytes to realign further down the road. 2. The second loop is iterating of all the symbols in the file and trying to add them to the symbols table. Because the classification of the sections we did previously are (currently) not available in this phase we still have to exclude the sections by hand. If they don't we will load in symbols from sections we've explicitly ignored the in # 1. This whole part should rewritten to avoid this. But didn't want to do it in this commit. 3. Finally the sections are relocated. But for some reason the PE files contain a Linux relocation constant in them `0x0011` This constant as far as I can tell does not come from GHC (or I couldn't find where it's being set). I believe this is probably a bug in GAS. But because the constant is in the output we have to handle it. I am thus mapping it to the constant I think it should be `0x0003`. Finally, static linking *should* work, but won't. At least not if you want to statically link `libgcc` with exceptions support. Doing so would require you to link `libgcc` and `libstd++` but also `libmingwex`. The problem is that `libmingwex` also defines a lot of symbols that the RTS automatically injects into the symbol table. Presumably because they're symbols that it needs. like `coshf`. The these symbols are not in a section that is declared with weak symbols support. So if we ever want to get this working, we should either a) Ask mingw to declare the section as such, or b) treat all a imported symbols as being weak. Though this doesn't seem like it's a good idea.. Test Plan: Running ./validate for both x86 and x86_64 Also running the specific test case for #10672 make TESTS="T10672_x86 T10672_x64" Reviewed By: ezyang, thomie, austin Differential Revision: https://phabricator.haskell.org/D1244 GHC Trac Issues: #9907, #10672, #10563
* Testsuite: mention the existence of ticket #10510Thomas Miedema2015-06-111-1/+1
| | | | [skip ci]
* Make validate more quietThomas Miedema2015-06-041-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * By default use V=0, and call the testsuite with VERBOSE=2, which we did before only with validate --quiet. This disables printing the test commands it runs. * When --quiet is used, call the testsuite with VERBOSE=1. This disables printing the '====> Scanning' lines, and doesn't print which test is being run. So it only prints something when a test accidentally prints to stdout or when it fails. Don't set this option on Travis, as Travis will cancel a build if it doesn't see any output for more than 10 minutes. * When --quiet is used, set the new test option NO_PRINT_SUMMARY, which skips printing the test summary. Only the list of unexpected failures is printed, if there are any. Note that the full summary can still be found in testsuite_summary.txt * When --quiet is used, don't pass the `-v` flag to `ghc-pkg check` * When --quiet is used, don't print the Oops! header. It shoud be clear from the list of failing tests that something is wrong. This is all done to get the most out of 30 lines of logfile. These changes can be disabled later by simply not passing the --quiet flag to validate. Differential Revision: https://phabricator.haskell.org/D942
* Testdriver: do not interfer with MinGW path magic (#10449)Thomas Miedema2015-05-281-2/+7
| | | | | | | | | | | | | | | | | | This should fix the testsuite driver on Windows using the MinGW tools with a native build of Python. MinGW automagically converts MinGW-style paths (e.g. '/c/programs/ghc/bin/ghc') into ordinary Windows paths (e.g. 'C:/programs/ghc/bin/ghc') when a native Windows program is invoked. But it doesn't do so when those paths are wrapped with a pair of escaped double quotes. The fix is to not call `eval` on the paths in Python, which let's us use one less pair of quotes, and makes MinGW happy. Reviewers: Rufflewind, austin Differential Revision: https://phabricator.haskell.org/D911
* Testdriver: don't use os.popen in config/ghcPhil Ruffwind2015-05-281-4/+1
| | | | | | | | | | | | | | | Rewrite config/ghc to use getStdout (which use subprocess.Popen) instead of os.popen, which is deprecated; this also avoids the use of shell Also: * Move getStdout to driver/testutil.py so both config/ghc and driver/runtests.py can use it * Remove support for Python below 2.4, which doesn't have subprocess Reviewed By: thomie Differential Revision: https://phabricator.haskell.org/D908
* Python 3 support, second attempt (Trac #9184)Krzysztof Gogolewski2014-10-191-28/+39
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This is a fixup of https://phabricator.haskell.org/D233 The only difference is in findTFiles (first commit), which previously broke Windows runner; now I translated literally instead attempting to improve it, and checked it works. Test Plan: I used validate under 2,3 on Linux and under 2 on msys2. On Windows I've seen a large number of failures, but they don't seem to be connected with the patch. Reviewers: hvr, simonmar, thomie, austin Reviewed By: austin Subscribers: thomie, carter, ezyang, simonmar Differential Revision: https://phabricator.haskell.org/D310 GHC Trac Issues: #9184
* Extend windows detection in testsuite to recognize MSYS targetGintautas Miliauskas2014-10-111-2/+2
| | | | | | | | | | | | | | Currently, the detection recognizes the following `uname -s` strings: - `CYGWIN_NT-6.3` - `MINGW32_NT-6.3` - `MINGW64_NT_6.3` However, MSYS2 provides an additional target, in which case `uname -s` returns a string such as `MSYS_NT-6.3`. In all these cases, the system ought to be recognized as being a `windows` os by the testsuite runner. See also #9604
* Fallback to `ctypes.cdll` if `ctypes.windll` unavailableGintautas Miliauskas2014-10-111-6/+4
| | | | | | | | | On Windows, we may be using a native build of Python or a mingw/msys build. The former exports `ctypes.windll`, the latter exports `cdll`. Previously the code threw an exception when using the msys Python because it expected `windll` to always be available on Windows. Differential Revision: https://phabricator.haskell.org/D308
* Revert "Basic Python 3 support for testsuite driver (Trac #9184)"Krzysztof Gogolewski2014-10-031-39/+28
| | | | | | This reverts commit 084d241b316bfa12e41fc34cae993ca276bf0730. This is a possible culprit of Windows breakage reported at ghc-devs.
* Basic Python 3 support for testsuite driver (Trac #9184)Krzysztof Gogolewski2014-10-011-28/+39
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Most of the changes is adaptation of old Python 2 only code. My priority was not breaking Python 2, and so I avoided bigger changes to the driver. In particular, under Python 3 the output is a str and buffering cannot be disabled. To test, define PYTHON=python3 in testsuite/mk/boilerplate.mk. Thanks to aspidites <emarshall85@gmail.com> who provided the initial patch. Test Plan: validate under 2 and 3 Reviewers: hvr, simonmar, thomie, austin Reviewed By: thomie, austin Subscribers: aspidites, thomie, simonmar, ezyang, carter Differential Revision: https://phabricator.haskell.org/D233 GHC Trac Issues: #9184
* Revert "Fix a couple test failures encountered when building on Windows"Austin Seipp2014-09-011-4/+1
| | | | | This reverts commit 9711f78f790d10d914e08851544c6fc96f9a030a, as it's causing build phailures in phabricator.
* Fix a couple test failures encountered when building on WindowsAustin Seipp2014-09-011-1/+4
| | | | | | | | | | | | | | Summary: * Adjusts performance tests * Change ghcpkg05.stderr-mingw32 to match ghcpkg05.stderr Test Plan: Ran 'sh validate' and observed fewer test failures afterwards Reviewers: austin Reviewed By: austin Differential Revision: https://phabricator.haskell.org/D191
* New testsuite verbosity level 4Joachim Breitner2014-07-141-2/+2
| | | | | which makes it print performance numbers even when the test succeeds (good for historic analysis)
* Detect msys2 via uname (different to msys)Simon Peyton Jones2013-11-121-1/+4
| | | | | | uname -s msys gives "MINGW32" msys2 gives "MINGW_NT-6.2"
View Lorry Controller Status