summaryrefslogtreecommitdiff
path: root/testsuite/driver
Commit message (Collapse)AuthorAgeFilesLines
* testsuite: Fix some ints used as boolsBen Gamari2019-07-181-10/+10
|
* testsuite: Print output from hp2psBen Gamari2019-07-181-1/+1
|
* testsuite: Fix req_thBen Gamari2019-07-181-1/+2
|
* testsuite: More type checking fixesBen Gamari2019-07-185-96/+172
|
* Testsuite tweaks and refactoringÖmer Sinan Ağacan2019-07-091-8/+10
| | | | | | | | | | | | | | | | - Rename requires_th to req_th for consistency with other req functions (e.g. req_interp, req_profiling etc.) - req_th (previously requires_th) now checks for interpreter (via req_interp). With this running TH tests are skipped when running the test suite with stage=1. - Test tweaks: - T9360a, T9360b: Use req_interp - recomp009, T13938, RAE_T32a: Use req_th - Fix check-makefiles linter: it now looks for Makefiles instead of .T files (which are actually Python files)
* testsuite: Fix #16818Ben Gamari2019-07-091-3/+10
| | | | | Renames performance metrics to include whether they are compile-time or runtime metrics.
* testsuite: Add more type annotations to perf_notesBen Gamari2019-06-272-54/+100
|
* testsuite: More type signatureswip/run-fragile-testsBen Gamari2019-06-262-9/+12
|
* testsuite: Run and report on fragile testsBen Gamari2019-06-263-9/+23
| | | | | | This allows us to run (but ignore the result of) fragile testcases. Hopefully this should allow us to more easily spot when a fragile test becomes un-fragile.
* testsuite: Fix a few issues in JUnit outputBen Gamari2019-06-251-5/+14
| | | | | | | * Make it pass mypy * Fix a typo in test name field * Report more stderr output * Report stdout output
* testsuite: A major revamp of the driverBen Gamari2019-06-256-354/+495
| | | | | | | | | This tries to put the testsuite driver into a slightly more maintainable condition: * Add type annotations where easily done * Use pathlib.Path instead of str paths * Make it pass the mypy typechecker
* testsuite: Add assertions that way lists are in fact listsBen Gamari2019-06-151-0/+6
| | | | | | Previously there were a few cases where operations like `omit_ways` were incorrectly passed a single way (e.g. `omit_ways('threaded2')`). This won't work as the author expected.
* gitlab-ci: Lint testsuite for framework failuresBen Gamari2019-06-141-1/+1
| | | | | This introduces a new lint job checking for framework failures and listing broken tests.
* testsuite: Don't run tests requiring TH in profasm way when GhcDynamicBen Gamari2019-06-121-0/+8
| | | | | | | | | | Since we can't load profiled objects when GhcDynamic==YES. Affects: * T16737 * T16384 * T16718 * T16619 * T16190
* testsuite: Fix omit_ways usageBen Gamari2019-06-121-0/+1
| | | | omit_ways expects a list but this was broken in several cases.
* testsuite: Fix fragile_for test modifierBen Gamari2019-06-121-3/+3
|
* TestRunner: Added --chart to display a chart of performance testsDavid Eichmann2019-06-045-124/+354
| | | | | | | | | | | | | This uses the Chart.js javascript library. Everything is put into a standalone .html file and opened with the default browser. I also simplified the text output to use the same data as the chart. You can now use a commit range with git's ".." syntax. The --ci option will use results from CI (you'll need to fetch them first): $ git fetch https://gitlab.haskell.org/ghc/ghc-performance-notes.git refs/notes/perf:refs/notes/ci/perf $ python3 testsuite/driver/perf_notes.py --ci --chart --test-env x86_64-darwin --test-name T9630 master~500..master
* testsuite: introduce 'static_stats' testsAlp Mestanogullari2019-05-292-5/+16
| | | | | | | | | | | | | | | | They are a particular type of perf tests. This patch introduces a 'stats_files_dir' configuration field in the testsuite driver where all haddock timing files (and possibly others in the future) are assumed to live. We also change both the Make and Hadrian build systems to pass respectively $(TOP)/testsuite/tests/perf/haddock/ and <build root>/stage1/haddock-timing-files/ as the value of that new configuration field, and to generate the timing files in those directories in the first place while generating documentation with haddock. This new test type can be seen as one dedicated to examining stats files that are generated while building a GHC distribution. This also lets us get rid of the 'extra_files' directives in the all.T entries for haddock.base, haddock.Cabal and haddock.compiler.
* Improve test runner logging when calculating performance metric baseline #16662David Eichmann2019-05-201-5/+16
| | | | | | We attempt to get 75 commit hashes via `git log`, but this only gave 10 hashes in a CI run (see #16662). Better logging may help solve this error if it occurs again in the future.
* Update terminal title while running test-suiteOleg Grenrus2019-05-143-4/+30
| | | | | Useful progress indicator even when `make test VERBOSE=1`, and when you do something else, but have terminal title visible.
* codegen: fix memset unroll for small bytearrays, add 64-bit setsArtem Pyanykh2019-04-091-2/+57
| | | | | | | | | | | | | | | | | | | | | | Fixes #16052 When the offset in `setByteArray#` is statically known, we can provide better alignment guarantees then just 1 byte. Also, memset can now do 64-bit wide sets. The current memset intrinsic is not optimal however and can be improved for the case when we know that we deal with (baseAddress at known alignment) + offset For instance, on 64-bit `setByteArray# s 1# 23# 0#` given that bytearray is 8 bytes aligned could be unrolled into `movb, movw, movl, movq, movq`; but currently it is `movb x23` since alignment of 1 is all we can embed into MO_Memset op.
* testsuite: Show exit code of GHCi tests on failureÖmer Sinan Ağacan2019-04-081-1/+1
|
* Update Wiki URLs to point to GitLabTakenobu Tani2019-03-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | This moves all URL references to Trac Wiki to their corresponding GitLab counterparts. This substitution is classified as follows: 1. Automated substitution using sed with Ben's mapping rule [1] Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy... New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy... 2. Manual substitution for URLs containing `#` index Old: ghc.haskell.org/trac/ghc/wiki/XxxYyy...#Zzz New: gitlab.haskell.org/ghc/ghc/wikis/xxx-yyy...#zzz 3. Manual substitution for strings starting with `Commentary` Old: Commentary/XxxYyy... New: commentary/xxx-yyy... See also !539 [1]: https://gitlab.haskell.org/bgamari/gitlab-migration/blob/master/wiki-mapping.json
* Print test suite results ("unexpected failures" etc.) in sorted orderÖmer Sinan Ağacan2019-03-201-1/+1
| | | | Fixes #16425
* testsuite: Display observed exit code on failure due to bad exit codeBen Gamari2019-03-201-1/+1
|
* Update Trac ticket URLs to point to GitLabRyan Scott2019-03-151-2/+2
| | | | | This moves all URL references to Trac tickets to their corresponding GitLab counterparts.
* testsuite: Introduce fragile modifierBen Gamari2019-03-061-0/+24
| | | | | | | | | | Now since we have been a bit more stringent in testsuite cleanliness we have been marking a lot of tests as fragile using the `skip` modifier. However, this unfortunately means that we lose the association with the ticket number documenting the fragility. Here we introduce `fragile` and `fragile_for` to retain this information.
* Test Runner: don't show missing baseline warning for performance tests with ↵David Eichmann2019-03-051-15/+18
| | | | | | expected changes on the current commit. Trac #16359
* Fix parsing of expected performance changes for tests with non-alpha characters.David Eichmann2019-03-011-1/+1
| | | | Python's split() function is used to split on all white space.
* testsuite: Fix whitespace in hp2ps error messageBen Gamari2019-02-231-1/+1
|
* Exit with exit code 1 when tests unexpectedly passMatthew Pickering2019-02-232-0/+2
| | | | | | This was causing gitlab to not report from builds as failing. It also highlighted a problem with the LLVM tests where some of the external interpreter tests are failing.
* Testsuite: opt-in to symlinks on WindowsTamar Christina2019-02-211-1/+1
|
* Fix test runner crash when not in a git repoDavid Eichmann2019-02-213-14/+18
| | | | Respect `inside_git_repo()` when checking performance stats.
* Test bit-manipulating primops under respective arch flags like -msse4.2Dmitry Ivanov2019-02-201-3/+1
|
* Testsuite: implement use_specs.Tamar Christina2019-02-172-0/+29
|
* Fix and Reapply "Performance tests: recover a baseline from ancestor commits ↵David Eichmann2019-02-165-65/+277
| | | | and CI results."
* testsuite: Report unexpected passes in junit outputMatthew Pickering2019-02-111-2/+5
|
* testsuite: Report stdout and stderr in JUnit outputBen Gamari2019-02-104-34/+69
| | | | | | | | | | This patch makes the JUnit output more useful as now we also report the stdout/stderr in the message which can be used to quickly identify why a test is failing without downloading the log. This also introduces TestResult, previously we were simply passing around tuples, making things the implementation rather difficult to follow and harder to extend.
* testsuite: Mark print037 as broken when GHC is built with LLVMBen Gamari2019-02-041-0/+9
| | | | As noted in #16205 this configuration reliably segfaults.
* Revert "Performance tests: recover a baseline from ancestor commits and CI ↵Ben Gamari2019-01-315-259/+62
| | | | | | | | | results." Unfortunately this has broken all future commits due to spurious(?) performance changes which I have been unable to work around. This reverts commit cc2261d42f6a954d88e355aaad41f001f65c95da.
* Performance tests: recover a baseline from ancestor commits and CI results.David Eichmann2019-01-305-62/+259
| | | | gitlab-ci: push performance metrics as git notes to the "GHC Performance Notes" repository.
* testsuite: Use makefile_testBen Gamari2019-01-301-1/+1
| | | | | This eliminates most uses of run_command in the testsuite in favor of the more structured makefile_test.
* testsuite: Introduce makefile_testBen Gamari2019-01-301-0/+7
|
* Revert "Batch merge"Ben Gamari2019-01-305-267/+63
| | | | This reverts commit 76c8fd674435a652c75a96c85abbf26f1f221876.
* Batch mergeBen Gamari2019-01-305-63/+267
|
* testsuite: Remove directories that already exist when seeding extra_filesBen Gamari2019-01-271-0/+2
| | | | | Otherwise the testsuite driver crashes when run multiple times with CLEANUP=NO on a test containing such extra_files.
* testsuite: Add predicate for CPU feature availabilityBen Gamari2019-01-273-0/+79
| | | | | | | Previously testing code-generation for ISA extensions was nearly impossible since we had no ability to determine whether the host supports the needed extension. Here we fix this by introducing a simple /proc/cpuinfo-based testsuite predicate. We really ought to
* testsuite: Ensure that config.{msys,cygwin} are initializedBen Gamari2019-01-231-0/+2
| | | | | | | | | | Reviewers: monoidal Reviewed By: monoidal Subscribers: monoidal, rwbarton, carter Differential Revision: https://phabricator.haskell.org/D5056
* Fix tests for `integer-simple`Alec Theriault2019-01-161-3/+0
| | | | | | | | | | | | A bunch of tests for `integer-simple` were now broken for a foolish reason: unlike the `integer-gmp` case, there is no CorePrep optimization for turning small integers directly into applications of `S#`. Rather than port this optimization to `integer-simple` (which would involve moving a bunch of `integer-simple` names into `PrelNames`), I switched as many tests as possible to use `Int`. The printing of `Integer` is already tested in `print037`.
* testsuite: Show both test name and way in JUnit outputBen Gamari2019-01-141-6/+6
|
View Lorry Controller Status