summaryrefslogtreecommitdiff
path: root/testsuite/driver
Commit message (Collapse)AuthorAgeFilesLines
...
* [testlib] ignore strip warningsMoritz Angermann2021-03-291-0/+4
|
* [ci/arm/darwin/testsuite] Forwards ports from GHC-8.10Moritz Angermann2021-03-213-13/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a set of forward ports (cherry-picks) from 8.10 - a7d22795ed [ci] Add support for building on aarch64-darwin - 5109e87e13 [testlib/driver] denoise - 307d34945b [ci] default value for CONFIGURE_ARGS - 10a18cb4e0 [testsuite] mark ghci056 as fragile - 16c13d5acf [ci] Default value for MAKE_ARGS - ab571457b9 [ci/build] Copy config.sub around - 251892b98f [ci/darwin] bump nixpkgs rev - 5a6c36ecb4 [testsuite/darwin] fix conc059 - aae95ef0c9 [ci] add timing info - 3592d1104c [Aarch64] No div-by-zero; disable test. - 57671071ad [Darwin] mark stdc++ tests as broken - 33c4d49754 [testsuite] filter out superfluous dylib warnings - 4bea83afec [ci/nix-shell] Add Foundation and Security - 6345530062 [testsuite/json2] Fix failure with LLVM backends - c3944bc89d [ci/nix-shell] [Darwin] Stop the ld warnings about libiconv. - b821fcc714 [testsuite] static001 is not broken anymore. - f7062e1b0c [testsuite/arm64] fix section_alignment - 820b076698 [darwin] stop the DYLD_LIBRARY_PATH madness - 07b1af0362 [ci/nix-shell] uniquify NIX_LDFLAGS{_FOR_TARGET} As well as a few additional fixups needed to make this block compile: - Fixup all.T - Set CROSS_TARGET, BROKEN_TESTS, XZ, RUNTEST_ARGS, default value. - [ci] shell.nix bump happy
* gitlab-ci: Ignore performance improvements in marge jobsBen Gamari2021-03-191-12/+13
| | | | | | | | Currently we have far too many merge failures due to cumulative performance improvements. Avoid this by accepting metric decreases in marge-bot jobs. Fixes #19562.
* testsuite: Make --ignore-perf-tests more expressiveBen Gamari2021-03-193-7/+20
| | | | Allow skipping of only increases/decreases.
* testsuite: Introduce flag to ignore performance failuresBen Gamari2021-02-243-1/+6
| | | | Needed by #19025.
* Test Driver: Tweak interval of test reportingMatthew Pickering2021-02-181-5/+12
| | | | | | | | | | | Rather than just display every 100 tests, work out how many to display based on the total number of tests. This improves the experience when running a small number of tests. For [0..100] - Report every test [100..1000] - Report every 10 tests [1000..10000] - Report every 100 tests and so on..
* Fix typosBrian Wignall2021-02-062-2/+2
|
* Reduce default test verbosityMatthew Pickering2021-01-283-3/+4
|
* Deprecate -h flagMatthew Pickering2021-01-271-4/+4
| | | | | | | | | | It is confusing that it defaults to two different things depending on whether we are in the profiling way or not. Use -hc if you have a profiling build Use -hT if you have a normal build Fixes #19031
* Add some additional information to the fail message based on exit codeHécate2021-01-171-2/+16
|
* Make proper fixed-width number literalsSylvain Henry2021-01-021-1/+1
| | | | | | | | (Progress towards #11953, #17377, #17375) Besides being nicer to use, this also will allow for better constant folding for the fixed-width types, on par with what `Int#` and `Word#` have today.
* Bump the # of commits searched for perf baselineRichard Eisenberg2020-12-011-1/+1
| | | | | | | | The previous value of 75 meant that a feature branch with more than 75 commits would get spurious CI passes. This affects #18692, but does not fix that ticket, because if a baseline cannot be found, we should fail, not succeed.
* [Sized Cmm] properly retain sizes.Moritz Angermann2020-11-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces all Word<N> = W<N># Word# and Int<N> = I<N># Int# with Word<N> = W<N># Word<N># and Int<N> = I<N># Int<N>#, thus providing us with properly sized primitives in the codegenerator instead of pretending they are all full machine words. This came up when implementing darwinpcs for arm64. The darwinpcs reqires us to pack function argugments in excess of registers on the stack. While most procedure call standards (pcs) assume arguments are just passed in 8 byte slots; and thus the caller does not know the exact signature to make the call, darwinpcs requires us to adhere to the prototype, and thus have the correct sizes. If we specify CInt in the FFI call, it should correspond to the C int, and not just be Word sized, when it's only half the size. This does change the expected output of T16402 but the new result is no less correct as it eliminates the narrowing (instead of the `and` as was previously done). Bumps the array, bytestring, text, and binary submodules. Co-Authored-By: Ben Gamari <ben@well-typed.com> Metric Increase: T13701 T14697
* CmmToLlvm: Declare signature for memcmpwip/angerman/arm64Ben Gamari2020-11-241-0/+10
| | | | | | Otherwise `opt` fails with: error: use of undefined value '@memcmp$def'
* Merge remote-tracking branch 'origin/wip/tsan/all'Ben Gamari2020-11-082-0/+10
|\
| * testsuite: Skip high memory usage tests with TSANBen Gamari2020-10-241-0/+4
| | | | | | | | | | ThreadSanitizer significantly increases the memory footprint of tests, so much so that it can send machines into OOM.
| * testsuite: Mark setnumcapabilities001 as broken with TSANGHC GitLab CI2020-10-242-0/+6
| | | | | | | | Due to #18808.
* | Testsuite: Support for user supplied package dbsAndreas Klebinger2020-11-043-1/+21
| | | | | | | | | | | | | | | | | | We can now supply additional package dbs to the testsuite. For make the package db can be supplied by passing PACKAGE_DB=/path/to/db. In the testsuite driver it's passed via the --test-package-db argument.
* | testsuite: Add --top flag to driverGHC GitLab CI2020-11-023-3/+8
| | | | | | | | | | This allows us to make `config.top` a proper Path. Previously it was a str, which caused the Ghostscript detection logic to break.
* | Use config.run_ways for multi_compile_and_run testsDavid Eichmann2020-10-271-1/+1
|/
* testsuite: Sort metrics by metric typeBen Gamari2020-10-161-1/+15
| | | | Closes #18838.
* testsuite: Allow whitespace before "Metric (in|de)crease"Ben Gamari2020-10-071-1/+1
| | | | | | | Several people have struggled with metric change annotations in their commit messages not being recognized due to the fact that GitLab's job log inserts a space at the beginning of each line. Teach the regular expression to accept this whitespace.
* testsuite: Fix WinIO error message normalizationGHC GitLab CI2020-09-201-4/+6
| | | | This wasn't being applied to stderr.
* Enhance metrics outputSylvain Henry2020-09-152-10/+11
|
* Make sure we can read past perf notesKrzysztof Gogolewski2020-09-121-1/+9
| | | | See #18656.
* testsuite: Output performance test results in tabular formatDaishi Nakajima2020-09-083-35/+98
| | | | | | | | | | | | | | | | | this was suggested in #18417. Change the print format of the values. * Shorten commit hash * Reduce precision of the "Value" field * Shorten metrics name * e.g. runtime/bytes allocated -> run/alloc * Shorten "MetricsChange" * e.g. unchanged -> unch, increased -> incr And, print the baseline environment if there are baselines that were measured in a different environment than the current environment. If all "Baseline commit" are the same, print it once.
* testsuite: Only run llvm ways if llc is availableBen Gamari2020-08-181-0/+3
| | | | | | | | | As noted in #18560, we previously would always run the LLVM ways since `configure` would set `SettingsLlcCommand` to something non-null when it otherwise couldn't find the `llc` executable. Now we rather probe for the existence of the `llc` executable in the testsuite driver. Fixes #18560.
* testsuite: Allow baseline commit to be set explicitlyBen Gamari2020-08-184-15/+32
|
* testsuite: Normalise WinIO error message differencesBen Gamari2020-07-261-0/+24
| | | | | Previously the old Windows IO manager threw different errors than WinIO. We now canonicalise these to the WinIO errors.
* Add test for #18064Sylvain Henry2020-07-222-0/+8
| | | | It has been fixed by 0effc57d48ace6b719a9f4cbeac67c95ad55010b
* Add tests for #17920Sylvain Henry2020-06-231-2/+1
| | | | | | Metric Decrease: T12150 T12234
* Update testsuiteSylvain Henry2020-06-172-7/+10
| | | | | | | | | | | | | | * support detection of slow ghc-bignum backend (to replace the detection of integer-simple use). There are still some test cases that the native backend doesn't handle efficiently enough. * remove tests for GMP only functions that have been removed from ghc-bignum * fix test results showing dependent packages (e.g. integer-gmp) or showing suggested instances * fix test using Integer/Natural API or showing internal names
* testsuite: Work around spurious mypy failureBen Gamari2020-06-011-1/+1
|
* testsuite/testutil: Fix bytes/str mismatchBen Gamari2020-06-011-1/+1
|
* testsuite/perf_notes: Fix ill-typed assignmentsBen Gamari2020-06-011-4/+5
|
* testsuite: Refactor ghostscript detectionBen Gamari2020-06-013-25/+45
| | | | Tamar reported that he saw crashes due to unhandled exceptions.
* testsuite: Don't fail if we can't unlink __symlink_testBen Gamari2020-06-011-1/+4
| | | | | Afterall, it's possible we were unable to create it due to lack of symlink permission.
* Implement cstringLength# and FinalPtrAndrew Martin2020-05-231-0/+20
| | | | | | | | | | | | | | | | | | | | This function and its accompanying rule resolve issue #5218. A future PR to the bytestring library will make the internal Data.ByteString.Internal.unsafePackAddress compute string length with cstringLength#. This will improve the status quo because it is eligible for constant folding. Additionally, introduce a new data constructor to ForeignPtrContents named FinalPtr. This additional data constructor, when used in the IsString instance for ByteString, leads to more Core-to-Core optimization opportunities, fewer runtime allocations, and smaller binaries. Also, this commit re-exports all the functions from GHC.CString (including cstringLength#) in GHC.Exts. It also adds a new test driver. This test driver is used to perform substring matches on Core that is dumped after all the simplifier passes. In this commit, it is used to check that constant folding of cstringLength# works.
* Sort deterministically metric outputKrzysztof Gogolewski2020-05-211-1/+1
| | | | | | Previously, we sorted according to the test name and way, but the metrics (max_bytes_used/peak_megabytes_allocated etc.) were appearing in nondeterministic order.
* testsuite: Print sign of performance changesBen Gamari2020-05-131-2/+2
| | | | | Executes the minor formatting change in the tabulated performance changes suggested in #18135.
* testsuite: Don't attempt to read .std{err,out} files if they don't existBen Gamari2020-04-231-2/+10
| | | | | | | | Simon reports that he was previously seeing framework failures due to an attempt to read the non-existing T13456.stderr. While I don't know exactly what this is due to, it does seem like a non-existing .std{out,err} file should be equivalent to an empty file. Teach the testsuite driver to treat it as such.
* testsuite: Move no_lint to the top level, tweak hie002Ömer Sinan Ağacan2020-04-101-0/+6
| | | | | | | | | | | | | | | | | | | | | | | - We don't want to benchmark linting so disable lints in hie002 perf test - Move no_lint to the top-level to be able to use it in tests other than those in `testsuite/tests/perf/compiler`. - Filter out -dstg-lint in no_lint. - hie002 allocation numbers on 32-bit are unstable, so skip it on 32-bit Metric Decrease: hie002 ManyConstructors T12150 T12234 T13035 T1969 T4801 T9233 T9961
* testsuite: Make hasMetricsFile RHS more descriptiveBen Gamari2020-02-261-1/+1
|
* testsuite: Allow tests to be marked as broken on the command lineBen Gamari2020-02-263-1/+13
| | | | This allows us to work-around distribution-specific breakage easily.
* testsuite: Fix symlink testBen Gamari2020-02-261-1/+1
| | | | Needs to `write` bytes, not str.
* Fix testsuite driver output (#17847)Vladislav Zavialov2020-02-181-1/+1
|
* testsuite: Probe whether symlinks are usable on WindowsBen Gamari2020-02-161-1/+23
| | | | Closes #17706.
* testsuite: Sort test names in expected change outputBen Gamari2020-02-151-3/+3
|
* testsuite: Assert the opsys names are knownBen Gamari2020-02-141-0/+11
| | | | | | Previously opsys would take any string. This meant it was very easy for a typo to silently render the predicate ineffective. Fix this by checking the given operating system name against a list of known values.
* testlib: Extend existing *_opts in extra_*_optsÖmer Sinan Ağacan2020-02-051-4/+6
| | | | | | | | | | | | Previously we'd override the existing {run,hc} opts in extra_{run,hc}_opts, which caused flakiness in T1969, see #17712. extra_{run,hc}_opts now extends {run,hc} opts, instead of overriding. Also we shrank the allocation area for T1969 in order to increase residency sampling frequency. Fixes #17712
View Lorry Controller Status