summaryrefslogtreecommitdiff
path: root/hadrian/src/Settings/Builders/RunTest.hs
Commit message (Collapse)AuthorAgeFilesLines
* ci: Try not buffering python outputMatthew Pickering2021-11-031-1/+2
|
* ci: test in-tree compiler in hadrianZubin Duggal2021-10-131-8/+2
|
* hadrian, testsuite: Teach Hadrian to query the testsuite driver for dependenciesZubin Duggal2021-10-131-11/+19
| | | | Issues #19072, #17728, #20176
* hadrian: Add exe suffix to executables in testsuiteMatthew Pickering2021-07-271-4/+4
|
* hadrian: Pass correct leading_underscore configuration to testsMatthew Pickering2021-06-231-0/+2
|
* Remove dynamic-by-default (#16782)Sylvain Henry2021-04-071-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Dynamic-by-default was a mechanism to automatically select the -dynamic way for some targets. It was implemented in a convoluted way: it was defined as a flavour option, hence it couldn't be passed as a global settings (which are produced by `configure` before considering flavours), so a build system rule was used to pass -DDYNAMIC_BY_DEFAULT to the C compiler so that deriveConstants could infer it. * Make build system has it disabled for 8 years (951e28c0625ece7e0db6ac9d4a1e61e2737b10de) * It has never been implemented in Hadrian * Last time someone tried to enable it 1 year ago it didn't work (!2436) * Having this as a global constant impedes making GHC multi-target (see !5427) This commit fully removes support for dynamic-by-default. If someone wants to reimplement something like this, it would probably need to move the logic in the compiler. (Doing this would probably need some refactoring of the way the compiler handles DynFlags: DynFlags are used to store and to pass enabled ways to many parts of the compiler. It can be set by command-line flags, GHC API, global settings. In multi-target GHC, we will use DynFlags to load the target platform and its constants: but at this point with the current DynFlags implementation we can't easily update the existing DynFlags with target-specific options such as dynamic-by-default without overriding ways previously set by the user.)
* Reduce default test verbosityMatthew Pickering2021-01-281-1/+8
|
* testsuite: Add --top flag to driverGHC GitLab CI2020-11-021-1/+1
| | | | | This allows us to make `config.top` a proper Path. Previously it was a str, which caused the Ghostscript detection logic to break.
* hadrian: Don't quote metric baseline argumentBen Gamari2020-10-281-1/+1
| | | Previously this was quoted inappropriately.
* Don't quote argument to Hadrian's test-env flag (#18656)Ryan Scott2020-09-121-1/+1
| | | | | | | Doing so causes the name of the test environment to gain an extra set of double quotes, which changes the name entirely. Fixes #18656.
* testsuite: Only run llvm ways if llc is availableBen Gamari2020-08-181-3/+0
| | | | | | | | | As noted in #18560, we previously would always run the LLVM ways since `configure` would set `SettingsLlcCommand` to something non-null when it otherwise couldn't find the `llc` executable. Now we rather probe for the existence of the `llc` executable in the testsuite driver. Fixes #18560.
* gitlab-ci: Use MR base commit as performance baselineBen Gamari2020-08-181-0/+4
|
* Update HadrianSylvain Henry2020-06-171-0/+4
| | | | | | | | | | | | | | | * support ghc-bignum backend selection in flavours and command-line * support ghc-bignum "--check" flag (compare results of selected backend against results of the native one) in flavours and command-line (e.g. pass --bignum=check-gmp" to check the "gmp" backend) * remove the hack to workaround #15286 * build GMP only when the gmp backend is used * remove hacks to workaround `text` package flags about integer-*. We fix `text` to use ghc-bignum unconditionally in another patch
* hadrian: Tell testsuite driver about LLVM availabilityBen Gamari2020-05-131-1/+2
| | | | | | This reflects the logic present in the Make build system into Hadrian. Fixes #18167.
* Hadrian: fix --summarySylvain Henry2020-04-091-1/+1
|
* hadrian: Fix executable extension passed to testsuite driverGHC GitLab CI2020-03-221-1/+1
|
* hadrian: Add --broken-test flagBen Gamari2020-02-261-0/+2
| | | | | This exposes the flag of the same name supported by the testsuite driver.
* hadrian: Add --test-metrics argumentBen Gamari2020-02-091-1/+4
| | | | | Allowing the test metric output to be captured to a file, a la the METRIC_FILE environment variable of the make build system.
* testsuite: Make ghc_built_by_llvm check more preciseBen Gamari2019-12-171-4/+0
| | | | | | | | | | Previously it would hackily look at the flavour name to determine whether LLVM was used to build stage2 ghc. However, this didn't work at all with Hadrian and would miss cases like ARM where we use the LLVM backend by default. See #16087 for the motivation for why ghc_built_by_llvm is needed at all. This should catch one of the ARMv7 failures described in #17555.
* hadrian: export METRICS_FILE to make it accessible to perf notes scriptAlp Mestanogullari2019-11-111-2/+1
| | | | | This addresses #17456 and also fixes the --metrics-file argument that Hadrian passes to the testsuite driver.
* hadrian: fix support for the recording of perf test resultsAlp Mestanogullari2019-11-071-0/+8
| | | | | | | | | | Before this patch, Hadrian didn't care about the TEST_ENV and METRICS_FILE environment variables, that the performance testing infrastructure uses to record perf tests results from CI jobs. It now looks them up right before running the testsuite driver, and passes suitable --test-env/--metrics-file arguments when these environment variables are set.
* hadrian: Make runtest invocation consistency with MakeBen Gamari2019-11-011-8/+9
| | | | | Use True/False instead of 0/1. This shouldn't be a functional change but we should be consistent.
* Fix formatting of --info's "Debug on" fieldBen Gamari2019-07-181-3/+1
| | | | | | As noted in #16914, the value `True` was used instead of `YES` here, in contrast to the other boolean fields emitted by `--info`. This confused the testsuite driver and broke the `ghc_debugged` testsuite predicate.
* Fix cyclic dependencies when using --configureAndrey Mokhov2019-06-251-4/+2
| | | | | | | | | | | This resolves #16809 (https://gitlab.haskell.org/ghc/ghc/issues/16809). This patch removes the unnecessary dependency on configure-generated flags `windowsHost`, `osxHost` and `iosHost`, using the information provided by the module `System.Info` instead. We also take care to use the `CrossCompiling` flag generated by the configure script only after the latter had a chance to run.
* testsuite: introduce 'static_stats' testsAlp Mestanogullari2019-05-291-1/+5
| | | | | | | | | | | | | | | | They are a particular type of perf tests. This patch introduces a 'stats_files_dir' configuration field in the testsuite driver where all haddock timing files (and possibly others in the future) are assumed to live. We also change both the Make and Hadrian build systems to pass respectively $(TOP)/testsuite/tests/perf/haddock/ and <build root>/stage1/haddock-timing-files/ as the value of that new configuration field, and to generate the timing files in those directories in the first place while generating documentation with haddock. This new test type can be seen as one dedicated to examining stats files that are generated while building a GHC distribution. This also lets us get rid of the 'extra_files' directives in the all.T entries for haddock.base, haddock.Cabal and haddock.compiler.
* Hadrian: add --test-root-dirs, to only run specific directories of testsAlp Mestanogullari2019-05-221-3/+5
| | | | | | | | | We can specify several of those, by using the flag multiple times or just once but combining the directories with ':'. Along the way, this patch also fixes the testsuite-related --only flag, so that we can use it many times instead of being force to specify a space-separated list of test in a single --only flag.
* Hadrian: use the testsuite driver's config.haddock arg more correctlyAlp Mestanogullari2019-04-221-1/+15
| | | | | | | | | | | | | | | 4 haddock tests assume that .haddock files have been produced, by using the 'req_haddock' modifier. The testsuite driver assumes that this condition is satisfied if 'config.haddock' is non-empty, but before this patch Hadrian was always passing the path to where the haddock executable should be, regardless of whether it is actually there or not. Instead, we now pass an empty config.haddock when we can't find all of <build root>/docs/html/libraries/<pkg>/<pkg>.haddock>, where <pkg> ranges over array, base, ghc-prim, process and template-haskell, and pass the path to haddock when all those file exists. This has the (desired) effect of skipping the 4 tests (marked as 'missing library') when the docs haven't been built, and running the haddock tests when they have.
* Hadrian: fix the value we pass to the test driver for config.compiler_debuggedAlp Mestanogullari2019-04-181-3/+5
| | | | | We used to pass YES/NO, while that particular field is set to True/False. This happens to fix an unexpected pass, T9208.
* Hadrian: add a --test-accept/-a flag, to mimic 'make accept'Alp Mestanogullari2019-04-121-0/+8
| | | | | | | | | | | | | | | | | When -a or --test-accept is passed, and if one runs the 'test' target, then any test failing because of mismatching output and which is not expected to fail will have its expected output adjusted by the test driver, effectively considering the new output correct from now on. When this flag is passed, hadrian's 'test' target becomes sensitive to the PLATFORM and OS environment variable, just like the Make build system: - when the PLATFORM env var is set to "YES", when accepting a result, accept it for the current platform; - when the OS env var is set to "YES", when accepting a result, accept it for all wordsizes of the current operating system. This can all be combined with `--only="..."` and `TEST="..." to only accept the new output of a subset of tests.
* Update Trac ticket URLs to point to GitLabRyan Scott2019-03-151-1/+1
| | | | | This moves all URL references to Trac tickets to their corresponding GitLab counterparts.
* Hadrian: remove unneeded imports.David Eichmann2019-03-151-2/+0
|
* Hadrian: various improvements around the 'test' ruleAlp Mestanogullari2019-03-081-49/+74
| | | | | | | | | | | | | | - introduce a -k/--keep-test-files flag to prevent cleanup - add -dstg-lint to the options that are always passed to tests - infer library ways from the compiler to be tested instead of getting them from the flavour (like make) - likewise for figuring out whether the compiler to be tested is "debugged" - specify config.exeext - correctly specify config.in_tree_compiler, instead of always passing True - fix formatting of how we pass a few test options - add (potential) extensions to check-* program names - build check-* programs with the compiler to be tested - set TEST_HC_OPTS_INTERACTIVE and PYTHON env vars when running tests
* Hadrian: enable -Wcompat=error in the testsuiteVladislav Zavialov2019-02-151-0/+1
|
* Revert "hadrian: Pass integer_backend to testsuite driver"Ben Gamari2018-12-241-2/+0
| | | | This reverts commit 442be690e1ca162e22d632df695c29f90625f33e.
* testsuite: Mark th tests as broken in ext-interp way in LLVM build flavoursBen Gamari2018-12-241-0/+4
| | | | | | | This is due to the failures documented in #16087. The condition here could be improved as it matches on `BUILD_FLAVOUR` instead of looking at the compiler flags. However, it's better than nothing and I hope we will be able to fix these issues before long.
* hadrian: Pass integer_backend to testsuite driverBen Gamari2018-12-141-0/+2
|
* Hadrian: ability to run a subset of the testsuiteAlp Mestanogullari2018-12-111-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was supposed to be working already but didn't work when we specified several tests with --only. This patch not only fixes this but also makes it possible to specify a subset of tests to run with the TEST environment variable, like the make build system. Here are some examples: hadrian/build.sh test --only=plugins01 hadrian/build.sh test --only="plugins01 plugins02" TEST="plugins01 plugins02" hadrian/build.sh test TEST=plugins03 hadrian/build.sh test --only="plugins01 plugins02" When both the TEST environment variable and the --only flag are used, we simply concatenate the list of tests from both sources and ask the testsuite driver to run them all. This patch addresses #16026. Test Plan: hadrian/build.sh test --only="plugins01 plugins02" Reviewers: bgamari, snowleopard Reviewed By: bgamari, snowleopard Subscribers: rwbarton, carter GHC Trac Issues: #16026 Differential Revision: https://phabricator.haskell.org/D5431
* Add 'hadrian/' from commit '45f3bff7016a2a0cd9a5455a882ced984655e90b'Ben Gamari2018-10-231-0/+205
| | | | | | git-subtree-dir: hadrian git-subtree-mainline: 575b35f4cdc18045bccd42d341d6f25d95c0696c git-subtree-split: 45f3bff7016a2a0cd9a5455a882ced984655e90b
* Remove Hadrian submoduleBen Gamari2018-10-231-7/+0
|
* Bump hadrian submoduleBen Gamari2018-08-051-21/+5
|
* Bump hadrian submoduleBen Gamari2018-06-121-5/+21
|
* Update hadrian submoduleBen Gamari2018-06-071-16/+6
|
* Update Hadrian submoduleAndrey Mokhov2018-04-261-11/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Link to Quick Start guide * Update README.md (hadrian/578) * Fix AppVeyor (hadrian/577) * Fix CircleCI * Generic library rules (hadrian/571) * Fix lint error (hadrian/575) * Fix missing libHSghc-8.5-0.a (hadrian/574) * Fix the path to touchy (hadrian/572) * Fix integer-gmp build (hadrian/568) * Undo fs*.h workaround * Fix copying of fs*.h files during RTS registration (hadrian/566) * Fix Windows build, improve error reporting (hadrian/565) * Fix Windows build (hadrian/563) * Fix boot and configure on AppVeyor (hadrian/561) * Preliminary bindist (hadrian/558, hadrian/555) * Unregister stage0 package first if it needs to be cloned (hadrian/552) * Fix Circle CI (hadrian/553) * Fix warnings (hadrian/547) * Merge pull request hadrian/542 from Mistuke/fix-specific-file * Use Cabal directly in place of ghc-cabal + make build root configurable (hadrian/531) * Add user-defined flavour example for turning off dynamic linking (hadrian/535) * Add clean routines for fs (hadrian/533) * Add 'git' to nativeBuildInputs in shell.nix (hadrian/530) * Add extra include paths when invoking ghc-cabal (hadrian/526) * Merge pull request hadrian/528 from snowleopard/bump-cabal * Merge pull request hadrian/521 from snowleopard/drop-chmod * Change permission bits for build.cabal.sh, fixes hadrian/517 (hadrian/520) * Pin nixpkgs and all-cabal-hashes in shell.nix (hadrian/511) * Add troubleshooting section
* Update Hadrian submoduleAndrey Mokhov2018-02-241-22/+8
| | | | | | | | | | | | | | | * Make shell.nix less broken (hadrian/510) * Add --configure flag to the script * Undo iserv changes (hadrian/507) * Fix ghc-cabal: Parsec modules are now found in libraries/parsec/src (hadrian/506) * Move a bunch of types into dedicated modules (hadrian/502) * Add --quickjump to Haddock (hadrian/505) * Add iserv library (hadrian/504) * Merge pull request hadrian/503 from snowleopard/angerman-patch-1 * Merge pull request hadrian/500 from snowleopard/runtime-deps * Fix Hadrian after Cabal changes (hadrian/498) * Drop custom logic for Scav_thr and Evac_thr (hadrian/497) * Fix Haddock (hadrian/496)
* Bump hadrian submoduleBen Gamari2018-01-261-18/+23
|
* Add hadrian as a submoduleBen Gamari2017-12-081-0/+21
It will remain a submodule until we are ready to merge Hadrian into the tree.