summaryrefslogtreecommitdiff
path: root/tests/testutils/runcli.py
Commit message (Collapse)AuthorAgeFilesLines
* Expose basic api for testing external plugins.phil/plugin-testing-apiPhil Dawson2019-02-081-611/+0
| | | | | | | | | | | | | | | | | | We want external plugins to be able to make use of the core testing utils. This commit exposes the basic utilities which are currently in use in bst-external plugins. If necessary, more utilities could be exposed in the future. Moves the following files from tests/testutils/ to buildstream/plugintestingutils/: o runcli.py o integration.py As part of this, this commit makes the following changes to runcli.py and integration.py: o runcli.py: Fix linting errors o runcli.py: Add user facing documentation o Integration.py: Add user facing documentation
* tests/testutils/runcli.py: Make get_element_states() take a list of targetsTristan Van Berkom2019-01-241-3/+2
| | | | | Instead of a single target, we can always provide a single target in a list.
* testutils/runcli.py: Allow removing artifacts from arbitrary dirsTristan Maat2019-01-181-1/+7
| | | | | `remove_artifact_from_cache` used a hard-coded path to remove artifacts, which wasn't sufficient for integration tests.
* testutils/runcli.py: Added cli.get_element_states()Tristan Van Berkom2019-01-181-0/+22
| | | | | | | | | With get_element_state(), you need to invoke BuildStream once for every element state you want to observe in a pipeline. The new get_element_states() reports a dictionary with the element state hashed by element name and is better to use if you have more than one element to observe the state of.
* conftest.py: Use different artifact directory for integration testsTristan Van Berkom2019-01-161-2/+2
| | | | | | | | To ensure we can run integration tests in parallel, use a tempdir for the artifact cache of each separate integration test run. This makes it possible to run multiple full test runs including integration tests in parallel under detox.
* testutils/runcli.py: Added Result.get_start_order()Tristan Van Berkom2019-01-101-0/+17
| | | | | Added a function to report the list of elements which appeared in a given queue in the order of their first appearance.
* tests/testutils/runcli.py: Fixed broken environment handlingTristan Van Berkom2018-12-261-2/+5
| | | | | Treat None values in the passed dictionary as keys to be removed from the environment, this was already happening at restoration time.
* Basic options for shell --build to use buildtreesWilliam Salmon2018-12-191-0/+14
| | | | Fixes issue #740
* tests/testutils/runcli.py: Support binary-mode capture of stdoutDaniel Silverstone2018-10-261-6/+7
| | | | | | | | In order to test things which write tarballs to stdout correctly, we need to capture the binary output cleanly. This ensures we're not potentially mismatching encodings in and out. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* cascache.py: Move push/pull messaging to cascacheJosh Smith2018-09-191-1/+1
| | | | | | | | Pulled/Pushed messages will no longer be produced from within element.py, instead they will be produced during CasCache push() and pull() appropriately. Message consistency has also been improved.
* Add tests for cyclic variables checkJosh Smith2018-08-291-2/+16
| | | | | Note: This modifies the docker containers used for testing to supply the pytest-timeout package.
* tests/integration/source-determinism.py: Use cli_integration.valentindavid/cli_integration_source_determinismValentin David2018-08-241-2/+4
|
* _platform: Use CAS artifact cacheJürg Billeter2018-07-171-6/+1
|
* tests: Do not rely on 'downloadable' statusJürg Billeter2018-05-111-1/+0
| | | | This is in preparation for moving away from summary files.
* tests/testutils/runcli.py: Improve detection of pulled/pushed elementsJürg Billeter2018-04-301-2/+2
|
* _exceptions.py: Adhere to policy on private symbolsTristan Van Berkom2018-04-031-3/+3
| | | | | | | | | | And adjust all surrounding sources for changed symbols. Also, added new LoadErrorReason.UNSUPPORTED_PLUGIN, required for changes in how the project will report format version errors for plugins at creation time This is a part of issue #285
* element.py: Make artifact lowercase in time_activity()James Ennis2018-03-281-1/+1
| | | | | NOTE: artifact was also made lowercase in the function get_pushed_elements() in runcli.py
* Remove unused importsJürg Billeter2018-03-251-1/+1
|
* integration tests: Refactoring for dynamic project configurationsTristan Van Berkom2018-03-061-8/+62
| | | | | | | | | | | | | | o CliIntegration.run() now takes a `project_config` option, this will be composited on top of the existing, substituted project.conf o Removing gnomesdk alias from integration tests project.conf o Using `sysroot` alias instead of `gnome7` alias o Make base-alpine.bst use the `sysroot` alias now o Removed `create_project_config()` from shell.py tests, now use the new built-in feature for this which is cleaner.
* tests/frontend/push.py: Test that we don't push stuff that we just pulledSam Thursfield2018-02-261-0/+14
|
* testutils/runcli.py: Print something helpful if BuildStream fails to exit ↵Tristan Van Berkom2018-02-251-0/+1
| | | | | | | | properly When testing a recent patch, I had missed a return of the exit code in `bst shell` resulting in the CLI exiting with `None`; this patch makes the fixture more helpful, where previously it just printed `None`
* Merge integration tests into general testsTristan Maat2018-02-071-2/+3
|
* Add test utilities for integration testsTristan Maat2018-02-071-24/+90
|
* runcli.py: Allow setting individual config settingsTristan Maat2018-02-071-4/+8
|
* tests/testutils/runcli.py: Assert that buildstream actually exited.Tristan Van Berkom2018-01-061-0/+12
| | | | | | Enhanced the error checking Result() methods to always assert that the CLI actually exited, there are no cases worth testing for where buildstream would be expected to exit on an unhandled exception.
* tests/testutils/runcli.py: Augment the Result object to store task errorsTristan Van Berkom2018-01-011-9/+100
| | | | | | | | | | | | Before this, we only stored the last exception to have been raised in the main process, now we additionally provide some Result members allowing tests to inspect a machine readable error `domain` and `reason` describing why a task has "failed". This adds some new APIs to the Result() object for tests: assert_main_error() - asserts the nature of the main buildstream error assert_task_error() - asserts the nature of the error from a child task
* tests/testutils/runcli.py: Enhanced to capture stdout/stderr separatelyTristan Van Berkom2017-12-131-11/+64
| | | | | We need this now in order to properly test the frontend and parse logs from stderr or output from stdout.
* Fix testsTristan Maat2017-12-131-0/+26
|
* tests/testutils/runcli.py: Always run with --no-colorsSam Thursfield2017-12-051-1/+1
| | | | | | | | | | Pytest defaults to capturing stdout, which has the side effect of making BuildStream realise it's not outputting to a terminal and disabling its ANSI colour code output. If you pass `--capture=no` then BuildStream detects the terminal and the colour codes are emitted, which breaks all of the tests which match on specific output from `bst`. Forcefully disable colours to fix that.
* tests/testutils/runcli.bst: Fix get_pipeline() to not use mutable default ↵Tristan Van Berkom2017-11-181-1/+4
| | | | argument
* load.py: Migrate to new test styleTristan Maat2017-11-101-0/+26
|
* Refactoring: Move exceptions module to be privateTristan Van Berkom2017-11-061-1/+1
| | | | Hide all of buildstream's internal exceptions from the API surface.
* tests/testutils/runcli.py: get_element_state() now uses --downloadableTristan Van Berkom2017-10-291-0/+1
| | | | | | | | This wont take any additional time because if there is a remote artifact share configured in the pytest suite, it can only be a locally created one - this makes the push/pull tests work after recently making downloading of remote refs optional for the sake of issue #140.
* tests/testutils/runcli.py: Optionally make the runner not verboseTristan Van Berkom2017-10-231-8/+10
| | | | This is so that we can use it in other contexts than running tests
* tests/testutils/runcli.py: Provide the last raised exception in the resultTristan Van Berkom2017-10-101-1/+14
|
* Fix tests for other platformsTristan Maat2017-09-281-0/+13
|
* tests/testutils/runcli.py: Now the cli can set the environment and cwd for a runTristan Van Berkom2017-09-051-7/+40
|
* tests/testutils/runcli.py: Print CLI commands unconditionallyTristan Van Berkom2017-09-051-5/+5
| | | | We want to see them in failed test logs even if they succeeded.
* tests/testutils/runcli.py: Added get_element_key() method to fetch cache keyTristan Van Berkom2017-09-051-0/+13
|
* tests/testutils/runcli.py: Added fixture for running the CLITristan Van Berkom2017-09-041-0/+113