summaryrefslogtreecommitdiff
path: root/buildstream
Commit message (Collapse)AuthorAgeFilesLines
* Workspace CLI updateWilliam Salmon2018-11-214-61/+104
| | | | | | | This is to update the workspace CLI to as agreed on the mailing list https://mail.gnome.org/archives/buildstream-list/2018-September/msg00046.html This patch also introduces the default workspace directory.
* utils.py: improve documentation of move_atomicBenjamin Schubert2018-11-211-2/+8
|
* utils.py: Ensure move_atomic booleans are keyword only argumentsBenjamin Schubert2018-11-211-1/+1
| | | | This is to be coherent with the rest of the codebase
* _yaml.py: Fix incorrect error messageJames Ennis2018-11-201-1/+7
| | | | | | This patch ensures that we receive an appropriate error message if we specify an absolute path that leads within the project.
* _sandboxremote.py: Add sigterm handler that sends CancelOperation725-job-cancellation-on-remote-buildsRaoul Hidalgo Charman2018-11-201-2/+29
|
* Add prompt.workspace-... optionsAngelos Evripiotis2018-11-203-3/+34
| | | | | | | | | | Provide options in project.conf to disable the 'Are you sure ...' prompts when making destructive changes: - Add prompt.really-workspace-close-remove-dir - Add prompt.really-workspace-reset-hard Add a NEWS item for these.
* Add prompt.auto-init buildstream.conf optionAngelos Evripiotis2018-11-203-4/+39
| | | | | | | | | | | | | | | Provide an option in buildstream.conf to disable the 'Would you like to ...' prompt when we cannot resolve a project. Some users prefer not to be interrupted by such prompts, so pave the way to creating options to disable all those that might get in the way. Follow the example of the advice.* options 'git-config', and create a namespace for these UI options grouped by behaviour, rather than an over-reaching 'ui.*' namespace. In later work perhaps we'll also add 'advice.*' options. Add a NEWS item for this.
* _context: refactor, extract _node_get_option_strAngelos Evripiotis2018-11-201-8/+29
| | | | | | | | | Use a new helper function to simplify working with nodes that can only accept certain strings. This will be used when adding the prompt.* config options. In later work we can see if this function would be useful elsewhere, and could be added to '_yaml.py'.
* _context: allow 'terminate' for scheduler.on-errorAngelos Evripiotis2018-11-201-1/+1
| | | | | | | | | Enable this option of 'terminate', which is mentioned in userconfig.yaml and handled in _frontend/app.py:_handle_failure(). It appears to have been left out of the valid_actions as an oversight. Originally introduced in https://gitlab.com/BuildStream/buildstream/commit/2622d5da9bd9fefd87436613d6e3e9770fdd0f28
* _yaml: document node_get()'s default_value argAngelos Evripiotis2018-11-201-0/+1
|
* _context.py: Drop duplicated default values for user configurationJürg Billeter2018-11-191-13/+15
| | | | | | The default values are in userconfig.yaml, together with the documentation. The default values should not be duplicated in _context.py.
* CasBasedDirectory: Change constructor to take a CASCache instead of a ContextRichard Maw2018-11-193-9/+7
| | | | | The Context was only used to obtain a reference to the CASCache and set the unused cas_directory field.
* element.py: Only cache on element error or successBenjamin Schubert2018-11-191-19/+6
| | | | | | Reverse the way we were handling caching by only caching when we know it's a problem with the build or the build is a success in order not to cache transient errors.
* Don't cache sandbox errorsBenjamin Schubert2018-11-191-89/+103
| | | | | | | | | | Sandbox errors (like missing host tools) are dependent on the host system and rarely on what is actually done. It is therefore better to not cache them as they are subject to change between two runs. Also add test to ensure sandbox failure are not cached
* plugins/elements/cmake.yaml: always specify variable typesAbderrahim Kitouni2018-11-191-1/+1
| | | | | | | cmake sometimes misinterprets relative paths as relative to the current directory if this is not specified. See freedesktop-sdk/freedesktop-sdk#431 adjust tests/format/variables.py accordingly.
* Add `bst source-checkout` commandChandan Singh2018-11-193-27/+151
| | | | | | | | | | | | As discussed in https://mail.gnome.org/archives/buildstream-list/2018-September/msg00064.html, add `bst source-checkout` command. This will allow users to checkout sources for a given target. * _frontend/cli.py: Add source-checkout command * _pipeline.py: Add assert_sources_cached() method * _stream.py: Add source_checkout method, abstract out __check_location_writable() method that used to part of checkout()
* buildstream/_pipeline.py: Simplify error message for assert_consistent()Chandan Singh2018-11-191-1/+1
| | | | | | | | | | | | | | | Currently, `Pipeline.assert_consistent()` prints an error message like the one below when certain elements have inconsistent sources: ``` ... Element: hello.bst is inconsistent Source tar source at hello.bst [line 16 column 2] is missing ref ... ``` Drop the word "Source" from the beginning of the message as "Source <source-kind> source ..." is kind of awkward to read.
* sources/pip.py: Use move_atomic instead of manual os.renameBenjamin Schubert2018-11-191-7/+7
| | | | | This uses move_atomic insteand of the manual os.rename and manual error checking and throws a SourceError for consistency with other modules.
* cascache.py: use move_atomic instead of manual error checkingBenjamin Schubert2018-11-191-10/+5
|
* Extract atomic move function to utils.pyBenjamin Schubert2018-11-192-19/+48
| | | | | | | | Moving atomically a file/directory can be tricky since different errors might be raised for the same underlying problem. Having a utility function to reduce this discrepancies will help in ensuring we have correct behavior
* Fix os.rename in git source element to correctly handle error codesBenjamin Schubert2018-11-191-15/+18
| | | | | | | | | | | | | According to the documentation (https://www.unix.com/man-page/POSIX/3posix/rename/), when the directory already is there, either EEXIST or ENOTEMPTY could be thrown. Previously only ENOTEMPTY was checked. Done: - Separated the move into its own function - Check for both errors - Create unit tests for it, covering most test cases
* plugins/sources/pip.py: also look for python version named "python"Benjamin Schubert2018-11-191-0/+1
| | | | | | | | In some virtual environments, we might not have the binary with the exact python version, which would lead the current pip plugin to fail to find a working pip version. This adds "python" at the start of the list of valid python versions
* Don't pull artifact build trees by default.tpollard/494Tom Pollard2018-11-173-53/+214
| | | | | | | | | | | | | | | | | | | | | | The addition of cached build trees being included in element artifacts has led to mostly redundant download overheads when pulling from a remote artifact server. As such the default behaviour of pull shouldn't fetch the build tree object if available. element.py: extend relevant pull logic for specified subdir consideration and ensure push logic does not lead to partial artifact pushes. Change assumption that the buildtree can be extracted if artifact is cached. __cached_buildtree() and __pull_directories() helpers added. _artifactcache/: artifactcache.py & cascache.py inclusion of helper functions for subdir artifact checking & fetching, fetch logic extended to only pull required artifact directories. extract & checkout updated to handle full/partial operation. tests/: addition of integration test pullbuildtrees.py, buildtrees.py adapted cli options, testutils/artifactshare.py has_artifact changed to return artifact digest on true condition.
* Add cli main and user config option for 'pull-buildtrees' context.Tom Pollard2018-11-174-2/+14
| | | | | | | | | | | | _context.py: Add pull_buildtrees global user context, the default of which is set to False via the addition of pull-buildtrees to userconfig.yaml cache group. _frontend/app.py & cli.py: Add --pull-buildtrees as a bst main option, which when passed will override the default or user defined context for pull_buildtrees. tests/completions/completions.py: Update for the added flag.
* buildstream/sandbox/_sandboxbwrap.py: Distinguish sandbox failure from ↵Richard Maw2018-11-142-20/+52
| | | | | | | | | | | command failure If `bwrap` fails to set up the sandbox and start the payload command it won't write an exit-code in --json-status-fd, so we can report if it was a sandboxing failure if we don't get exit-code status and a payload command failure if we do and it's non-zero. Closes https://gitlab.com/BuildStream/buildstream/issues/286
* _platform/linux.py: Move get_bwrap_version into _site.pyRichard Maw2018-11-142-20/+23
| | | | | It's inconvenient to have to create a Linux platform to parse the bwrap version and we want to get the version in a consistent manner.
* _project.py: Validate nodes early in Project._loadPhillip Smyth2018-11-121-10/+16
| | | | | | _project.py: Added validate_nodes() helper function to prevent duplicate lists element-path was being used before node validation resulting in uncaught errors
* source.py: don't let StopIteration propagate to silence() contextmanagerbschubert/fix-silence-stopiteration-pep-0479Benjamin Schubert2018-11-091-20/+22
| | | | | | As per PEP 0479 (https://www.python.org/dev/peps/pep-0479/), StopIteration thrown in context managers are not valid starting from Python 3.7.
* _casbaseddirectory: Fix order of list_modified_pathsJim MacArthur2018-11-091-9/+26
| | | | This will now return paths in the same order as list_relative_paths.
* _casbaseddirectory.py: Enable direct CAS-to-CAS import.Jim MacArthur2018-11-091-85/+355
| | | | Fixes issue #574.
* plugin.py: Redirect DEBUG from `__del__` to `sys.stderr`danielsilverstone-ct/plugin-destroy-debug-to-stderrDaniel Silverstone2018-11-091-1/+2
| | | | | | | | | Previously, debug messages from `Plugin.__del__()` went to stdout which was a problem for the test suite in extremely rare and hard to predict circumstances. This corrects that by sending the message to stderr where all the rest of the messages tend to go. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* artifactcache.py: Fix misleading error message when using % cache quotajennis/quota_declaration_fixJames Ennis2018-11-081-6/+13
| | | | | Due to the changed Exception message, this patch also changes the test_parse_size_over_1024T test in misc.py
* Fix bug with root mounted as non-artifact in script plugin.Valentin David2018-11-081-2/+5
| | | | The issue was introduced by 6ccfab0b1b25990e406446d5cbe5aee83a5e158a.
* Test that helpful messages are raised when missing dependenciesBenjamin Schubert2018-11-081-1/+2
| | | | | This adds a `reason` to the SandboxEror thrown in sandboxdummy to be able to understand where the error comes from
* Move bwrap checks in platform/linux.pyBenjamin Schubert2018-11-082-48/+30
| | | | | | | | Remove the bwraps checks from _site.py and put them in platform.linux which is the only place where they are run. This allows the removal of a double level of caching, making reasoning about tests easier
* Fix infinite recursion in default strip debug commandValentin David2018-11-081-0/+5
| | | | Fixes #645.
* Fix cache corruption by scripts when layout and integration commands are usedValentin David2018-11-071-7/+8
| | | | | | | | Root directory was marked as a non-artifact mount, so not using SafeHardLink. However integration commands executed with write access to the root directory. Fixes #749
* element.py: Use _force_rmtree instead of custom code.jmac/make_writable_combinationJim MacArthur2018-11-071-10/+3
| | | | | There are arguments for and against using onerror - see issue #735 for details.
* _sandboxremote.py: Use the standard SandboxError.jmac/remote-sandbox-errorsJim MacArthur2018-11-061-4/+1
| | | | | Replaces the custom one which was erroneously added during development. Fixes #746.
* utils: Fix _pretty_size() for sizes > 1024TJosh Smith2018-11-061-3/+4
|
* Split up artifact cache and CAS cacheJürg Billeter2018-11-056-387/+531
| | | | | | | | This changes CASCache from a subclass to a delegate object of ArtifactCache. As the lower layer, CASCache no longer deals with elements or projects. Fixes #659.
* element.py: Do not include type name of artifact cache in cache keyJürg Billeter2018-11-051-1/+1
| | | | | | | The artifact cache backend does not affect build outputs and we anyway no longer have pluggable artifact cache backends. This hardcodes CASCache instead of removing the entry completely to avoid cache key changes.
* _project.py: Remove unused artifacts parameter from load_elements()Jürg Billeter2018-11-052-3/+2
|
* element.py: Remove unused artifacts parameter from _new_from_meta()Jürg Billeter2018-11-053-6/+5
|
* _project.py: Remove unused artifacts parameter from create_element()Jürg Billeter2018-11-052-3/+2
|
* _elementfactory.py: Remove unused artifacts parameter from create()Jürg Billeter2018-11-052-4/+3
|
* element.py: Remove artifacts parameter from constructorJürg Billeter2018-11-052-3/+3
| | | | Get the artifact cache from the context instead.
* element.py: Pass bare_directory also to SandboxRemote()juerg/remote-executionJürg Billeter2018-11-031-0/+1
| | | | Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
* plugins/sources/pip.py: Accomodate characters '-','.','_' for packagesctolentino82018-11-021-4/+19
|
* _yaml.py: Implement `get()` for `ChainMap`Daniel Silverstone2018-11-011-0/+6
| | | | | | | | | | | | Since the core Python `ChainMap.get()` implements with: self[key] if key in self else default The double-chain-lookup is expensive. This simple change solves that for our ChainMap derived structure. As such it improves matters for #466 somewhat. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
View Lorry Controller Status