summaryrefslogtreecommitdiff
path: root/buildstream
Commit message (Collapse)AuthorAgeFilesLines
* Fix cache corruption by scripts when layout and integration commands are usedvalentindavid/script-artifact-corruptionValentin David2018-11-061-7/+8
| | | | | | | | Root directory was marked as a non-artifact mount, so not using SafeHardLink. However integration commands executed with write access to the root directory. Fixes #749
* _sandboxremote.py: Use the standard SandboxError.jmac/remote-sandbox-errorsJim MacArthur2018-11-061-4/+1
| | | | | Replaces the custom one which was erroneously added during development. Fixes #746.
* utils: Fix _pretty_size() for sizes > 1024TJosh Smith2018-11-061-3/+4
|
* Split up artifact cache and CAS cacheJürg Billeter2018-11-056-387/+531
| | | | | | | | This changes CASCache from a subclass to a delegate object of ArtifactCache. As the lower layer, CASCache no longer deals with elements or projects. Fixes #659.
* element.py: Do not include type name of artifact cache in cache keyJürg Billeter2018-11-051-1/+1
| | | | | | | The artifact cache backend does not affect build outputs and we anyway no longer have pluggable artifact cache backends. This hardcodes CASCache instead of removing the entry completely to avoid cache key changes.
* _project.py: Remove unused artifacts parameter from load_elements()Jürg Billeter2018-11-052-3/+2
|
* element.py: Remove unused artifacts parameter from _new_from_meta()Jürg Billeter2018-11-053-6/+5
|
* _project.py: Remove unused artifacts parameter from create_element()Jürg Billeter2018-11-052-3/+2
|
* _elementfactory.py: Remove unused artifacts parameter from create()Jürg Billeter2018-11-052-4/+3
|
* element.py: Remove artifacts parameter from constructorJürg Billeter2018-11-052-3/+3
| | | | Get the artifact cache from the context instead.
* element.py: Pass bare_directory also to SandboxRemote()juerg/remote-executionJürg Billeter2018-11-031-0/+1
| | | | Fixes: 825963b5 ("element: Make "--sysroot" take a bare directory")
* plugins/sources/pip.py: Accomodate characters '-','.','_' for packagesctolentino82018-11-021-4/+19
|
* _yaml.py: Implement `get()` for `ChainMap`Daniel Silverstone2018-11-011-0/+6
| | | | | | | | | | | | Since the core Python `ChainMap.get()` implements with: self[key] if key in self else default The double-chain-lookup is expensive. This simple change solves that for our ChainMap derived structure. As such it improves matters for #466 somewhat. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* BREAK: manual.yaml: don't set any default env varsAngelos Evripiotis2018-11-012-20/+1
| | | | | | | | | | | | | | | | Remove the 'MAKEFLAGS' and 'V' defaults. Now that there is a specialised 'make' element, these make-specific defaults don't make sense here. This element is meant to be a blank slate for folks to build on. Note that this is a breaking change, that will require some users to make changes to their .bst files if they are expecting these environment variable to be set. _versions.py: BST_FORMAT_VERSION bumped to 18. This fixes issue #718
* _frontend/app.py: Set correct element-path in interactive bst-initchiaratolentino/bst-init-interactive-elementpathctolentino82018-10-311-1/+1
|
* element: Make "--sysroot" take a bare directoryJonathan Maw2018-10-303-8/+22
| | | | | | | | | | | | | | | | | | i.e. instead of taking a directory that must contain "root" and "scratch", and treating "root" as the root, use the directory directly. In element.py: * __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor In sandbox.py: * If bare_sandbox, `_root` is the passed-in directory, and `__scratch` is None. * Trying to use `__scratch` when bare_sandbox is True is a bug. In _mount.py: * Don't get the value of `__scratch` if it's not needed. This is part of #539
* sandbox.py: Remove redundant Sandbox.__directoryJonathan Maw2018-10-301-2/+1
|
* Element: Use cached buildtree in build shells and failure shellsJonathan Maw2018-10-305-21/+12
| | | | | | | | | | | | | | | | | | | | | | | | This includes changes in app.py: * Interactive failure shell no longer uses the failed build sysroot, defaulting to the cached build tree. Changes in element.py are: * Errors caused by building don't store the failed build sysroot, instead storing that a sandbox can be created to debug the error. * When staging sources, will stage the element's cached build tree if it exists. Changes in _exceptions.py: * BstError.sandbox is now a flag of whether a sandbox can be opened up to debug the error. Changes in widget.py: * Don't try to print any information about the sandbox. Changes in _message.py: * Fix documentation so Message.sandbox is not a directory any more. This is part of #539
* element.py: Always clean up the rootdirJonathan Maw2018-10-301-2/+2
| | | | | | | | | | | | | We shouldn't need it to persist now that we cache failed build dirs. This change breaks the test `tests/integration/shell.py::test_sysroot_workspace_visible`. I can no longer see a use-case for this test. AIUI, it tested that the failed build sysroot stored in the builddir has the workspace's files in, despite the workspace being unmounted. I believe this behaviour is made redundant by cached buildtrees. This fixes part of #539
* _stream.py: Preserve stdout FD across checkout-to-stdoutDaniel Silverstone2018-10-261-3/+10
| | | | | | | | In order to support things which might need to use the stdout FD after checkout has completed writing a tarball to it, preserve it by means of `os.dup()` and `os.dup2()` Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* various: Move _sentinel from utils.py to _yaml.pyDaniel Silverstone2018-10-264-18/+7
| | | | | | | | | The `_sentinel` in `utils.py` was used only for yaml node tracking. As such, simplify matters by removing it from `utils.py` and move it to `_yaml.py` which means that we no longer need to try and avoid a circular import issue by means of runtime importing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _loader/loadelement.py: Use enumerate() in dependency extractiondanielsilverstone-ct/use-enumerateDaniel Silverstone2018-10-261-5/+3
| | | | | | | | | In cases where elements end up with large numbers of dependencies, the n-squared performance in `_extract_depends_from_node` can produce significant delays during project load. This patch corrects that. In addition we were double-retrieving provenance which was unnecessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Check if command is a str and replace by list before checking existenceBenjamin Schubert2018-10-263-8/+12
| | | | | | | | Currently we would make sure the sandbox had a command before converting it to a list if it was given as a string. That meant that a string command would never exist and the check be invalid. This also adds the same logic in the dummy sandbox for consistency.
* _yaml.py: Do not insert into cache if retrieved from cacheDaniel Silverstone2018-10-261-3/+2
| | | | | | | | We were unconditionally inserting into the yaml cache anything we had retrieved from the yaml cache. This had the side-effect of being expensive on re-runs of commands where yaml hadn't changed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* build elements: Add reference to BuildElement's built-in configchandan/source-doc-directoryChandan Singh2018-10-2512-0/+35
| | | | | | | | | | | For each element plugin derived from BuildElement, add reference to the built-in functionality documentation of BuildElement. * buildelement.py: Add anchor for built-in functionality. * plugins/elements/*.py: Add refence to BuildElement's built-in functionality for each build element. Related to #713.
* sources: Don't redundantly document directory configurationChandan Singh2018-10-2511-34/+28
| | | | | | | | | | | | Instead of redundantly documenting "directory" configuration option in each source plugin, refer to the built-in functionality documentation of the Source class. * source.py: Add anchor for built-in functionality. * sources/plugins/*.py: Remove directory config option, and add reference to Source's built-in functionality. Fixes #713.
* _artifactcache/cascache.py: Cleanup lint warning about indexingDaniel Silverstone2018-10-251-2/+3
| | | | | | | In order to quieten the linter, and to make a more correct comparison, we switch to using the `Code.OK` enumeration from `google.rpc.Code` Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _artifactcache/cascache.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _artifactcache/cascache.py: Prepare attributes in __init__Daniel Silverstone2018-10-251-0/+4
| | | | | | | Python linters prefer that attributes are defined in __init__ rather than left to later routines. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _artifactcache/cascache.py: Silence len(SEQUENCE) as condition warningsDaniel Silverstone2018-10-251-3/+3
| | | | | | | The lint tool dislikes the use of `if len(SEQ) == 0` type statements because nominally `len(SEQ)` may be more expensive than `bool(SEQ)`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _artifactcache/cascache.py: Refactor slightly for indentationDaniel Silverstone2018-10-251-3/+4
| | | | | | | Correct some multi-line conditional indentation by slight refactor if necessary. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxchroot.py: Silence warning about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | Sometimes `dict.get()` is preferable to an `if` statement, but this time it's clearer if we keep the structure. As such, silence the warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxchroot.py: Silence warning about preexec_fnDaniel Silverstone2018-10-251-1/+1
| | | | | | | We are super-careful to not use threads in places where we might use Popen and as such this warning is save to quash. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxchroot.py: Silence warning about unused variableDaniel Silverstone2018-10-251-1/+0
| | | | | | The variable was unused, remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxbwrap.py: Disable lint about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | Sometimes `dict.get()` is preferable, in this instance it's less clear so we'll disable that lint Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_mounter.py: Remove useless inheritance on `object`Daniel Silverstone2018-10-251-1/+1
| | | | | | In Python 3, inheriting from `object` explicitly is pointless. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_mount.py: Do not use dict literals in argument defaultsDaniel Silverstone2018-10-251-3/+6
| | | | | | | The use of dictionary literals in argument defaults is disrecommended due to the way that they are static and thus potentially very confusing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* plugins/source/git.py: Rationalised unused return valuesDaniel Silverstone2018-10-251-2/+2
| | | | | | | The `assert_ref_in_track()` method's return value was never used so rationalise it to never return a value. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _scheduler/queue/trackqueue.py: Remove unused importDaniel Silverstone2018-10-251-1/+0
| | | | | | The `SourceError` import was never used Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _scheduler/queue/pullqueue.py: Remove unused return valueDaniel Silverstone2018-10-251-1/+1
| | | | | | The `done()` method does not return anything anywhere else. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _scheduler/jobs/job.py: Correct lint issue around abstract methodDaniel Silverstone2018-10-251-1/+1
| | | | | | | The linter cannot detect abstract methods the way we use them, so this silences the warning which ensues. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _platform/linux.py: Simplify return statementDaniel Silverstone2018-10-251-5/+1
| | | | | | | The linter dislikes an if/return/else/return where it can be simplified to a return of the if condition. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _platform/linux.py: Remove unused importsDaniel Silverstone2018-10-251-1/+0
| | | | | | The imports of `Message` and `MessageType` were unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _platform/darwin.py: Remove unused importsDaniel Silverstone2018-10-251-2/+0
| | | | | | | The import of `resource` was unused. The import of `PlatformError` was unused. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _platform/darwin.py: Remove useless __init__()Daniel Silverstone2018-10-251-4/+0
| | | | | | | An otherwise empty __init__ passing no arguments to super.__init__ is worthless. Remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _options/optionbool.py: Refactor conditionsDaniel Silverstone2018-10-251-2/+2
| | | | | | | The pylint tool prefers that conditions of the form "var == foo or var == bar" be refactored into "var in (foo, bar)". Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _fuse/mount.py: Deal with abstract method lint issueDaniel Silverstone2018-10-251-1/+1
| | | | | | | | Unfortunately pylint can't tell this is an abstract method which will only ever end up invoked from a class which concretises it. As such we need to silence this warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _fuse/{hardlinks,mount}.py: Remove dangerous use of {} in defaultsDaniel Silverstone2018-10-252-3/+5
| | | | | | | | Since default values in arguments to functions and methods are created once at the compilation of the code, it is dangerous to include list and dict literals in them. This changes the use of {} to None. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _frontend/status.py: Reorder importsDaniel Silverstone2018-10-251-1/+1
| | | | | | | The `curses` import is "standard" whereas `click` is not. As such, despite alphabetical order normally being the rule, `curses` must come before `click`. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _frontend/app.py: Remove unused import of `resource`Daniel Silverstone2018-10-251-1/+0
| | | | | | The `resource` import was unused. Removed. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
View Lorry Controller Status