summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* BuildElement: Fix logging regressiontristan/fix-logging-regressionTristan van Berkom2020-09-261-2/+3
| | | | | Staging activities are usually silent nested, this regressed when porting to recently changed APIs.
* ScriptElement: Fix logging regressionTristan van Berkom2020-09-261-2/+3
| | | | | Staging activities are usually silent nested, this regressed when porting to recently changed APIs.
* Merge branch 'juerg/filter-pass-integration' into 'master'bst-marge-bot2020-09-242-6/+49
|\ | | | | | | | | filter.py: Combine integration commands in assemble() See merge request BuildStream/buildstream!2073
| * tests/integration/filter.py: Add test_filter_pass_integration_uncachedJürg Billeter2020-09-241-0/+32
| | | | | | | | | | This tests that built filter artifacts don't depend on build dependencies for integration.
| * filter.py: Combine integration commands in assemble()Jürg Billeter2020-09-241-6/+17
|/ | | | | | | | | Plugins must not access public data of build dependencies in `integrate()` as the build dependencies are not guaranteed to be cached at that stage. This combines integration commands of the sole build dependency and the filter element itself at time of assembly instead of integration.
* Merge branch 'tristan/remove-bst-key-requires-stage' into 'master'bst-marge-bot2020-09-247-90/+224
|\ | | | | | | | | source.py: Remove BST_KEY_REQUIRES_STAGE See merge request BuildStream/buildstream!2072
| * tests/frontend/track.py: Test behavior of SKIP messagesTristan van Berkom2020-09-241-0/+79
| | | | | | | | | | | | | | | | This test checks that: * We get SKIP messages for tracking local sources * We get SKIP messages for tracking workspaced elements * We go no messages at all for elemenents which have no sources
| * _elementsources.py: Raise SkipJob() when tracking is unimplemented by all ↵Tristan van Berkom2020-09-241-0/+7
| | | | | | | | | | | | | | | | sources This ensures we get a SKIP message instead of a SUCCESS message when tracking an element where all of it's sources did not implement track(), which is the case for sources like `local`, `workspace` or `patch`.
| * source.py: Remove BST_KEY_REQUIRES_STAGETristan van Berkom2020-09-244-90/+137
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored this to remove unneeded complexity in the code base, as described here: https://lists.apache.org/thread.html/r4b9517742433f07c79379ba5b67932cfe997c1e64965a9f1a2b613fc%40%3Cdev.buildstream.apache.org%3E Changes: * source.py: Added private Source._cache_directory() context manager We also move the assertion about nodes which are safe to write to a bit lower in Source._set_ref(), as this was unnecessarily early. When tracking a workspace, the ref will be none and will turn out to be none afterwards, it is not a problem that a workspace's node is a synthetic one, as tracking will never affect it. * local plugin: Implement get_unique_key() and stage() using the new context manager in order to optimize staging and cache key calculations here. * workspace plugin: Implement get_unique_key() and stage() using the new context manager in order to optimize staging and cache key calculations here. * trackqueue.py: No special casing with Source._is_trackable()
| * __init__.py: Added missing import of Directory, this is public APITristan van Berkom2020-09-231-0/+1
|/ | | | | We don't allow importing symbols from sub-packages of BuildStream, so any public API must find it's way to the toplevel __init__.py.
* Merge branch 'tristan/fix-overnight-tests' into 'master'bst-marge-bot2020-09-234-14/+11
|\ | | | | | | | | .gitlab-ci.yml: Use ported plugins and fdsdk for overnight tests See merge request BuildStream/buildstream!2068
| * .gitlab-ci.yml: Use ported plugins and fdsdk for overnight testsTristan van Berkom2020-09-231-1/+1
| |
| * Sandbox: Remove Sandbox.set_output_directory()Tristan van Berkom2020-09-211-10/+0
| | | | | | | | | | This was a dead codepath, not used by any sandbox implementation for any reason, and used to be called inconsistently by some elements.
| * ArtifactElement: Don't call Sandbox.set_output_directory()Tristan van Berkom2020-09-211-3/+0
| | | | | | | | That API is useless fluff and does not have any affect on anything.
| * Sandbox: Assert absolute paths where they are setTristan van Berkom2020-09-211-0/+4
| |
| * Element: Substitute variables on dependency configurationsTristan van Berkom2020-09-211-0/+6
|/ | | | This was missing in the initial implementation.
* Merge branch 'tristan/fix-cache-key-comments' into 'master'bst-marge-bot2020-09-211-9/+11
|\ | | | | | | | | element.py: Clarify weak and strict key descriptions in __update_cache_keys() See merge request BuildStream/buildstream!2069
| * element.py: Clarify weak and strict key descriptions in __update_cache_keys()tristan/fix-cache-key-commentsTristan van Berkom2020-09-211-9/+11
|/
* Merge branch 'tristan/dependency-config' into 'master'Tristan Van Berkom2020-09-1966-549/+1472
|\ | | | | | | | | Implement Element.configure_dependencies() See merge request BuildStream/buildstream!2032
| * NEWS: Documenting BuildElement's new dependency configurationtristan/dependency-configTristan van Berkom2020-09-181-0/+3
| |
| * tests/integration/manual.py: Test staging files in custom directoriesTristan van Berkom2020-09-183-0/+37
| |
| * tests/cachekey: Added new cachekey test for new BuildElement configurationTristan van Berkom2020-09-184-1/+16
| |
| * buildelement.py: Implement "location" dependency configurationTristan van Berkom2020-09-181-11/+86
| | | | | | | | | | | | | | This addresses the feature request to stage dependencies in sysroots from a couple years back: https://mail.gnome.org/archives/buildstream-list/2018-August/msg00009.html
| * buildelement.py: Remove legacy command stepsTristan van Berkom2020-09-184-16/+4
| | | | | | | | | | | | | | | | | | | | | | This was actually deadcode, since node.validate_keys() was called on the configure dictionary without the legacy command steps. If any element was using the legacy commands, they would have been met with a load time error anyway. This commit also updates the cache key test, since removing these legacy commands affects BuildElement internally in such a way as to affect the cache keys.
| * NEWS: Updated for breaking changes related to ScriptElementTristan van Berkom2020-09-181-0/+9
| |
| * ScriptElement: Porting to new APITristan van Berkom2020-09-1811-161/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of relying on Element.search(), use Element.configure_dependencies() to configure the layout. Summary of changes: * scriptelement.py: Change ScriptElement.layout_add() API to take an Element instead of an element name, this is now not optional (one cannot specify a `None` element). This is an API breaking change * plugins/elements/script.py: Implement Element.configure_dependencies() in order to call ScriptElement.layout_add(). This is a breaking YAML format change. * tests/integration: Script integration tests updated to use the new YAML format * tests/cachekey: Updated for `script` element changes
| * tests/frontend/overlaps.py: Test multiple calls to ↵Tristan van Berkom2020-09-188-2/+192
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Element.stage_dependency_artifacts() This patch adds a new test plugin which implements Element.configure_dependencies() in order to offer better flexibility for testing overlaps. Newly added tests: * Test overlap warnings and errors when staging elsewhere than in the sandbox root. * Test unstaged files failure modes when staging elsewhere than in the sandbox root. * Test various overlap behaviors of OverlapAction, when different calls to Element.stage_dependency_artifacts() cause overlaps to occur after staging files into separate directories.
| * doc: Documenting dependency configurationsTristan van Berkom2020-09-182-6/+51
| |
| * tests/format/dependencies.py: Adding tests for dependency `config` attributesTristan van Berkom2020-09-189-0/+119
| |
| * element.py: Adding new configure_dependencies() public API methodTristan van Berkom2020-09-184-4/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements the essentials of the proposal to extend the dependency attributes: https://lists.apache.org/thread.html/r850aeb270200daade44261f16dbad403bf762e553b89dcafa0848fe7%40%3Cdev.buildstream.apache.org%3E And essentially this will obsolete issue #931 by providing a more suitable replacement for Element.search(). Summary of changes: * _loader/loadelement.pyx: The Dependency object now loads the `config` node, and raises an error if the `config` node is specified on a runtime-only dependency. * element.py: Created the new Element.configure_dependencies() virtual method. If elements implement this method, then a list of all dependencies are collected at element construction time and given to the implementing (depending) element. If a build dependency has more than one `config` specified, then it will be given to the Element twice, and if there is no `config` specified, then the tuple will still be given to the element with a Null `config`. Elements are provided via a new DependencyConfiguration type.
| * _loader/loadelement.pyx: Added `path` propertyTristan van Berkom2020-09-181-0/+11
| | | | | | | | | | This helps simplify the following Element.configure_dependencies() implementing patch.
| * tests/format/dependencies.py: Test merging of multiple dependenciesTristan van Berkom2020-09-183-0/+34
| | | | | | | | | | | | Test that when the same dependency is added as a build and runtime dependency separately, they end up being the same dependency which is both a build & runtime dependency in the loaded build graph.
| * _loader/loadelement.pyx: Merge duplicate dependencies.Tristan van Berkom2020-09-182-29/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the same element is specified multiple times as a direct dependency, merge and accumulate the results into the already loaded dependency. * If the same element is a runtime and build dependency separately, it will be a single dependency of both runtime and build. * If either of the dependencies are `strict`, it will be a strict dependency. The build graph retains the invariant that an element only ever depends on another element once directly, only the YAML can express the same dependency differently more than once, and the results are accumulated. This consequently remoces LoadErrorReason.DUPLICATE_DEPENDENCY as this is no longer relevant.
| * tests/format/dependencies.py: Remove tests for ↵Tristan van Berkom2020-09-189-97/+0
| | | | | | | | LoadErrorReason.DUPLICATE_DEPENDENCY
| * _loader: Added DependencyTypeTristan van Berkom2020-09-185-32/+54
| | | | | | | | | | | | This is a bit nicer than relying on strings in the Symbol enumeration, and allows for some bitwise operations so we can test for BUILD or RUNTIME.
| * tests/frontend/overlaps.py: Test CoreWarnings.UNSTAGED_FILESTristan van Berkom2020-09-186-2/+38
| |
| * tests/frontend/overlaps.py: Minor cleanup and refactoringTristan van Berkom2020-09-181-37/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit: * Removes testing of the deprecated `fail-on-overlap` project configuration option, this is going away soon and unneeded. * Tests that warnings are issued whenever they should be (some tests were happy to see a successful run but failed to check for an expected warning). * Test error/warning more evenly across tests, some were missing the warning mode. * Use `bst show` instead of `bst build` for the undefined_variable test, it should fail without needing a build.
| * element.py, _elementproxy.py: Use new OverlapCollectorTristan van Berkom2020-09-182-163/+162
| | | | | | | | | | | | | | | | | | | | | | Setup the OverlapCollector in Element.stage() routines, and ensure we call OverlapCollector.start_session() and OverlapCollector.end_session() in the right places. This adds the OverlapAction `action` parameter to the Element.stage_artifact() and Element.stage_dependency_artifacts() APIs so that Elements can choose how to behave when multiple artifact staging calls overlap with files staged by previous artifact staging calls.
| * _overlapcollector.py: Adding the OverlapCollector as a separate moduleTristan van Berkom2020-09-181-0/+328
| |
| * types.py: Adding CoreWarnings.UNSTAGED_FILESTristan van Berkom2020-09-181-1/+7
| | | | | | | | | | Makes the warning fatal if we fail to stage a file because it would have otherwise overwritten a non-empty directory.
| * types.py: Adding OverlapAction enumerationTristan van Berkom2020-09-182-1/+41
| | | | | | | | | | Used to define the behavior of multiple calls to Element.stage_artifact() and Element.stage_dependency_artifacts()
| * tests/elements/filter/basic/element_plugins/dynamic.py: Stage in Element.stage()Tristan van Berkom2020-09-181-5/+2
| | | | | | | | | | This test element was also staging artifacts in Element.assemble(), which is now illegal.
| * filter plugin: Stage the artifacts in Element.stage()Tristan van Berkom2020-09-181-3/+2
| | | | | | | | Staging artifacts at Element.assemble() time is now illegal
| * compose plugin: Stage the artifacts in Element.stage()Tristan van Berkom2020-09-181-5/+3
|/ | | | | It will now be illegal to call Element.stage_dependency_artifacts() outside of the Element.stage() abstract method.
* Merge branch 'tristan/fix-overnight-tests' into 'master'bst-marge-bot2020-09-172-6/+36
|\ | | | | | | | | | | | | Fix overnight tests Closes #1365 See merge request BuildStream/buildstream!2066
| * .gitlab-ci.yml: Update references to externalsTristan van Berkom2020-09-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Point the FDSDK branch to the newly branched: freedesktop-sdk-20.08beta.1-buildstream2-overnight-tests We used to use freedesktop-sdk-20.08beta.1-buildstream2, but this branch is apparently used by unknown parties for unknown reasons. Use a branch that has the specific purpose of BuildStream overnight tests, so we can safely update the branch as needed. * Point to latest commit sha on bst-plugins-experimental which works with the latest BuildStream APIs.
| * .gitlab-ci.yml: Added comment explaining how we test without `usedevelop`Tristan van Berkom2020-09-171-0/+6
| |
| * .gitlab-ci.yml: Fix randomized overnight testsTristan van Berkom2020-09-171-1/+1
| | | | | | | | | | | | | | This was failing because we are trying to run the py38-randomized tox environment in the test, which is not supported. Use py38-randomized-nocover instead.
| * .gitlab-ci.yml: Finish removing the external testsTristan van Berkom2020-09-171-1/+0
| | | | | | | | | | | | | | | | In december, Ben removed the "external" tox environment marker, in commit: 2d2bd16a1f931b4aece069baf94257df22e869cc The overnight tests have still been trying to run tests with environments such as `py36-randomized-external` which is invalid.
| * element.py: Cache ElementProxiesTristan van Berkom2020-09-171-3/+28
|/ | | | | | This allows plugins to keep making statements such as `element in dependencies` or `elementA is elementB`, which was currently broken due to creating proxies on demand.
View Lorry Controller Status