summaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* _frontend/cli.py: Drop support for bst shell --use-buildtree=tryJürg Billeter2020-10-274-85/+18
| | | | | `--use-buildtree` is now a boolean option for `bst shell`. If no buildtree is available, an error is raised.
* _stream.py: Simplify code in shell()Jürg Billeter2020-10-271-15/+7
| | | | | | Use single scheduler run for pulling dependencies and buildtree. Deduplicate cache checks and corresponding warning/error messages with and without pulling enabled.
* Move shell element loading logic from Cli to StreamJürg Billeter2020-10-271-3/+3
|
* tests/frontend/track.py: Conditionally skip a testtristan/conditional-subsecond-precisionTristan van Berkom2020-10-251-0/+6
| | | | | | Skip a test which relies on mtimes differing within a short timespan, this will fail if it happens fast enough (which it should) on systems which do not support subsecond precision mtimes.
* tests/internals/utils_move_atomic.py: Conditionally skip a testTristan van Berkom2020-10-251-0/+7
| | | | | Skip the mtime related test if the underlying filesystem does not support subsecond precision mtime.
* tests/artifactcache/expiry.py: Conditionally skip some testsTristan van Berkom2020-10-251-0/+45
| | | | | Skip some of the artifact expiry tests in the case we don't have subsecond mtime precision.
* tests/internals/storage_vdir_import.py: Conditionally skipTristan van Berkom2020-10-251-0/+7
| | | | Conditionally skip tests which require subsecond mtime precision.
* tests/format/junctions.py: Test multiple layers of link indirection in overridesTristan van Berkom2020-10-065-0/+26
|
* tests/format/junctions.py: Test that links work for override keys in nested ↵Tristan van Berkom2020-10-063-0/+18
| | | | projects
* tests/format/junctions.py: Test that we can use a link to address which ↵Tristan van Berkom2020-10-064-1/+28
| | | | subproject to override
* _stream.py: Pull missing artifacts in push()juerg/pushJürg Billeter2020-09-292-12/+2
| | | | | | | | | | | As per #819, BuildStream should pull missing artifacts by default. The previous behavior was to only pull missing buildtrees. A top-level `--no-pull` option can easily be supported in the future. This change makes it possible to use a single scheduler session (with concurrent pull and push jobs). This commit also simplifies the code as it removes the `sched_error_action` emulation, using the regular frontend code path instead.
* tests/format/dependencies.py: Test bad filename configurationTristan van Berkom2020-09-282-0/+24
| | | | Test a dictionary instead of a string when given to the filename list.
* tests/format/dependencies.py: Testing shorthand dependency configurationsTristan van Berkom2020-09-289-0/+51
|
* tests/format/dependencies3/elements/supported2.bst: Adding commentTristan van Berkom2020-09-281-0/+6
| | | | Adding some clarifications about an existing test
* tests/integration/filter.py: Add test_filter_pass_integration_uncachedJürg Billeter2020-09-241-0/+32
| | | | | This tests that built filter artifacts don't depend on build dependencies for integration.
* tests/frontend/track.py: Test behavior of SKIP messagesTristan van Berkom2020-09-241-0/+79
| | | | | | | | This test checks that: * We get SKIP messages for tracking local sources * We get SKIP messages for tracking workspaced elements * We go no messages at all for elemenents which have no sources
* tests/integration/manual.py: Test staging files in custom directoriesTristan van Berkom2020-09-183-0/+37
|
* tests/cachekey: Added new cachekey test for new BuildElement configurationTristan van Berkom2020-09-184-1/+16
|
* buildelement.py: Remove legacy command stepsTristan van Berkom2020-09-183-3/+3
| | | | | | | | | | | This was actually deadcode, since node.validate_keys() was called on the configure dictionary without the legacy command steps. If any element was using the legacy commands, they would have been met with a load time error anyway. This commit also updates the cache key test, since removing these legacy commands affects BuildElement internally in such a way as to affect the cache keys.
* ScriptElement: Porting to new APITristan van Berkom2020-09-188-44/+20
| | | | | | | | | | | | | | | | | | | | | | | | Instead of relying on Element.search(), use Element.configure_dependencies() to configure the layout. Summary of changes: * scriptelement.py: Change ScriptElement.layout_add() API to take an Element instead of an element name, this is now not optional (one cannot specify a `None` element). This is an API breaking change * plugins/elements/script.py: Implement Element.configure_dependencies() in order to call ScriptElement.layout_add(). This is a breaking YAML format change. * tests/integration: Script integration tests updated to use the new YAML format * tests/cachekey: Updated for `script` element changes
* tests/frontend/overlaps.py: Test multiple calls to ↵Tristan van Berkom2020-09-188-2/+192
| | | | | | | | | | | | | | | | | | | | Element.stage_dependency_artifacts() This patch adds a new test plugin which implements Element.configure_dependencies() in order to offer better flexibility for testing overlaps. Newly added tests: * Test overlap warnings and errors when staging elsewhere than in the sandbox root. * Test unstaged files failure modes when staging elsewhere than in the sandbox root. * Test various overlap behaviors of OverlapAction, when different calls to Element.stage_dependency_artifacts() cause overlaps to occur after staging files into separate directories.
* tests/format/dependencies.py: Adding tests for dependency `config` attributesTristan van Berkom2020-09-189-0/+119
|
* tests/format/dependencies.py: Test merging of multiple dependenciesTristan van Berkom2020-09-183-0/+34
| | | | | | Test that when the same dependency is added as a build and runtime dependency separately, they end up being the same dependency which is both a build & runtime dependency in the loaded build graph.
* tests/format/dependencies.py: Remove tests for ↵Tristan van Berkom2020-09-189-97/+0
| | | | LoadErrorReason.DUPLICATE_DEPENDENCY
* tests/frontend/overlaps.py: Test CoreWarnings.UNSTAGED_FILESTristan van Berkom2020-09-186-2/+38
|
* tests/frontend/overlaps.py: Minor cleanup and refactoringTristan van Berkom2020-09-181-37/+32
| | | | | | | | | | | | | | | | | This commit: * Removes testing of the deprecated `fail-on-overlap` project configuration option, this is going away soon and unneeded. * Tests that warnings are issued whenever they should be (some tests were happy to see a successful run but failed to check for an expected warning). * Test error/warning more evenly across tests, some were missing the warning mode. * Use `bst show` instead of `bst build` for the undefined_variable test, it should fail without needing a build.
* tests/elements/filter/basic/element_plugins/dynamic.py: Stage in Element.stage()Tristan van Berkom2020-09-181-5/+2
| | | | | This test element was also staging artifacts in Element.assemble(), which is now illegal.
* tests/frontend/rebuild.py: Add test_modify_and_revertJürg Billeter2020-09-161-0/+41
| | | | | | This tests that, in non-strict mode, a cached artifact matching the strict cache key is preferred to a more recent artifact matching only the weak cache key.
* tests/frontend/rebuild.py: Add assertion for built elementsJürg Billeter2020-09-161-0/+8
|
* tests/frontend/push.py: Test artifact push with globJürg Billeter2020-09-101-0/+30
|
* tests/frontend/buildcheckout.py: Add non-strict cache key testjuerg/cache-keyJürg Billeter2020-09-081-0/+34
| | | | | This is a regression test for the previously broken dependency cache key check in non-strict mode.
* element.py: Hide dependencies which are irrelevant to the ElementTristan van Berkom2020-09-042-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is a large breaking change, a summary of the changes are that: * The Scope type is now private, since Element plugins do not have the choice to view any other scopes. * Element.dependencies() API change Now it accepts a "selection" (sequence) of dependency elements, so that Element.dependencies() can iterate over a collection of dependencies, ensuring that we iterate over every element only once even when we need to iterate over multiple element's dependencies. The old API is moved to Element._dependencies() and still used internally. * Element.stage_dependency_artifacts() API change This gets the same treatment as Element.dependencies(), and the old API is also preserved as Element._stage_dependency_artifacts(), so that the CLI can stage things for `bst artifact checkout` and such. * Element.search() API change The Scope argument is removed, and the old API is preserved as Element._search() temporarily, until we can remove this completely.
* Element.dependencies() now yields ElementProxy wrappers by default.Tristan van Berkom2020-09-041-1/+1
| | | | | | | | | | | | | | | This prepares the ground for policing the dependencies which are visible to an Element plugin, such that plugins are only allowed to see the elements in their Scope.BUILD scope, even if they call Element.dependencies() on a dependency. This commit does the following: * Element.dependencies() is now a user facing frontend which yields ElementProxy elements instead of Elements. * Various core codepaths have been updated to call the internal Element._dependencies() codepath which still returns Elements.
* Move handling of the source `directory` configuration to ElementSourcesjuerg/element-source-cacheJürg Billeter2020-09-0328-28/+28
| | | | | | | | | | | | | | | | | The `directory` value determines where a source is staged within the build root of an element, however, it does not directly affect individual sources. With this change the sources will individually be cached in CAS independent of the value of `directory`. `ElementSources` will use the value of `directory` when staging all element sources into the build root. This results in a cache key change as the `directory` value is moved from the unique key of individual sources to the unique key of `ElementSources`. This is in preparation for #1274.
* Add ElementSourcesCacheJürg Billeter2020-09-034-20/+44
| | | | | | | | | | | | | | | | | | Sources have been cached in CAS individually, except for sources that transform other sources, which have been cached combined with all previous sources of the element. This caching structure may be confusing as sources are specified in the element as a list and this is not a good fit for #1274 where we want to support caching individual sources in a Remote Asset server with a BuildStream-independent URI (especially the `directory` configuration would be problematic). This replaces the combined caching of 'previous' sources with an element-level source cache, which caches all sources of an element staged together. Sources that don't depend on previous sources are still cached individually. This also makes it possible to add a list of all element sources to the source proto used by the element-level source cache.
* tests/sourcecache: Do not inspect uninitialized elementsJürg Billeter2020-09-033-0/+10
|
* tests/sourcecache/push.py: Add test_push_missing_source_after_build()Jürg Billeter2020-09-011-0/+30
| | | | | Test that source push succeeds if the source needs to be fetched even if the artifact of the corresponding element is already cached.
* runcli.py: Configurable index/storage artifact typesJosh Smith2020-08-261-0/+8
| | | | | This enables for seperate index/storage artifact servers to be configured by environment variables passed through tox.
* _elementsources.py: Fix source names in unique keyjuerg/source-cache-keyJürg Billeter2020-08-2528-28/+28
| | | | | | | | | | | | | | The goal was to include the source plugin kind in the element cache key as the unique key of a source may not be unique across different source plugins. This is the source equivalent of the `element-plugin-name` value in the element cache key. However, `Source._get_source_name()` was the wrong method for this as that also includes the key itself, which may not even be set yet. This results in a cache key change. Fixes: 3953bcc6 ("element.py: clobber sources with workspace")
* _fixtures.py: Only get the normal number of threads at the start of sessionBenjamin Schubert2020-08-231-1/+5
| | | | | This ensures that we also have our tests correctly shutting down background threads and not interferring with each other
* buildstream/element.py: Consider "build-root" variable in the cache key ↵tristan/build-root-cache-keyTristan van Berkom2020-08-2028-28/+28
| | | | | | | | | | | | | | | | | | | unconditionally Not all elements use the "build-root" variable, but it is the standard variable to use for the build directory, and the build directory must be considered in the cache key. Handling this unilaterally in the core is safer than delegating this to element implementations, as we have less chance of plugin authors missing this detail and possibly introducing binary variance for artifacts where only the build directory differs (something which happens when the project name or element names change). This commit also updates the hard coded cache keys in the cache key test, so as to ensure every commit passes it's own tests. This fixes #1386.
* tests/cachekey/cachekey.py: Use the same project name for cache key testsTristan van Berkom2020-08-201-1/+1
| | | | | | | | In the test_cache_key_fatal_warnings() test, use the same project name in both generated project directories in order to pass the tests in the case that elements are guaranteed to have differing cache keys for differeing element/project names (which is the case when we consider the "build-root" in the cache key unconditionally).
* Bump minimum version requirement for ruamel.yamlchandan/upgrade-ruamelChandan Singh2020-08-182-0/+36
| | | | | | | | | | | | ruamel.yaml <= 0.16.6 suffers from a bug where ruamel's yaml dumper crashes when used on a sequence that has comments before it. In BuildStream, this manifests in form of issues like #1265. See upstream issue at https://sourceforge.net/p/ruamel-yaml/tickets/335. Also, add a regression test for it. Fixes #1265.
* _frontend/cli: Add --deps `build` & `run` values for `source track --deps`Chandan Singh2020-08-148-2/+65
| | | | | | | This is part of #1349. This patch will conclude the first part of that issue, i.e. ensuring that the possible options for `--deps` are consistent across all commands (with the exception of `--deps plan` that we will handle separately).
* regression-test for ".bst" artifact checkout bugdwinship/test_for_dot_bst_artifact_checkout_bugDouglas Winship2020-08-131-0/+24
|
* casserver.py: Drop BuildStream Artifact and Source servicesJürg Billeter2020-08-131-107/+0
| | | | Replaced by Remote Asset API Fetch and Push services.
* _sourcecache.py: Use AssetRemoteJürg Billeter2020-08-131-11/+23
| | | | | This migrates the source cache from the BuildStream Source protocol to the Remote Asset API.
* _artifactcache.py: Use AssetRemoteJürg Billeter2020-08-131-11/+38
| | | | | | | This migrates the artifact cache from the BuildStream Artifact protocol to the Remote Asset API. Co-authored-by: Sander Striker <s.striker@striker.nl>
* tests/frontend/push.py: Pull complete artifactJürg Billeter2020-08-131-7/+8
| | | | | | | | Use a separate Cli instance with a separate local cache for the second pull in `test_recently_pulled_artifact_does_not_expire()` to ensure the complete artifact is pulled. If only a part of the artifact is pulled, there is no guarantee that the other blobs of that artifact won't expire.
* Completely remove MetaElementTristan van Berkom2020-08-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This dramatically affects the load process and removes one hoop we had to jump through, which is the creation of the extra intermediate MetaElement objects. This allows us to more easily carry state discovered by the Loader over to the Element constructor, as we need not add additional state to the intermediate MetaElement for this. Instead we have the Element initializer understand the LoadElement directly. Summary of changes: * _loader/metaelement.py: Removed * _loader/loadelement.py: Added some attributes previously required on MetaElement * _loader/loader.py: Removed _collect_element() and collect_element_no_deps(), removing the process of Loader.load() which translates LoadElements into MetaElements completely. * _loader/init.py: Export LoadElement, Dependency and Symbol types, stop exporting MetaElement * _loader/metasource.py: Now take the 'first_pass' parameter as an argument * _artifactelement.py: Use a virtual LoadElement instead of a virtual MetaElement to instantiate the ArtifactElement objects. * _pluginfactory/elementfactory.py: Adjust to now take a LoadElement * _project.py: Adjust Project.create_element() to now take a LoadElement, and call the new Element._new_from_load_element() instead of the old Element._new_from_meta() function * element.py: - Now export Element._new_from_load_element() instead of Element._new_from_meta() - Adjust the constructor to do the LoadElement toplevel node parsing instead of expecting members on the MetaElement object - Added __load_sources() which parses out and creates MetaSource objects for the sake of instantiating the element's Source objects. Consequently this simplifies the scenario where workspaces are involved. * source.py: Adjusted to use the new `first_pass` parameter to MetaSource when creating a duplicate clone.
View Lorry Controller Status