summaryrefslogtreecommitdiff
path: root/tests/sourcecache
Commit message (Collapse)AuthorAgeFilesLines
* tests/sourcecache/config.py: Fixed generated project to include min-versionTristan van Berkom2020-11-301-0/+1
| | | | | | This test was broken as it was failing for the wrong reason, even though in both cases it was a missing yaml key. Fix the test to fail due to it being missing the required cert specified in the cache config.
* Add ElementSourcesCacheJürg Billeter2020-09-034-20/+44
| | | | | | | | | | | | | | | | | | Sources have been cached in CAS individually, except for sources that transform other sources, which have been cached combined with all previous sources of the element. This caching structure may be confusing as sources are specified in the element as a list and this is not a good fit for #1274 where we want to support caching individual sources in a Remote Asset server with a BuildStream-independent URI (especially the `directory` configuration would be problematic). This replaces the combined caching of 'previous' sources with an element-level source cache, which caches all sources of an element staged together. Sources that don't depend on previous sources are still cached individually. This also makes it possible to add a list of all element sources to the source proto used by the element-level source cache.
* tests/sourcecache: Do not inspect uninitialized elementsJürg Billeter2020-09-033-0/+10
|
* tests/sourcecache/push.py: Add test_push_missing_source_after_build()Jürg Billeter2020-09-011-0/+30
| | | | | Test that source push succeeds if the source needs to be fetched even if the artifact of the corresponding element is already cached.
* tests/sourcecache/fetch.py: Use `bst source push` to replace hackJürg Billeter2020-07-061-12/+20
|
* plugin.py/pluginfactory.py: Implementing BST_MIN_VERSIONTristan Van Berkom2020-04-291-0/+3
| | | | | | | | | | | | | | The BST_MIN_VERSION guards assert that the BuildStream core which loaded the plugin is compatible with the plugin itself. This commit adds BST_MIN_VERSION to the base plugin.py with documentation informing Plugin authors how to set the minimum version, and also adds the assertions at plugin loading time in pluginfactory.py. This commit also: * Adds the BST_MIN_VERSION specification to all current core plugins * Adds the BST_MIN_VERSION specification to plugins used in test cases
* Plugin loading refactor, removing all versioningTristan Van Berkom2020-04-291-1/+1
| | | | | | | | | | | | | | | | Plugin format versioning was decided to be removed for local plugins and any plugins for which we do not load an explicitly provided plugin. For pip, this will be handled with a standard distutils/setuptools approach, allowing users to specify pip style version boundaries in the plugin origins. This patch refactors plugin loading so that all related code goes into the private _pluginfactory module, a new small PluginOrigin type was added to better manipulate loaded origins. Test cases have been removed and will be readded in a following commit, adjusted to new expectations.
* Replace format-version with min-versionTristan Van Berkom2020-04-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * "min-version" is specified as a <major>.<minor> point version and uses the installed BuildStream version instead of having a separate versioning number for the format. * The presence of "format-version" is now used to indicate that we might be loading a BuildStream 1 project. * For now, where parsing the version at startup is concerned, and also where `bst init` is concerned, we artificially bump the detected BuildStream version to 2.0 if we detect a version < 2.0, these exceptions can be removed once 2.0 is tagged and released. Summary of changes: _project.py: Now parse "min-version" and detect "format-version" to warn about loading a BuildStream 1 project _versions.py: Remove obsolete BST_FORMAT_VERSION numbers from here data/projectconfig.yaml: Remove old "format-version" from defaults utils.py: Added new private _parse_version() helper function, and another _get_bst_api_version() to get an adjusted API version. frontend/app.py, frontend/cli.py: Updated `bst init` implementation testing (buildstream.testing): Updated testing utilities to generate and use projects with min-version instead of format-version. tests and examples: Updated to use min-version across the board.
* exceptions: Expose ErrorDomain, ErrorLoadReasonThomas Coldrick2020-01-234-4/+4
| | | | | | | Plugin tests are already accessing this API, but using imports from private modules. For motivation for this to be exposed publicly, note that ErrorDomain is an argument for most things in runcli.py, and LoadErrorReason may be another.
* element.py: Rename '_source_cached' to '_has_all_sources_in_source_cache'Benjamin Schubert2020-01-163-11/+11
| | | | | | '_source_cached' is not explicit enough as it doesn't distinguishes between sources in their respective caches and sources in the global sourcecache.
* tests: Drop buildbox xfailsJürg Billeter2019-12-101-2/+0
|
* Remove newly unused API surfaces in CASCacheTristan Maat2019-12-032-10/+6
| | | | | | This also involves a number of changes to tests and other parts of the codebase since they were hacking about wit API that shouldn't have existed.
* Reformat code using BlackChandan Singh2019-11-149-299/+215
| | | | | | | As discussed over the mailing list, reformat code using Black. This is a one-off change to reformat all our codebase. Moving forward, we shouldn't expect such blanket reformats. Rather, we expect each change to already comply with the Black formatting style.
* _sourcecache.py: Fix reporting of capabilities for the source cacheBenjamin Schubert2019-11-121-0/+54
| | | | This also adds a test to ensure we are correctly setting the cache up
* Remove `commit`ting sources inside `Source()._generate_key`Darius Makovsky2019-11-051-2/+1
| | | | | | | | `Stream.shell()` should check that the element's sources are cached before calling the shell. If the sources are not cached raise a StreamError and recommend a fetch. closes #1182
* Replace BST_NO_PRESTAGE_KEY with BST_KEY_REQUIRES_STAGEDarius Makovsky2019-11-041-1/+1
| | | | Correct version number for BST_KEY_REQUIRES_STAGE
* local.py: use extended API, set BST_NO_PRESTAGE_KEYDarius Makovsky2019-10-301-1/+2
| | | | | | | tests: * local sources are cached * use git sources instead of local when removing objects * update expected cache keys
* _sourcecache: Fallback to fetch source when remote has missing blobsBenjamin Schubert2019-10-101-0/+47
| | | | | | If a remote has some missing blobs for a source, we should not fail abruptly but instead continue to the next remote, and, in the worst case, fetch the source again.
* tests/sourcecache/fetch.py: Extract configuration of bstBenjamin Schubert2019-10-101-48/+26
| | | | | | | | Extract the configuration of the BuildStream cli as it is every time the same into a separate function. This helps seeing what is setup of the tests and what is the actual test itself.
* tests/sourcecache/fetch.py: Extract element's creation to a functionBenjamin Schubert2019-10-101-30/+17
| | | | | | All tests in this file use the same method for creating an element. It is easier to write new tests to have this factored out.
* tests/sourcecache/fetch.py: Don't hardcode the element nameBenjamin Schubert2019-10-101-7/+7
| | | | | This removes the need of having a hardcoded element name, by simply reusing the 'element_name' variable.
* tests/sourcecache/fetch.py: Extract logic to move local cas to remoteBenjamin Schubert2019-10-101-10/+9
| | | | This method can be reused and shared, and makes the tests more readable.
* setup.cfg: make xfail marks strict by defaulttraveltissues/strictxfailDarius Makovsky2019-10-091-1/+1
| | | | tests: remove strict kwargs from xfail marks
* cli.py: Make source checkout more consistent with artifact checkoutJames Ennis2019-09-121-3/+3
| | | | | | | | | | | | This MR changes the behaviour of source checkout so that it is more like artifact checkout. That is, a --directory option can be provided, rather than a mandatory LOCATION. In addition to this, the --tar option is no longer a boolean flag, rather it expects a tarfile name (just like artifact checkout) The appropriate tests have also been altered so that they support the new API
* Allow splitting artifact cachesTristan Maat2019-09-061-0/+78
| | | | | | | This is now split into storage/index remotes, where the former is expected to be a CASRemote and the latter a BuildStream-specific remote with the extensions required to store BuildStream artifact protos.
* Mark buildbox tests as xfail if they require non implemented featuresWilliam Salmon2019-07-251-1/+2
|
* tests/sourcecache: Fix typos in commentsJürg Billeter2019-07-163-3/+3
| | | | Spotted by Darius.
* tests/sourcecache/staging.py: Use dummy_context()Jürg Billeter2019-07-161-62/+48
|
* tests/sourcecache/push.py: Use dummy_context()Jürg Billeter2019-07-161-29/+26
|
* tests/sourcecache/fetch.py: Use dummy_context()Jürg Billeter2019-07-161-86/+74
|
* tests: Change all calls to _yaml.dump to _yaml.rountrip_dumpBenjamin Schubert2019-07-154-15/+15
| | | | | Now that both are equivalent, we can skip the sanitization part before the yaml call.
* Source cache uses new proto methodsRaoul Hidalgo Charman2019-07-084-21/+27
| | | | | | | | | This changes the location of source refs from `cache/cas/refs` to `cache/sources_protos` and fixes tests that looked in the old reference place. Source cache now uses the new protocol buffer and grpc methods defined, for storing locally, and pushing and pulling to remotes. Part of #1038
* Refactor, use context.messenger directlyAngelos Evripiotis2019-07-053-7/+7
| | | | | | Instead of having methods in Context forward calls on to the Messenger, have folks call the Messenger directly. Remove the forwarding methods in Context.
* Refactor: message handlers take 'is_silenced'Angelos Evripiotis2019-07-053-3/+3
| | | | | | | | Remove the need to pass the Context object to message handlers, by passing what is usually requested from the context instead. This paves the way to sharing less information with some child jobs - they won't need the whole context object, just the messenger.
* Fetch sources as needed for bst source checkoutJürg Billeter2019-06-271-8/+2
| | | | | This removes the --fetch option of bst source checkout, always enabling fetch support. This is in line with fetching subprojects as needed.
* _yaml: Remove useless calls to '_yaml.node_sanitize'Benjamin Schubert2019-06-252-6/+6
| | | | | | Calling '_yaml.dump' will itself call '_yaml.node_sanitize', therefore we can remove all calls to it in places where we directly after call dump.
* tests/sourcecache/push.py: Less strict initialisation checkjennis/fix_failing_testJames Ennis2019-06-061-2/+2
| | | | | | | | When we are testing push failures, we are being too strict when we try to verify that the remote has failed to be initialized. The assertion was failing because of slightly different gRPC error messages. Now the assertion is less strict but still appropriate.
* _artifact.py: Rework to use artifact protoRaoul Hidalgo Charman2019-05-151-0/+1
| | | | | | | | This will replace the previous use of a directory structure. Quite a lot is changed here, predominantly _artifact and _artifactcache modules. Part of #974
* Add tests for BST_REQUIRES_PREVIOUS_SOURCES_STAGERaoul Hidalgo Charman2019-05-145-0/+156
| | | | | | Tests that sources are cached seperately where appropriate. Part of #982
* plugintestutils: Rename 'plugintestutils' package to 'testing'phil/rename-plugintestutilsPhil Dawson2019-04-166-8/+8
| | | | | | | | | - Rename plugintestutils to testing. - Don't run the tests from bst-plugins-template. This imports buildstream.plugintestutils so will have to be disabled to get through CI. This can be re nabled once bst-plugins-template has been patched.
* testutils: move repo.py into buildstream.plugintestutilsPhil Dawson2019-04-122-3/+4
| | | | | | | | This needs to be exposed as part of the plugin author facing API so that plugin authors can define custom repo types which will can be passed to the set of tests which iterate over multiple source types. Part of the work towards #944
* tests: check source push for workspaced elementsRaoul Hidalgo Charman2019-03-251-0/+40
| | | | | | | Workspaced elements should not push their sources as consistency is marked as cached even if the sources are not necessarily available. Part of #440
* tests: check sources are pushed even if build failsRaoul Hidalgo Charman2019-03-254-0/+100
| | | | | | | Adds a plugin build element that always fails. The test tries to build an element of this kind. Part of #440
* tests: Add source cache push failure testRaoul Hidalgo Charman2019-03-251-0/+42
| | | | Part of #440
* tests: Add source cache pull after a push testRaoul Hidalgo Charman2019-03-251-0/+45
| | | | Part of #440
* tests: Add source cache push testRaoul Hidalgo Charman2019-03-251-0/+96
| | | | Part of #440
* tests: Add source cache fetch failure testRaoul Hidalgo Charman2019-03-251-0/+52
| | | | Part of #440
* tests: Add source cache test for fallbackRaoul Hidalgo Charman2019-03-251-0/+59
| | | | | | | Checks buildstream calls fetch method when remote source cache does not have a source. Part of #440
* tests: Add source cache fetch testRaoul Hidalgo Charman2019-03-251-0/+111
| | | | Part of #440
* element.py: move _generate_key to _source_cachedRaoul Hidalgo Charman2019-03-211-1/+1
| | | | | | This means that keys are only generated after tracking. Part of a fix for !1124
View Lorry Controller Status