summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _artifactcache.py: display the actual key being pulledabderrahim/non-strict-key-display-1.2Abderrahim Kitouni2019-05-041-4/+4
| | | | Fixes #981
* Merge branch 'jjardon/remove_F28' into 'bst-1.2'Javier Jardón2019-05-041-5/+0
|\ | | | | | | | | .gitlab-ci.yml: Remove deprecated F28 See merge request BuildStream/buildstream!1321
| * .gitlab-ci.yml: Remove deprecated F28jjardon/remove_F28Javier Jardón2019-04-301-5/+0
|/
* NEWS: Updating for 1.2.6 release1.2.6Tristan Van Berkom2019-04-301-0/+22
|
* Merge branch 'tristan/fix-cloned-plugin-ids-1.2' into 'bst-1.2'bst-marge-bot2019-04-302-10/+21
|\ | | | | | | | | Tristan/fix cloned plugin ids 1.2 See merge request BuildStream/buildstream!1315
| * source.py: Inherit unique ID in cloned sourcestristan/fix-cloned-plugin-ids-1.2Tristan Van Berkom2019-04-291-3/+5
| | | | | | | | | | | | When cloning a Source, we should inherit the same unique ID for the sake of sending a valid ID in any messages sent back to the frontend from a source cloned in a child task.
| * plugin.py: Allow passing a unique_id through the constructor.Tristan Van Berkom2019-04-291-5/+9
| | | | | | | | | | | | | | In the case of cloned Sources, they should not be allocating a new ID in track() and fetch() in case they do communicate their ID back to the main process, they should inherit the same ID of the Source they were cloned from.
| * plugin.py: Start plugin ID counter at ID 1Tristan Van Berkom2019-04-291-2/+7
|/ | | | | | | | | | | This was always intended, but was not well commented. The reason we start plugin ID counters at 1 is that we prefer relying on a falsy value to determine whether an ID holding variable has been set or not. This patch also adds a more informative assertion in Plugin._lookup() This by itself essentially fixes #1012
* Merge branch 'bochecha/fix-docs-build' into 'bst-1.2'Tristan Van Berkom2019-04-222-2/+2
|\ | | | | | | | | Fix building the docs See merge request BuildStream/buildstream!1302
| * doc/source/format_declaring.rst: Fix the YAML blockMathieu Bridon2019-04-221-1/+1
| | | | | | | | String starting with a "%" character need to be quoted.
| * doc: Fix the build with Sphinx 2.0Mathieu Bridon2019-04-221-1/+1
|/ | | | | | | | | | | | Sphinx 1.7, released in February 2018 moved the sphinx.apidoc module to sphinx.ext.apidoc, with an alias and a deprecation warning in place so users know to port their code. The compatibility alias was removed in Sphinx 2.0, so we need to move to the new module name. Fortunately, since the new module name is more than a year old, this shouldn't break anything for anybody.
* Merge branch 'tristan/rename-errno-fix-1.2' into 'bst-1.2'bst-marge-bot2019-04-161-3/+3
|\ | | | | | | | | plugins/sources/git.py: Cope with rename returning error EEXIST See merge request BuildStream/buildstream!1294
| * plugins/sources/git.py: Cope with rename returning error EEXISTTristan Van Berkom2019-04-161-3/+3
|/ | | | Thanks to Matthew Yates for pointing this out.
* Merge branch 'tristan/fix-non-strict-push-1.2' into 'bst-1.2'bst-marge-bot2019-04-152-0/+60
|\ | | | | | | | | Fix non strict push 1.2 See merge request BuildStream/buildstream!1289
| * tests/frontend/push.py: Test pushing artifacts in non-strict modetristan/fix-non-strict-push-1.2Tristan Van Berkom2019-04-151-0/+46
| | | | | | | | This is a regression test for issue #990
| * _stream.py: Mark all elements as pulled in Stream.push()Tristan Van Berkom2019-04-151-0/+14
|/ | | | | | | | | Marking all elements as pulled in Stream.push() ensures that cache keys are resolved before pushing elements, otherwise state is left unresolved in non-strict mode while elements are awaiting to download an artifact by it's strict cache key. Fixes #990
* Merge branch 'tristan/platform-cache-quota-1.2' into 'bst-1.2'bst-marge-bot2019-04-1523-140/+466
|\ | | | | | | | | Cache quote related backports See merge request BuildStream/buildstream!1288
| * tests/testutils/artifactshare.py: Add message handlertristan/platform-cache-quota-1.2Tristan Van Berkom2019-04-141-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | This needs to be added along with the status messages added to the artifact cache, and this detail diverges from master. This is because we have not bothered to backport !1071 which refactors the CASCache to be a delegate object of the ArtifactCache istead of a derived class - backporting !1071 would allow us to remove these message handlers because the CAS server and test fixture only use the CASCache object directly, not the business logic in the ArtifactCache.
| * _artifactcache/casserver.py: Add message handlerTristan Van Berkom2019-04-141-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | This needs to be added along with the status messages added to the artifact cache, and this detail diverges from master. This is because we have not bothered to backport !1071 which refactors the CASCache to be a delegate object of the ArtifactCache istead of a derived class - backporting !1071 would allow us to remove these message handlers because the CAS server and test fixture only use the CASCache object directly, not the business logic in the ArtifactCache.
| * _scheduler/jobs/cleanupjob.py: Update cache size while processingTristan Van Berkom2019-04-141-1/+14
| | | | | | | | | | | | Updates the known cache size in the main process while the cleanup process is ongoing, so that the status indicators update live while the cleanup happens.
| * _scheduler/jobs/job.py: Allow subclasses to message the frontendTristan Van Berkom2019-04-141-19/+50
| |
| * _artifactcache.py: Added client progress callback to ArtifactCache.clean()Tristan Van Berkom2019-04-141-1/+12
| |
| * _artifactcache.py: Add status messages in cache management operationsTristan Van Berkom2019-04-141-5/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | Added some useful status messages when: * Calculating a new artifact cache usage size * Starting a cleanup * Finishing a cleanup Also enhanced messaging about what was cleaned up so far when aborting a cleanup.
| * _frontend/status.py: Added Cache size usage indicator to status barTristan Van Berkom2019-04-141-7/+35
| | | | | | | | | | This also adds some comments around the main status bar heading rendering function.
| * _frontend/widget.py: Added cache usage entry in the startup headingTristan Van Berkom2019-04-141-0/+1
| |
| * _context.py: Added get_artifact_cache_usage()Tristan Van Berkom2019-04-141-1/+11
| | | | | | | | | | | | | | | | A frontend facing API for obtaining usage statistics. I would have put this on Stream instead, but the Context seems to be the de facto place for looking up the artifact cache in general so let's put it here.
| * _artifactcache/__init__.py: Add ArtifactCacheUsage to __init__.pyTristan Van Berkom2019-04-141-0/+1
| |
| * _artifactcache.py: Added ArtifactCacheUsage()Tristan Van Berkom2019-04-141-0/+35
| | | | | | | | | | A simple object which creates a snapshot of current usage statistics for easy reporting in the frontend.
| * _artifactcache.py: Raise ArtifactError() when quota size exceeds disk space.Tristan Van Berkom2019-04-143-20/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is not an error related to loading data, like a parse error in the quota specification is, but a problem raised by the artifact cache - this allows us to assert more specific machine readable errors in test cases (instead of checking the string in stderr, which this patch also fixes). This also removes a typo from the error message in the said error. * tests/artifactcache/cache_size.py Updated test case to expect the artifact error, which consequently changes the test case to properly assert a machine readable error instead of asserting text in the stderr (which is the real, secret motivation behind this patch). * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota() Now expect the artifact error for the tests which check configurations which create caches too large to fit on the disk.
| * _artifactcache.py: Refactored to use utils._get_volume_size()Tristan Van Berkom2019-04-143-11/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will benefit from a better UtilError being raised, and and turns the artifact cache's local function into a one liner. The loop which finds the first existing directory in the given path has been removed, being meaningless due to the call to os.makedirs() in ArtifactCache.__init__(). The local function was renamed to _get_cache_volume_size() and no longer takes any arguments, which is more suitable for the function as it serves as a testing override surface for unittest.mock(). The following test cases which use the function to override the ArtifactCache behavior have been updated to use the new overridable function name: tests/artifactcache/cache_size.py tests/artifactcache/expiry.py
| * utils.py: Add _get_volume_size()Tristan Van Berkom2019-04-141-2/+22
| | | | | | | | We can streamline this call to os.statvfs() in a few places.
| * _frontend/app.py: Initialize logging before preflighting the artifact cacheTristan Van Berkom2019-04-141-5/+7
| | | | | | | | The artifact cache emits messages, and we want to allow that in preflight.
| * Use lazy platform instantiationJürg Billeter2019-04-142-4/+4
| | | | | | | | | | | | Now that the platform is independent of the context, explicit instantiation is no longer required. This avoids issues with platform instances used across test cases with mismatching context.
| * _platform: Remove unused contextJürg Billeter2019-04-144-10/+7
| |
| * tests/testutils: Use Context for artifact cache creationJürg Billeter2019-04-141-2/+1
| |
| * _artifactcache/casserver.py: Use Context for artifact cache creationJürg Billeter2019-04-141-3/+1
| |
| * _platform/platform.py: Remove unused artifactcache propertyJürg Billeter2019-04-141-7/+0
| |
| * _stream.py: Migrate to Context.artifactcacheJürg Billeter2019-04-141-3/+1
| |
| * _scheduler: Migrate to Context.artifactcacheJürg Billeter2019-04-144-13/+8
| |
| * _loader: Migrate to Context.artifactcacheJürg Billeter2019-04-141-3/+1
| |
| * Move artifact cache creation to ContextJürg Billeter2019-04-145-16/+16
| | | | | | | | The artifact cache is no longer platform-specific.
| * _artifactcache: Add preflight() methodJürg Billeter2019-04-142-0/+13
| |
| * _artifactcache/cascache.py: Remove unused enable_push parameterJürg Billeter2019-04-141-5/+2
| |
| * _platform/linux.py: Do not disable push globallyJürg Billeter2019-04-141-1/+1
| | | | | | | | | | | | On systems without user namespace support, elements with unsupported sandbox config (UID/GID) are now individually tainted, which disables artifact push.
| * _platform/linux.py: Remove redundant messageJürg Billeter2019-04-141-7/+2
| | | | | | | | Unsupported sandbox config (UID/GID) is now reported by the element.
| * element.py: Validate sandbox configJürg Billeter2019-04-141-1/+17
| |
| * _platform: Add check_sandbox_config() methodJürg Billeter2019-04-143-1/+26
|/
* Merge branch 'tristan/fix-bzr-race-1.2' into 'bst-1.2'bst-marge-bot2019-04-143-93/+116
|\ | | | | | | | | Backport bzr source plugin race condition to 1.2 See merge request BuildStream/buildstream!1287
| * tests/frontend/track.py: Use test_track_recurse() to stress test SourcesTristan Van Berkom2019-04-141-13/+47
| | | | | | | | | | | | | | | | This causes multiple source instances to interact with the same backing data store at the same time, increasing the likelyhood of triggering issues around concurrent access. This more reliably triggers issue #868
| * testutils/runcli.py: Added cli.get_element_states()Tristan Van Berkom2019-04-141-0/+22
| | | | | | | | | | | | | | | | | | With get_element_state(), you need to invoke BuildStream once for every element state you want to observe in a pipeline. The new get_element_states() reports a dictionary with the element state hashed by element name and is better to use if you have more than one element to observe the state of.