summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _yaml.py: Add node_keys()danielsilverstone-ct/more-yaml-cleanupsDaniel Silverstone2019-04-033-6/+23
| | | | | | | | There were a number of places where `node_items()` was called and the values discarded. Since values can be expensive to compute, this replaces any such places with a `node_keys()` function call instead Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'kinnison/cleanups' into 'master'bst-marge-bot2019-04-021-2/+8
|\ | | | | | | | | Some small cleanups See merge request BuildStream/buildstream!1272
| * _yaml.py::node_get(): Only construct path on errorkinnison/cleanupsDaniel Silverstone2019-04-011-2/+6
| | | | | | | | | | | | | | We don't need to spend the CPU constructing path except when there is an error which needs it, so don't. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * _yaml.py::load(): Re-raise LoadError with displaynameDaniel Silverstone2019-04-011-0/+2
|/ | | | | | | | | In the case that `load_data()` raises a `LoadError` we want to catch and re-raise it, augmenting the message with where the load came from which is best done with the `displayname` to include junction name if available. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'bschubert/lint/cyclic-import' into 'master'bst-marge-bot2019-04-0120-36/+38
|\ | | | | | | | | lint: Fix or silence 'cyclic-import' errors and enable pylint for it See merge request BuildStream/buildstream!1250
| * lint: Fix or silence 'cyclic-import' errors and enable pylint for itBenjamin Schubert2019-04-0120-36/+38
|/ | | | | | | | Cyclic imports can be confusing because the order in which we import dependencies can make the import fail or not. We should not rely on ordering of imports for our code. This fixes everywhere possible the imports and silence explicitely some which are not convenient or would require big refactors
* Merge branch 'bschubert/fix-typos' into 'master'bst-marge-bot2019-03-306-13/+13
|\ | | | | | | | | typo: writtent -> written See merge request BuildStream/buildstream!1270
| * typo: writtent -> writtenBenjamin Schubert2019-03-306-13/+13
|/
* Merge branch 'jennis/add_NEWS_entry' into 'master'bst-marge-bot2019-03-291-0/+8
|\ | | | | | | | | FOLLOW UP: Add NEWS entry for YAML New World Order See merge request BuildStream/buildstream!1264
| * NEWS: Add YAML New World Order entryJames Ennis2019-03-291-0/+8
|/ | | | | | This is a follow-up to !1257 - YAML New World Order. Now that !1257 has landed, plugin's *must* use the Plugin.node_* family of functions to access or modify node members.
* Merge branch 'aevri/depency' into 'master'bst-marge-bot2019-03-291-1/+1
|\ | | | | | | | | app.py: fix spelling 'depency' -> 'dependency' See merge request BuildStream/buildstream!1266
| * app.py: fix spelling 'depency' -> 'dependency'Angelos Evripiotis2019-03-291-1/+1
|/
* Merge branch 'bschubert/cache-project-path' into 'master'bst-marge-bot2019-03-291-5/+6
|\ | | | | | | | | _project.py: cache full project path for quicker lookup See merge request BuildStream/buildstream!1269
| * _project.py: cache full project path for quicker lookupbschubert/cache-project-pathBenjamin Schubert2019-03-291-5/+6
|/ | | | | | | | The resolved path to the project directory is queried a lot when using local elements, patch elements, and such. Having the path cached and stored on the project gives nice speedups when the underlying storage system is slow.
* Merge branch 'bschubert/profiler-as-cm' into 'master'bst-marge-bot2019-03-298-169/+143
|\ | | | | | | | | | | | | Cleanup profiler and make is as a context manager Closes #877 and #876 See merge request BuildStream/buildstream!1255
| * Update documentation with the new profile changesbschubert/profiler-as-cmBenjamin Schubert2019-03-291-20/+4
| |
| * Allow nested profilesBenjamin Schubert2019-03-291-6/+28
| | | | | | | | | | | | This adds the ability to launch nested profilers in BuildStream. That means the "BST_PROFILE=all" now works as expected.
| * _profile.py: Remove unused profiler topicsBenjamin Schubert2019-03-291-3/+0
| |
| * Rework profiler to act as a context managerBenjamin Schubert2019-03-297-146/+117
|/
* Merge branch 'tpollard/945' into 'master'bst-marge-bot2019-03-297-55/+129
|\ | | | | | | | | | | | | Add initial TestArtifact() abstraction class to testutils Closes #945 See merge request BuildStream/buildstream!1252
| * plugintestutils/runcli.py: Add TestArtifact() 'abstraction' classTom Pollard2019-03-295-48/+123
| | | | | | | | | | | | | | | | | | This adds the initial Artifact 'abstractions', taking existing test methods into a dedicated Artifact class. The class is accessed via an instance attribute of the generic Cli() fixture. This is in preperation for AaaP, to ensure existing tests pass whilst creating an entry point for further abstractions when necessary.
| * _cas/cascache.py: Add opt check_exists arg to _reachable_refs_dir()Tom Pollard2019-03-292-7/+6
|/ | | | | | | | | | | By default the reachable set() includes all file digests from a given reachable directory/tree digest, however this does not guarantee the file objects are actually reachable. testutils/artifactshare.py has_artifact() ascertained if a given artifact is available by ensuring any 'reachable' digests are actually on the filesystem, this is made redudant with the optional check_exists objpath check during the reachable set creation.
* Merge branch 'tpollard/pin-sphinx' into 'master'bst-marge-bot2019-03-291-1/+1
|\ | | | | | | | | tox.ini: pin [testenv:docs] sphinx dependency to 1.8.5 See merge request BuildStream/buildstream!1267
| * tox.ini: pin [testenv:docs] sphinx dependency to 1.8.5Tom Pollard2019-03-291-1/+1
|/ | | | | | In sphinx 2.0.0 sphinx.apidoc has moved to sphinx.ext.apidoc, which causes a build import error. Pin the version to 1.8.5 to allow docs to be built until issues can be fixed.
* Merge branch 'jjardon/overnigth_tests_fix' into 'master'Javier Jardón2019-03-281-1/+1
|\ | | | | | | | | | | | | .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth tests Closes #976 See merge request BuildStream/buildstream!1265
| * .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth testsJavier Jardón2019-03-281-1/+1
|/
* Merge branch 'jennis/track_is_overworking' into 'master'bst-marge-bot2019-03-281-0/+40
|\ | | | | | | | | Write test to ensure that track does not needlessly rewrite See merge request BuildStream/buildstream!1235
| * tests: Add a track test to ensure that we do not needlessly overwrite filesjennis/track_is_overworkingJames Ennis2019-03-281-0/+40
|/
* Merge branch 'shared/yaml-rework' into 'master'bst-marge-bot2019-03-2857-1599/+1821
|\ | | | | | | | | | | | | YAML New World Order Closes #591 See merge request BuildStream/buildstream!1257
| * _basecache.py: Raise appropriate error if 'artifacts' is not dict/list of dictsshared/yaml-reworkJames Ennis2019-03-271-9/+10
| |
| * _cachekey.py _artifactelement.py: Create new is_key() and use itDaniel Silverstone2019-03-272-2/+27
| | | | | | | | | | | | | | | | To more cleanly check if a given string could be a cache key, create a new _cachekey.is_key() function and use that in _artifactelement's verify_artifact_ref() function. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * The new YAML World OrderDaniel Silverstone2019-03-2754-1027/+1774
| | | | | | | | | | | | | | | | | | Replace YAML internals with a new Node type, and refactor everything to use it cleanly. This work was also by James Ennis <james.ennis@codethink.co.uk> Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * YAML Cache: Remove the YAML CacheDaniel Silverstone2019-03-275-572/+21
|/ | | | | | | The new YAML world order doesn't need the YAML cache, so remove it before we begin the change. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'juerg/partial-cas-remote' into 'master'bst-marge-bot2019-03-275-220/+177
|\ | | | | | | | | Initial partial CAS support for remote execution See merge request BuildStream/buildstream!1232
| * cascache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-17/+0
| |
| * _artifactcache.py: Remove unused method push_directory()Jürg Billeter2019-03-271-24/+0
| |
| * tests/artifactcache/push.py: Remove push_directory() testJürg Billeter2019-03-271-108/+0
| | | | | | | | push_directory() is unused and will be removed.
| * _sandboxremote.py: Fetch missing blobs from artifact serversJürg Billeter2019-03-271-4/+28
| | | | | | | | | | | | | | | | | | | | If any blobs required for the action are missing both on the remote execution CAS server and in the local cache, attempt to fetch these blobs from the configured artifact servers. Artifacts are currently still guaranteed to be complete in the local cache. However, this is in preparation for the planned partial CAS support.
| * _artifactcache.py: Add fetch_missing_blobs() methodJürg Billeter2019-03-271-0/+21
| |
| * cascache.py: Make send_blobs() publicJürg Billeter2019-03-271-7/+15
| |
| * cascache.py: Do not fetch files in _fetch_directory()Jürg Billeter2019-03-271-18/+21
| | | | | | | | | | Separate fetching of directory objects and fetching of files in preparation of partial CAS support.
| * cascache.py: Add fetch_blobs() methodJürg Billeter2019-03-271-2/+42
| |
| * cascache.py: Add local_missing_blobs() methodJürg Billeter2019-03-271-0/+17
| |
| * cascache.py: Add remote_missing_blobs_for_directory() methodJürg Billeter2019-03-271-19/+33
| | | | | | | | Extracted from _send_directory().
| * casremote.py: Add missing_blobs parameter to _CASBatchRead.send()Jürg Billeter2019-03-271-3/+7
| | | | | | | | | | This optionally returns a list of missing blobs instead of raising an error.
| * casremote.py: Remove unused verify_digest_on_remote() methodJürg Billeter2019-03-271-22/+0
| |
| * _sandboxremote.py: Remove call to verify_digest_on_remote()Jürg Billeter2019-03-271-3/+0
|/ | | | | | If upload is not successful, there should be an error. Skip the extra round trip verifying the upload of the top-level directory object was successful.
* Merge branch 'aevri/copyright' into 'master'bst-marge-bot2019-03-271-0/+19
|\ | | | | | | | | testutils/yaml.py: add copyright notice See merge request BuildStream/buildstream!1263
| * testutils/yaml.py: add copyright noticeAngelos Evripiotis2019-03-271-0/+19
|/
* Merge branch 'aevri/junction_load_provenance' into 'master'bst-marge-bot2019-03-2610-19/+116
|\ | | | | | | | | loader: provenance in LoadErrors from _get_loader See merge request BuildStream/buildstream!1237