summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* man/: update with changes since Apr 2018aevri/update_manAngelos Evripiotis2018-11-2919-105/+158
| | | | | | | | | | | | | | | | It seems that the regenerated man pages haven't been committed for some time. Add them all in one lump now. Had to manually strip the version number from the generated files. Had to manually fix the NAME field of bst.1 to not be 'bst - None'. Oddly had to invoke click-man directly in order to get the changes to be applied, the usual setup.py route mentioned in contributing.rst didn't work. Eyeballed the result of invoking man on each of the commands, seems good.
* Merge branch 'jmac/no-verify-digests' into 'master'Jim MacArthur2018-11-291-8/+1
|\ | | | | | | | | | | | | _sandboxremote.py: Remove unnecessary tests. Closes #786 See merge request BuildStream/buildstream!976
| * _sandboxremote.py: Remove unnecessary tests.Jim MacArthur2018-11-291-8/+1
|/ | | | | | | push_message and push_directory will both raise assertion failures if they fail to send the digest to the server. Checking the digest returned by each only tests that the content was hashed locally, which cannot reasonably fail.
* Merge branch 'valentindavid/cache_server_fill_up' into 'master'Valentin David2018-11-294-96/+254
|\ | | | | | | | | | | | | Fix cleanup of cache in server when disk is full Closes #678 See merge request BuildStream/buildstream!830
| * Lock cache cleanup in cas servervalentindavid/cache_server_fill_upValentin David2018-11-281-46/+56
| | | | | | | | Cleaning up in parallel might slow down the cleaning process
| * Cleanup cache in cas server more agressivelyValentin David2018-11-283-67/+107
| | | | | | | | | | When there is less than 2GB left, it cleans up have 10GB available. These values are configurable.
| * Move cas server from ref-based to object-based garbage collection.Valentin David2018-11-282-3/+46
| |
| * Update mtimes of objects for requested references.Valentin David2018-11-283-11/+37
| | | | | | | | | | This also remove references when some objects are missing. This is in preparation for the move from reference to object garbage collection.
| * "Fallocate" object disk space to avoid getting NOSPC error when writingValentin David2018-11-281-6/+24
| | | | | | | | | | This locks the temporary object file so that cleanup does not need to be done for every write.
| * Avoid copying temporary file when adding object to CAS in server.Valentin David2018-11-282-18/+26
| | | | | | | | | | | | | | The file is already a temporary file and does not need copy. ENOSPC is thrown during that copy in issue #609. Fixes #678.
| * Fix type error in RPC messagesValentin David2018-11-281-2/+3
| |
| * Make cache clients not fail when a blob is not available.Valentin David2018-11-281-0/+12
| | | | | | | | | | | | | | We plan to make cache incomplete. That is some blobs are missing. For most of cases we will delete references when requested if they are incomplete. But there will be corner cases where objects are removed after the reference is requested.
| * Use f_bavail to query available space. Not f_bfree.Valentin David2018-11-281-1/+1
|/ | | | | | f_bfree space might not be usable. In practice we see failures in big disks because f_bfree is over 2GB and f_bavail is 0. We get ENOSPC if writing on disk then.
* Merge branch 'valentindavid/ruamel_0_15_41' into 'master'Valentin David2018-11-281-1/+8
|\ | | | | | | | | setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52 See merge request BuildStream/buildstream!975
| * setup.py: require ruamel.yaml >= 0.15.41 < 0.15.52valentindavid/ruamel_0_15_41Valentin David2018-11-281-1/+8
|/
* Merge branch 'jmac/remote_execution_split' into 'master'Jim MacArthur2018-11-2712-116/+288
|\ | | | | | | | | | | | | Split remote execution from artifact cache Closes #750 See merge request BuildStream/buildstream!946
| * Docs: Correct URLs in artifact cache exampleJim MacArthur2018-11-271-2/+2
| | | | | | | | | | The port number must come after the hostname and before the path. The path is ignored by CASCache anyway, so including it is misleading.
| * artifactcache.py: Remove unused verify_digest_pushedJim MacArthur2018-11-271-28/+0
| |
| * tests/sandboxes/remote-exec-config.py: New test.Jim MacArthur2018-11-274-0/+102
| | | | | | | | | | Tests that we get a sensible error message when we specify incomplete or wrong information in the configuration for remote execution.
| * Documentation: Update to show new config optionsJim MacArthur2018-11-271-3/+17
| |
| * _sandboxremote: Remove references to artifact cache.Jim MacArthur2018-11-274-37/+109
| | | | | | | | | | Uses the new config options introduced earlier to specify remote exec storage and execution service.
| * artifactcache.py, _context.py: Move CASCache object into contextJim MacArthur2018-11-272-2/+9
| | | | | | | | | | | | Since the artifact cache and remote execution share the same local CAS store, they should share the same CASCache object. Moving this into context allows us to do this.
| * artifactcache: Move ArtifactCacheSpec code into CASRemoteSpec.Jim MacArthur2018-11-272-45/+50
|/ | | | | | There is nothing in ArtifactCacheSpec that's actually specific to artifacts, so I've made it a CAS class so we can use the same spec for remote execution.
* Merge branch 'juerg/command-batching' into 'master'Jürg Billeter2018-11-2714-108/+498
|\ | | | | | | | | | | | | Command batching Closes #675 See merge request BuildStream/buildstream!915
| * NEWS: Add entry for command batchingJürg Billeter2018-11-271-0/+3
| |
| * sandbox/_sandboxremote.py: Implement command batchingJürg Billeter2018-11-271-1/+69
| | | | | | | | Execute batched commands in a single shell script.
| * compose.py: Support batching for integration commandsJürg Billeter2018-11-271-2/+3
| |
| * scriptelement.py: Support batching for integration and build commandsJürg Billeter2018-11-271-25/+26
| |
| * buildelement.py: Support batching for integration and build commandsJürg Billeter2018-11-272-13/+12
| |
| * element.py: Add batch_prepare_assemble() methodJürg Billeter2018-11-271-6/+47
| | | | | | | | | | This allows batching of all sandbox commands aross prepare() and assemble().
| * element.py: Support command batching for integration commandsJürg Billeter2018-11-271-8/+7
| |
| * element.py: Defer workspace.prepared setting in case of pending commandsJürg Billeter2018-11-271-1/+6
| |
| * element.py: Cache artifact also on SandboxCommandErrorJürg Billeter2018-11-271-2/+2
| |
| * element.py: Pass element object to sandboxJürg Billeter2018-11-271-0/+2
| | | | | | | | This allows the sandbox to output messages using the plugin id.
| * sandbox/sandbox.py: Add command batching APIJürg Billeter2018-11-273-6/+251
| | | | | | | | This adds the batch() context manager.
| * scriptelement.py: Use SandboxFlags.NONE instead of 0Jürg Billeter2018-11-271-2/+5
| |
| * element.py: Use SandboxFlags.NONE instead of 0Jürg Billeter2018-11-271-1/+2
| |
| * sandbox/sandbox.py: Add SandboxFlags.NONEJürg Billeter2018-11-271-0/+4
| | | | | | | | This improves readability.
| * sandbox: Deduplicate code to process cwd, env, and command argumentsJürg Billeter2018-11-275-45/+38
| |
| * tests/integration/manual.py: Add test for command loggingJürg Billeter2018-11-271-0/+25
|/
* Merge branch 'juerg/local-junctions' into 'master'Jürg Billeter2018-11-273-5/+31
|\ | | | | | | | | Optimization for local junctions See merge request BuildStream/buildstream!290
| * _loader.py: Use Source._get_local_path() for subprojectsJürg Billeter2018-11-271-5/+11
| | | | | | | | | | This eliminates unneeded copies of the project directory for junctions with a single local source.
| * local.py: Implement _get_local_path()Jürg Billeter2018-11-271-0/+3
| |
| * source.py: Add optional _get_local_path() methodJürg Billeter2018-11-271-0/+17
|/
* Merge branch 'jmac/fix-test-hangs-2' into 'master'Jim MacArthur2018-11-273-27/+55
|\ | | | | | | | | Avoid hanging artifact cache tests See merge request BuildStream/buildstream!964
| * tests: Avoid hangs due to exceptions in subprocessesJim MacArthur2018-11-273-27/+55
|/
* Merge branch 'jmac/execution-architecture-docs' into 'master'Jim MacArthur2018-11-264-0/+533
|\ | | | | | | | | Docs: Add remote execution architecture documentation. See merge request BuildStream/buildstream!923
| * Docs: Add remote execution architecture documentation.Jim MacArthur2018-11-264-0/+533
|/
* Merge branch 'valentindavid/overnight-tests-bump-fdo-sdk-version' into 'master'778-bst-example-html-in-the-doc-s-is-out-of-date-and-is-hard-to-maintainJavier Jardón2018-11-221-2/+2
|\ | | | | | | | | Bump version of fd.o SDK and bst_external in overnight tests See merge request BuildStream/buildstream!962
| * .gitlab-ci.yml: Bump version of fd.o SDK and bst_external in overnight testsValentin David2018-11-221-2/+2
|/