summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* loader: removing the NO_PROGRESS objecttristan/remove-loader-no-progress-objectTristan van Berkom2020-05-302-18/+12
| | | | | | | | | | | While adding a mock task object to track progress in some loader tests in commit 17144d84c2b63daf6e3aa9b42c6c773f134e8660, a new `_NO_PROGRESS` object was added as an explicit marker to denote that progress information should not be reported. This complicates the code, None should be a sufficient value for not reporting progress, while still permitting mock task objects to capture progress if desired.
* Merge branch 'tristan/version-specific-config' into 'master'bst-marge-bot2020-05-293-3/+50
|\ | | | | | | | | Support version specific configuration files See merge request BuildStream/buildstream!1941
| * doc/source/using_config.rst: Document support for version specific config files.Tristan van Berkom2020-05-291-0/+11
| |
| * tests/internals/context.py: Test correct config file is chosen.Tristan van Berkom2020-05-291-0/+28
| |
| * _context.py: Support parallel installations of BuildStreamTristan van Berkom2020-05-291-3/+11
|/ | | | | | | | | | Allow users to keep separate buildstream.conf files in their XDG_CONFIG_HOME for separate BuildStream versions, in case they have BuildStream 1 and 2 installed. Like other major version safeguards and warnings, this new addition also supports future major versions if they should appear.
* Merge branch 'bschubert/resolve-public-variables' into 'master'bst-marge-bot2020-05-295-14/+48
|\ | | | | | | | | | | | | element.py: Always expand public data's variables Closes #1310 See merge request BuildStream/buildstream!1943
| * element.py: Always expand public data's variablesBenjamin Schubert2020-05-295-14/+48
|/ | | | | | | | | | | | | | | | | This fixes a bug introduced by d7d18c1a2e454c507afd9e1d3f1358639dd43871, where public data integration commands and others would never get their data expanded and would thus fail to run. The previous however revelead a previous bug, where variables values in public data of elements would not be part of the cache key of an element or it's reverse dependencies, and thus, on variable change, rebuilds would not happen when they would have been needed. This ensures that all the public data is always resolved and part of the element's cache key. This however will bring a rebuild of an element whenever its integration commands variables change, which is simpler to handle than to try to push that responsibility on reverse dependencies, since public data can contain plugin-defined values.
* Merge branch 'juerg/cas-send-directory' into 'master'bst-marge-bot2020-05-281-3/+5
|\ | | | | | | | | cascache.py: Drop unnecessary FindMissingBlobs from _send_directory() See merge request BuildStream/buildstream!1944
| * cascache.py: Drop unnecessary FindMissingBlobs from _send_directory()juerg/cas-send-directoryJürg Billeter2020-05-281-3/+5
|/ | | | | | buildbox-casd `UploadMissingBlobs` will call `FindMissingBlobs` before the actual upload. Having BuildStream call `FindMissingBlobs` as well adds unnecessary overhead.
* Merge branch 'juerg/sast' into 'master'bst-marge-bot2020-05-281-0/+1
|\ | | | | | | | | .gitlab-ci.yml: Configure SAST analyzers See merge request BuildStream/buildstream!1939
| * .gitlab-ci.yml: Configure SAST analyzersJürg Billeter2020-05-281-0/+1
|/ | | | | Disable flawfinder and eslint jobs as this repository contains no C or JavaScript code to be analyzed.
* Merge branch 'juerg/expandsandbox' into 'master'bst-marge-bot2020-05-285-31/+52
|\ | | | | | | | | | | | | element.py: Expand variables in sandbox config Closes #1303 See merge request BuildStream/buildstream!1938
| * tests/integration/sandbox.py: Test variable expansion in sandbox configjuerg/expandsandboxJürg Billeter2020-05-283-0/+25
| |
| * element.py: Expand variables in sandbox configJürg Billeter2020-05-281-3/+5
| |
| * Move sandbox YAML processing from Element to SandboxConfigJürg Billeter2020-05-282-29/+23
|/
* Merge branch 'tristan/junction-docs-touchup' into 'master'bst-marge-bot2020-05-281-4/+4
|\ | | | | | | | | doc/source/format_declaring.rst: Fixing spelling mistake See merge request BuildStream/buildstream!1940
| * doc/source/format_declaring.rst: Fixing spelling mistakeTristan van Berkom2020-05-281-4/+4
|/ | | | | Making some examples more consistent with regards to indentation of YAML, and fixing "recursice -> recursive" spelling mistake.
* Merge branch 'tristan/junction-plugin-origin' into 'master'bst-marge-bot2020-05-2817-237/+774
|\ | | | | | | | | Introduce new `junction` plugin origin See merge request BuildStream/buildstream!1935
| * doc/source/format_project.rst: Documenting the junction plugin origin.tristan/junction-plugin-originTristan van Berkom2020-05-281-0/+63
| |
| * tests/plugins/loading.py: Adding tests for junction plugin originTristan van Berkom2020-05-285-0/+193
| |
| * _pluginfactory/pluginoriginjunction.py: Add support for junction plugin origin.Tristan van Berkom2020-05-283-0/+89
| |
| * _pluginfactory: Delegating the work of locating plugins to the PluginOriginTristan van Berkom2020-05-2810-237/+429
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | This way we split up the logic of how to load plugins from different origins into their respective classes. This commit also: o Introduces PluginType (which is currently either SOURCE or ELEMENT) o Reduces the complexity of the PluginFactory constructor o Kills the loaded_dependencies list and the all_loaded_plugins API, and replaces both of these with a new list_plugins() API. Consequently the jobpickler.py from the scheduler, and the widget.py from the frontend, are updated to use list_plugins(). o Split up the PluginOrigin implementations into separate files Instead of having all PluginOrigin classes in pluginorigin.py, split it up into one base class and separate files for each implementation, which is more inline with BuildStream coding style. This has the unfortunate side effect of adding load_plugin_origin() into the __init__.py file, because keeping new_from_node() as a PluginOrigin class method cannot be done without introducing a cyclic dependency with PluginOrigin and it's implementations.
* Merge branch 'juerg/dynamic-plan' into 'master'bst-marge-bot2020-05-275-14/+139
|\ | | | | | | | | Restore dynamic build plan feature See merge request BuildStream/buildstream!1926
| * tests/frontend/pull.py: Add test for dynamic build planJürg Billeter2020-05-271-0/+47
| | | | | | | | Make sure build-only dependencies are pulled only when necessary.
| * Revert "Simplify queue management"Jürg Billeter2020-05-272-3/+14
| | | | | | | | This reverts commit aa25f6fcf49f0015fae34dfd79b4626a816bf886.
| * Revert "Schedule elements instead of "requiring" them"Jürg Billeter2020-05-274-11/+70
| | | | | | | | This reverts commit 14e32a34f67df754d9146efafe9686bfe6c91e50.
| * Revert "element: remove _is_required()/__required"Jürg Billeter2020-05-271-0/+8
|/ | | | This reverts commit 750aa60ab3809167ff14c479068a328dc477de3e.
* Merge branch 'juerg/node-properties' into 'master'bst-marge-bot2020-05-2715-264/+386
|\ | | | | | | | | Field-based node properties See merge request BuildStream/buildstream!1918
| * .gitlab-ci.yml: Update BuildBox components to 0.0.8juerg/node-propertiesJürg Billeter2020-05-271-5/+5
| |
| * Update node property support to match proto changesJürg Billeter2020-05-279-59/+87
| |
| * _protos: Update local_cas.protoJürg Billeter2020-05-252-54/+53
| |
| * _protos: Update remote_execution.protoJürg Billeter2020-05-253-146/+241
|/
* Merge branch 'tristan/loader-api-fixup' into 'master'bst-marge-bot2020-05-242-197/+197
|\ | | | | | | | | _loader/loader.py: Reoganized public/private methods See merge request BuildStream/buildstream!1936
| * _loader/loader.py: Reoganized public/private methodstristan/loader-api-fixupTristan van Berkom2020-05-242-197/+197
|/ | | | | | | | | | | | | * This file had `clean_caches()` documented as public but the function was actually private `_clean_caches()`: Moved this to the end of the class in the private section. * The `_get_loader()` was marked as private but is in fact public, and used by the project to load cross junction include files. This patch also updates `_includes.py` to call the public `get_loader()` method instead of sneaking in and calling the private `_get_loader()` method (while also removing one redundant line of code from the same function).
* Merge branch 'tristan/improve-frontend-crash-handling' into 'master'bst-marge-bot2020-05-192-6/+29
|\ | | | | | | | | | | | | Improve handling of frontend crashes Closes #1312 See merge request BuildStream/buildstream!1933
| * _scheduler/scheduler.py: Reset the schedule handler at the beginning of ↵Tristan Van Berkom2020-05-191-3/+15
| | | | | | | | | | | | | | | | | | | | real_schedule() In case queuing jobs results in jobs completing, we need to reset the schedule handler at the beginning of the function and not after queueing the jobs. This fixes the failure to exit the main loop in #1312
| * _scheduler: Fix order of launching jobs and sending notifications.Tristan Van Berkom2020-05-192-3/+14
|/ | | | | | | | | | | | | | | | | | | | | | | Sending notifications causes potentially large bodies of code to run in the abstracted frontend codebase, we are not allowed to have knowledge of the frontend from this code. Previously, we were adding the job to the active jobs, sending the notification, and then starting the job. This means that if a BuildStream frontend implementation crashes, we handle the excepting in an inconsistent state and try to kill jobs which are not running. In addition to making sure that active_jobs list adjustment and job starting does not have any code body run in the danger window in between these, this patch also adds some fault tolerance and assertions around job termination so that: o Job.terminate() and Job.kill() do not crash with None _process o Job.start() raises an assertion if started after being terminated This fixes the infinite looping aspects of frontend crashes at job_start() time described in #1312.
* Merge branch 'bschubert/ensure-composite-works-with-variables' into 'master'bst-marge-bot2020-05-196-2/+42
|\ | | | | | | | | node.pyx: Deep clone ScalarNode too See merge request BuildStream/buildstream!1929
| * node.pyx: Deep clone ScalarNode toobschubert/ensure-composite-works-with-variablesBenjamin Schubert2020-05-196-2/+42
|/ | | | | This ensures that when resolving variables, we do not overwrite the values for a different element in the case we were using composition.
* Merge branch 'chandan/fix-subst-non-greedy' into 'master'bst-marge-bot2020-05-182-1/+21
|\ | | | | | | | | | | | | _frontend/profile: Use non-greedy search to substitute variables Closes #1307 See merge request BuildStream/buildstream!1932
| * _frontend/profile: Use non-greedy search to substitute variablesChandan Singh2020-05-182-1/+21
|/ | | | | | | | Use non-greedy search to ensure that we stop at the next closing brace. Otherwise, for a string like `%{variable} {variable}`, the second variable will also get substituted. Fixes #1307.
* Merge branch 'chandan/duplicate-format-deps' into 'master'bst-marge-bot2020-05-183-8/+32
|\ | | | | | | | | | | | | Ensure there are no duplicates in Elements.dependencies() Closes #1308 See merge request BuildStream/buildstream!1931
| * Ensure there are no duplicates in Elements.dependencies()Chandan Singh2020-05-183-8/+32
|/ | | | | | | | | | | | | | When we are not recursing, `Element.dependencies()` uses a much more light weight codepath since it just needs to print the direct dependencies. However, this simple codepath was not accounting for duplicates, in case something is both a build time and run time dependency. One way this manifested itself was in `bst show --format %{deps}`, but it would also affect anything that was using this method to iterate on the dependencies. Fixes #1308.
* Merge branch 'ctolentino/check-casd-process' into 'master'bst-marge-bot2020-05-151-0/+3
|\ | | | | | | | | | | | | Check that buildbox-casd process is not a zombie Closes #1311 See merge request BuildStream/buildstream!1930
| * casdprocessmanager.py: Check that buildbox-casd process is not a zombiectolentino/check-casd-processctolentino82020-05-151-0/+3
|/
* Merge branch 'chandan/cli-deps-2' into 'master'bst-marge-bot2020-05-1436-54/+196
|\ | | | | | | | | CLI: Add support for `--deps build` and `--deps run` in more commands See merge request BuildStream/buildstream!1904
| * Regenerate man pageschandan/cli-deps-2Chandan Singh2020-05-1324-30/+37
| |
| * _frontend/cli: Support "build" and "run" values for `artifact push --deps`Chandan Singh2020-05-133-13/+30
| |
| * _frontend/cli: Support "build" and "run" values for `artifact pull --deps`Chandan Singh2020-05-133-9/+27
| |
| * _frontend/cli: Support "build" and "run" values for `source fetch --deps`Chandan Singh2020-05-139-1/+81
| |