summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _signals.py: allow calling signal handler from non-main threadsbschubert/signalsBenjamin Schubert2020-06-232-8/+12
| | | | | | | | | | | | | | | | | | This modifies the signal terminator so that it can be called from any thread. This checks that either: - The signal handler is already in place - Or the caller is in the main thread, allowing to set the signal handler. This also removes the exact callback that was added instead of removing the last one, and fixes the `suspend_handler` to do the same. This is required, as we don't know which interleaving of calls will be done, and we can't guarantee that the last one is the right one to remove
* Merge branch 'bschubert/cleanups' into 'master'bst-marge-bot2020-06-182-11/+4
|\ | | | | | | | | Small cleanups See merge request BuildStream/buildstream!1966
| * conftest.py: Ensure the `basetemp` is a resolved pathbschubert/cleanupsBenjamin Schubert2020-06-171-1/+1
| | | | | | | | | | Otherwise some of BuildStream's config will fail and it is therefore impossible to just run `pytest tests/`
| * _signals.py: Remove code for python3.5 supportBenjamin Schubert2020-06-171-10/+3
|/
* Merge branch 'tristan/load-context' into 'master'bst-marge-bot2020-06-169-138/+166
|\ | | | | | | | | _loader: Adding LoadContext See merge request BuildStream/buildstream!1964
| * _loader: Adding LoadContexttristan/load-contextTristan van Berkom2020-06-169-138/+166
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of passing around many details though calling signatures throughout the loader code, create a single LoadContext object which holds any overall loading state along with any values which are constant to a full load process. Overall this patch does: * _frontend/app.py: No need to pass Stream.fetch_subprojects() along anymore * _loader/loadelement.pyx: collect_element_no_deps() no longer takes a task argument * _loader/loader.py: Now the Loader has a `load_context` member, and no more `_fetch_subprojects` member or `_context` members Further, `rewritable` and `ticker` is no longer passed along through all of the recursing calling signatures, and `ticker` itself is finally removed because this has been replaced a long time ago with `Task` API from `State`. * _pipeline.py: The load() function no longer has a `rewritable` parameter * _project.py: The Project() is responsible for creating the toplevel LoadContext() if one doesn't exist yet, and this is passed through to the Loader() (and also passed to the Project() constructor by the Loader() when instantiating subprojects). * _stream.py: The `Stream._fetch_subprojects()` is now private and set on the project when giving the Project to the Stream in `Stream.set_project()`, also the Stream() sets the `rewritable` state on the `LoadContext` at the earliest opportunity, as the `Stream()` is the one who decides this detail. Further, some double underscore private functions are now regular single underscores, there was no reason for this inconsistency. * tests/internals/loader.py: Updated for API change
* Merge branch 'tristan/nuke-pickle-jobber' into 'master'Tristan Van Berkom2020-06-1619-573/+1
|\ | | | | | | | | Completely abolish job pickling. See merge request BuildStream/buildstream!1965
| * Completely abolish job pickling.tristan/nuke-pickle-jobberTristan van Berkom2020-06-1519-573/+1
|/
* NEWS: Preparing for 1.93.4 snapshot1.93.4Tristan van Berkom2020-06-121-3/+10
|
* Merge branch 'tristan/remove-some-internal-api' into 'master'Tristan Van Berkom2020-06-124-96/+46
|\ | | | | | | | | Remove a not very important message from the cli See merge request BuildStream/buildstream!1963
| * _project.py: Reorganize methodstristan/remove-some-internal-apiTristan van Berkom2020-06-111-31/+46
| | | | | | | | | | | | Add some comments clearly separating public and private methods, and moving the _validate_node() private method into the private section, renaming it _validate_toplevel_node() for better clarity.
| * _project.py: Remove invoked_from_workspace_element()Tristan van Berkom2020-06-111-8/+0
| | | | | | | | This is no longer needed.
| * _stream.py: Remove workspace_is_required()Tristan van Berkom2020-06-111-14/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This API was only used to issue a not very useful message in the CLI when removing a workspace the user happened to have invoked BuildStream from. The API also didn't really make any sense, being documented as: "Checks whether the workspace belonging to element_name is required to load the project" This in fact meant whether having that workspace open was required for BuildStream to be invoked, due to the metadata encoded into the workspace directory.
| * _frontent/cli.py: Removed message to user informing the workspace is closed.Tristan van Berkom2020-06-111-13/+0
| | | | | | | | | | | | | | | | | | When closing a workspace while the `workspace close` command was issued from the closed workspace, we go to great lengths to inform the user that they can no longer run bst commands from the closed workspace directory. This is not worthwhile, the user will know this the next time they run `bst` anyway.
| * tests/frontend/workspace.py: Removing testsTristan van Berkom2020-06-111-30/+0
|/ | | | | | Remove tests which check for a user message to be issued upon closing a workspace who's metadata was used to launch BuildStream and find the BuildStream project directory.
* Merge branch 'tristan/reorganize-project-context-init' into 'master'bst-marge-bot2020-06-102-11/+8
|\ | | | | | | | | Reorginize some __init__() functions See merge request BuildStream/buildstream!1961
| * _project.py: reorganize __init__()Tristan van Berkom2020-06-101-8/+5
| | | | | | | | | | | | | | | | Put the private members in the private section and the public members in public section. The exception is `_context` which is resolved early, and this is required by other code which runs in __init__().
| * _context.py: Put the source_cache_specs close to the other cache specsTristan van Berkom2020-06-101-3/+3
|/ | | | Instead of mixing it up with the directories
* Merge branch 'juerg/buildbox' into 'master'bst-marge-bot2020-06-101-5/+5
|\ | | | | | | | | .gitlab-ci.yml: Update BuildBox components to 0.0.9 See merge request BuildStream/buildstream!1962
| * .gitlab-ci.yml: Update BuildBox components to 0.0.9Jürg Billeter2020-06-101-5/+5
|/
* Merge branch 'juerg/object-file-mode' into 'master'bst-marge-bot2020-06-104-36/+45
|\ | | | | | | | | cascache.py: Fix file modes in checkout() See merge request BuildStream/buildstream!1959
| * cascache.py: Fix file modes in checkout()Jürg Billeter2020-06-103-19/+25
| | | | | | | | | | Do not copy file mode from casd object file. Do not change non-executable mode bits if file is executable.
| * utils.py: Make copystat optional in safe_copy()Jürg Billeter2020-06-101-15/+17
| |
| * utils.py: Fix error handling in safe_copy()Jürg Billeter2020-06-101-4/+5
|/ | | | Catch errors raised by `shutil.copyfile()`.
* Merge branch 'scottclarke/remote-build-message' into 'master'bst-marge-bot2020-06-101-27/+30
|\ | | | | | | | | | | | | Improved remote build log message Closes #1336 See merge request BuildStream/buildstream!1955
| * sandbox/_sandboxremote.py: Improved log messagesscottclarke/remote-build-messageScott Clarke2020-06-101-27/+30
|/ | | | | | | | | | | | | | The log message which shows that the remote build is ongoing has been extended to include the name of the element to which the remote build corresponds. Likewise the message indicating that an action result has been found in the action cache will also no include the element name. A new timed_activity() call has been added to show when missing blobs are being determined and uploaded. Fixed #1336
* Merge branch 'tristan/get-loader-provenance' into 'master'bst-marge-bot2020-06-105-34/+25
|\ | | | | | | | | Make provenance mandatory in `Loader.get_loader()` See merge request BuildStream/buildstream!1960
| * _loader/loader.py: Make provenance a positional argumentTristan van Berkom2020-06-103-8/+6
| | | | | | | | | | | | This forces the `provenance` to `Loader.get_loader()` to be a mandatory argument, ensuring that there are never any callers which fail to provide provenance.
| * _includes.py: Propagate provenance through Loader.get_loader()Tristan van Berkom2020-06-102-26/+19
| | | | | | | | | | | | | | | | | | | | | | Instead of raising a customized error message which adds little value to the provenance, just pass the provenance along. This is important so that the Loader is aware of the provenance of loaded junctions, so that it can more precisely report errors about conflicting junctions when includes cause conflicts. This commit also adjusts tests/format/includes.py
| * _yaml.pdx: Remove false presumption from EISDIR error messageTristan van Berkom2020-06-101-1/+1
|/ | | | | | | | | | Change the error message to reflect that the file is a directory, do not suggest that it should be a .bst file, it should not be a .bst file. A .bst file might be expected from the loader code, which can feel free to express that, but other files can be loaded, for example include files.
* Merge branch 'tristan/element-full-paths' into 'master'bst-marge-bot2020-06-0831-258/+555
|\ | | | | | | | | Support full paths See merge request BuildStream/buildstream!1956
| * docs: Updating documentation regarding element names and addressing.Tristan van Berkom2020-06-083-61/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit does the following: * Move the section about rules on element naming to the beginning of the document instead of the end. Element names are probably the first thing you need to know about elements, not the last. * Expand this element naming section to also speak about element addressing, and how to express element paths and how these can be used across the boundaries of multiple junctions. It helps to have a single place to discuss this and link to in the documentation, instead of repetitively explaining in multiple places that one must specify a filename relative to a project's element path. * Update documentation about dependencies to refer to the element naming and addressing documentation. * Update the documentation in the `link` and `junction` elements to link back to the element name documentation.
| * tests/format/include.py: Test including of files using full pathsTristan van Berkom2020-06-0811-0/+54
| |
| * tests/format/link.py: Testing full path link targetsTristan van Berkom2020-06-085-1/+23
| | | | | | | | | | | | | | | | * Test that we succeed to load links with full path targets * Test that we get correct provenance information in errors when linking to non-existing elements in existing subprojects, using full paths.
| * tests/plugins/loading.py: Testing junction plugins with full pathsTristan van Berkom2020-06-081-1/+109
| | | | | | | | | | | | | | | | | | | | Added tests to cover: * Successfully loading plugins with full paths in junction origins * Failing to load missing plugins in sub-junctions which are successfully loaded * Failing to load missing plugins in sub-junctions which are invalid and couldnt load
| * tests/format/junctions.py: Test loading dependencies and targets with full pathsTristan van Berkom2020-06-085-0/+66
| |
| * _includes.py: Use rsplit to split included yaml instead of splitTristan van Berkom2020-06-081-1/+1
| | | | | | | | | | Need to consider only the last element of a path as the filename while the rest of the path is a junction path.
| * _pluginfactory/pluginoriginjunction.py: Call get_loader() with provenance.Tristan van Berkom2020-06-081-1/+1
| | | | | | | | | | This is important for reporting errors due to missing files while loading a junction.
| * _loader.py: Support full path dependencies and targetsTristan van Berkom2020-06-082-191/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit does the following: * Moves the `get_loader()` implementation to a private `_get_loader()` implementation again, without changing it. * Replaces the public `get_loader()` method with a simple one which parses the passed junction name and walks the `:` splits until the ultimate loader is found (by calling _get_loader() internally). * Removes an assertion in the Dependency constructor about having multiple `:` in the element name, which is now allowed, and adjusts the Dependency constructor to properly rsplit() the passed dependency target from the right, storing the full junction path in dep.junction and the target only in the dep.target. * Adds missing `provenance` argument to Loader._parse_name(), ensuring that the provenance of where a target name was found is propagated through `get_loader()` and `_load_file()` preserving the original provenance of a loaded element name.
| * plugin.py: Load full element names recursively.Tristan Van Berkom2020-06-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | When opening up the possibility of loading the same junction twice, it becomes possible to have multiple elements with the same name following the current implementation logic which enforces that you must have a toplevel junction to coalesce junctions and synchronize their configuration. Since we no longer coalesce junction names, we need to allow elements to be reported in the UI with their full depth.
| * _loader/loadelement.pyx: Load full element names recursively.Tristan Van Berkom2020-06-081-1/+1
|/ | | | | | | | | | | When opening up the possibility of loading the same junction twice, it becomes possible to have multiple elements with the same name following the current implementation logic which enforces that you must have a toplevel junction to coalesce junctions and synchronize their configuration. Since we no longer coalesce junction names, we need to allow elements to be reported in the UI with their full depth.
* Merge branch 'abderrahim/protect-blobs' into 'master'bst-marge-bot2020-06-081-2/+2
|\ | | | | | | | | | | | | casdprocessmanager.py: make sure --protect-session-blobs is always passed Closes #1268 See merge request BuildStream/buildstream!1958
| * casdprocessmanager.py: make sure --protect-session-blobs is always passedabderrahim/protect-blobsAbderrahim Kitouni2020-06-081-2/+2
|/ | | | | | an indentation error meant it was only passed when quota was defined Fixes #1268
* Merge branch 'juerg/sandbox-drop-bwrap' into 'master'bst-marge-bot2020-06-0330-2753/+75
|\ | | | | | | | | Drop bwrap sandboxing backend See merge request BuildStream/buildstream!1952
| * Drop Platform subclasses and BST_FORCE_BACKENDjuerg/sandbox-drop-bwrapJürg Billeter2020-06-037-184/+18
| | | | | | | | | | With the use of buildbox-run on all platforms, there is no longer any platform-specific code in the Platform subclasses.
| * Drop sandbox selection and BST_FORCE_SANDBOXJürg Billeter2020-06-037-112/+22
| | | | | | | | | | | | | | buildbox-run is the only local sandbox and there are no plans to add other sandboxing backends in the future. New platforms can be supported by new buildbox-run implementations without requiring any changes in BuildStream.
| * NEWS: Add note about dropping bwrap sandboxing backendJürg Billeter2020-06-031-0/+7
| |
| * doc/source/arch_sandboxing.rst: Update for buildbox-runJürg Billeter2020-06-031-33/+24
| |
| * linux.py: Remove unused can_crossbuild() and _uid/_gidJürg Billeter2020-06-031-26/+0
| |
| * Drop unused _fuse packageJürg Billeter2020-06-036-1460/+1
| |