| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |\
| |
| |
| |
| |
| |
| | |
Optimize consistency and state handling
Closes #1126
See merge request BuildStream/buildstream!1739
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
This assert would only be run in development mode and would actually not
mark the source as downloaded. Therefore forcefully mark them as fetched
and remove the assert. Our test suite will catch if there is a problem.
|
| | |
| |
| |
| |
| | |
'Consistency' is not needed anymore, now that we have everything in
place to not use it
|
| | |
| |
| |
| |
| | |
By looking at the flag first, we can avoid expensive checks on whether
the element is cached or not.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This removes the early call to get whether sources are locally cached
in `_update_source_state` by delegating it to the call of `is_cached`.
Once it is cached, the element is assumed to stay that way for the
duration of the pipeline, we can therefore cache the result once it is
true.
Also remove `Consistency.IS_CACHED`, which is not used anywhere else.
|
| | |
| |
| |
| |
| | |
This is not needed now that we have 'is_resolved' and 'is_cached'.
We can therefore drop all calling places and implementations of it.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This removes the need to use consistency in Sources, by asking
explicitely whether the source is cached or not.
This introduces a new public method on source: `is_cached` that needs
implementation and that should return whether the source has a local
copy or not.
- On fetch, also reset whether the source was cached or set if as
cached when we know it was.
- Validate the cache's source after fetching it
This doesn't need to be run in the scheduler's process and can be
offloaded to the child, which will allow better multiprocessing
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`get_consistency` is coarse grained and hard to optimize, in addition
to being un-userfriendly.
This adds a new `is_resolved` that has for default implementation
`get_ref() is not None`, which is true for most sources in BuildStream.
Sources for which this is not true can override the method to give a
more accurate description.
Checking for this before looking whether the source is cached can
reduce the amount of work necessary in some pipeline and opens the
door for more optimizations and the removal of the source state check.
|
| | |
| |
| |
| |
| |
| |
| | |
This replaces the _get_consistency method by two methods:
`_has_all_sources_resolved` and `_has_all_sources_cached` which allows
a more fine grained control on what information is needed.
|
| | |
| |
| |
| |
| |
| | |
'_source_cached' is not explicit enough as it doesn't distinguishes
between sources in their respective caches and sources in the global
sourcecache.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
`get_consistency` doesn't allow being fine grained and asking only for a
specific bit of information.
This introduces methods `is_cached` and `is_resolved` which will be more
flexible for refactoring.
|
| |/ |
|
| |\
| |
| |
| |
| | |
tests/integration/project.conf: Remove unused external plugins
See merge request BuildStream/buildstream!1799
|
| |/
|
|
|
| |
There are no tests that rely on these plugins anymore, so remove the
corresponding declarations from project.conf.
|
| |\
| |
| |
| |
| |
| |
| | |
element.py: Remove unused __staged_sources_directory & assert
Closes #1249
See merge request BuildStream/buildstream!1798
|
| |/ |
|
| |\
| |
| |
| |
| | |
Make message & status rendering be tick driven by default
See merge request BuildStream/buildstream!1745
|
| | |
| |
| |
| | |
Also highlight the change & configuration in NEWS
|
| |/ |
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Simplify test environment declaration
See merge request BuildStream/buildstream!1795
|
| |/
|
|
|
|
|
| |
Defining `EXTERNAL_TESTS_COMMAND` as an extra command makes it easy to
miss it, especially when adding new environments (as I was recently
bitten by it). Instead of that, we can control the environments using
`TOXENV` variable.
|
| |\
| |
| |
| |
| | |
loader.py: Optimize sorting of elements when they are multiple targets
See merge request BuildStream/buildstream!1794
|
| |/
|
|
|
|
|
| |
Currently, with multiple targets, we would go through all the elements
in the target and sort them, even though they might already be sorted.
This ensures we sort every element only once.
|
| |\
| |
| |
| |
| | |
introduce cross-repo standardized source tests
See merge request BuildStream/buildstream!1757
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This runs two versions of the plugins:
- The latest stable is not allowed failures and is run on every platform
- The master version is allowed failure, and only runs on a single
architecture
This also adds a new entrypoint to register source tests to run against
BuildStream.
|
| | |
| |
| |
| |
| | |
This is not part of the test running itself, but more handling
afterwards
|
| | |
| |
| |
| |
| | |
These tests don't do anything, as they have no plugins registered at
all.
|
| | | |
|
| | |
| |
| |
| | |
This stops the tests from being interlinked for BuildStream itself.
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| | |
_sandboxreapi.py: Improve batch command logging
See merge request BuildStream/buildstream!1793
|
| | |
| |
| |
| |
| | |
When commands are batched, the failure message can't include the command
group label with buildbox-run.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
This adds a timed activity message to bring batch command logging for
REAPI-based sandboxes in line with the default sandbox as far as
possible.
Thix fixes 3 xfails with buildbox-run.
|
| |/
|
|
| |
This will be used to improve batch command logging with REAPI.
|
| |\
| |
| |
| |
| |
| |
| | |
scheduler.py: Handle exceptions that are caught under the event loop
Closes #1245
See merge request BuildStream/buildstream!1788
|
| |/
|
|
|
|
| |
The default exception handler of the async event loop bypasses
our custom global exception handler. Ensure that isn't the case,
as these exceptions are BUGS & should cause bst to exit.
|
| |\
| |
| |
| |
| |
| |
| | |
_frontend/widget.py: show_pipeline() Don't show buildable for junction
Closes #1205
See merge request BuildStream/buildstream!1764
|
| |/
|
|
|
| |
Instead output "junction" in magenta for info. Also include test
in format/junctions.py
|
| |\
| |
| |
| |
| |
| |
| | |
Make PipelineSelection a proper enum type
Closes #1135
See merge request BuildStream/buildstream!1787
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PipelineSelection is one of the few stringy types that weren't
converted to FastEnum, presumably because we lacked a mechanism for
only allowing a sub-set of options as CLI arguments.
We've re-designed this since, and as part of the UI/UX refactor we'd
like to generally clean this, but that is probably still a while out.
Since that hasn't happened, for now, this adds a feature to the
FastEnumType that allows specifying only a subset of values is allowed
for a specific command, so that we can use the type as a proper
enum.
We also get rid of a number of accidental uses of strings, and move
PipelineSelection to buildstream.types so that we don't have a
significant import overhead for it.
|
| |\
| |
| |
| |
| | |
tests/artifactcache/config.py: Cast our tmpdir to a string
See merge request BuildStream/buildstream!1790
|
| |/
|
|
| |
Monkeypatch is *really* scared of non-str values. Let's calm it down.
|
| |\
| |
| |
| |
| | |
setup.py: Remove check for bubblewrap
See merge request BuildStream/buildstream!1789
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bubblewrap is not a hard requirement for BuildStream now that it
supports other sandboxing mechanisms, i.e. buildbox-run and remote
execution.
Aside from that, this check is already a bit misplaced in `setup.py`.
Bubblewrap is a runtime requirement, so warning about it at build time
isn't quite right. It makes sense when BuildStream is being installed
from source, but not when it is installed from any binary distribution.
Note that the corresponding sandbox implementation already does this
check at runtime.
|
| |\
| |
| |
| |
| | |
Prepare NEWS for 1.91.3 development snapshot release
See merge request BuildStream/buildstream!1784
|
| |/ |
|
