| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
This is needed since 629a6e524, and was lost in the conversion to requirements.in
|
| |
|
|
| |
This is needed since 0f2bc3754
|
| |\
| |
| |
| |
| |
| |
| | |
Download buildtrees on demand for bst shell --use-buildtree
Closes #829
See merge request BuildStream/buildstream!1050
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Provide bst shell --use-buildtree the ability to attempt to
acquire missing buildtrees, given respective option, user
pull-buildtree context and remote availability.
_frontend/cli.py: Refactor logic for determining --use-buildtree
option with given opportunity to attempt pulling a non-local
buildtree. Element loaded with artifact_config to allow remote
querying.
_stream.py: With given user option and element state, construct
PullQueue to fetch remote buildtree. Continue or Error without
buildtree if cannot be attained.
tests/integration/build-tree.py: Update to support new usecases
|
| |/
|
|
|
| |
use_artifact_config added as an optional default arg, allowing
for loading of given elements artifact remote config.
|
| |\
| |
| |
| |
| | |
Fixup refs to 'bst track' and 'bst fetch'
See merge request BuildStream/buildstream!1086
|
| | |
| |
| |
| |
| | |
Now that 'bst fetch' is obsolete, change guidance to refer to the
replacement 'bst source fetch' instead.
|
| | |
| |
| |
| |
| | |
Now that 'bst track' is obsolete, change guidance to refer to the
replacement 'bst source track' instead.
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| | |
Generate man pages using tox & update them
Closes #880 and #881
See merge request BuildStream/buildstream!1107
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If the `man` directory is empty, then it won't be copied in the source
distribution, and `list_man_pages()` will throw an exception when trying
to list files in a non-existent directory. This prevents us from
installing the BuildStream package when the man pages are not there.
The most common use-case for this is when we want to re-generate the man
pages but want to install the package before re-generating them.
|
| | |
| |
| |
| |
| |
| | |
Simplify our docs, by requesting users to run `tox -e man` to update man
pages, instead of manually installing `click-man` and running the
command manually.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Notable changes:
* New `source` and `artifact` command groups
* Man pages corresponding to obsolete commands, that were marked as
hidden in Click, have now been removed.
Fixes #881.
|
| |/
|
|
|
|
| |
Previously, one had to manually install `click-man` package, and
remember to run the correct command. Now, we can simply run `tox -e man`
to update the man pages.
|
| |\
| |
| |
| |
| |
| |
| | |
_artifactcache.py: Don't require the quota to be available on disk.
Closes #869 and #733
See merge request BuildStream/buildstream!1106
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead only rely on the headroom to be enough to protect against
out of space conditions. The headroom can become configurable as
a separate step is required.
The changes to achieve this are:
* Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().
* ArtifactCache.full() now also reports True if the available
space on the artifact cache volume is smaller than the headroom.
This ensures jobs get triggered to cleanup the cache when
reaching the end of the disk.
* When loading the artifact quota, it is now only an error if
the quota exceeds the overall disk space, not if it does not
fit in the available space.
It is still a warning if the quota does not fit in the
available space on the artifact cache volume.
* Updated scheduler.py and buildqueue.py for the API rename
* tests: Updated the artifactcache/expiry.py test for its
expectations in this regard.
Added a new test to test an error when quota was specified to
exceed total disk space, and adjusted the existing tests to
expect a warning when the quota does not fit in the available
space.
This fixes issue #733 and #869.
|
| |\
| |
| |
| |
| | |
Add new 'scheduler' and 'load-selection' profiling topics
See merge request BuildStream/buildstream!1088
|
| | | |
|
| | |
| |
| |
| |
| | |
profile_start() and profile_end() calls have been added to
Stream.load_selection()
|
| |/
|
|
|
| |
profile_start() and profile_end() calls have been incorporated into
Scheduler.run()
|
| |\
| |
| |
| |
| | |
Cache management logging enhancements
See merge request BuildStream/buildstream!1105
|
| | |
| |
| |
| |
| |
| | |
This seems to have been copy/pasted from cascache, and
documents the function to possibly return None if defer_prune
was specified, but this function does not expose defer_prune.
|
| | |
| |
| |
| |
| |
| | |
Updates the known cache size in the main process while the cleanup
process is ongoing, so that the status indicators update live
while the cleanup happens.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Added some useful status messages when:
* Calculating a new artifact cache usage size
* Starting a cleanup
* Finishing a cleanup
Also enhanced messaging about what was cleaned up so far when
aborting a cleanup.
|
| | |
| |
| |
| |
| | |
This also adds some comments around the main status bar heading
rendering function.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
A frontend facing API for obtaining usage statistics.
I would have put this on Stream instead, but the Context
seems to be the de facto place for looking up the artifact cache
in general so let's put it here.
|
| | |
| |
| |
| |
| | |
A simple object which creates a snapshot of current
usage statistics for easy reporting in the frontend.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This will benefit from a better UtilError being raised, and
and turns the artifact cache's local function into a one liner.
The loop which finds the first existing directory in the
given path has been removed, being meaningless due to the
call to os.makedirs() in ArtifactCache.__init__().
The local function was renamed to _get_cache_volume_size() and
no longer takes any arguments, which is more suitable for the
function as it serves as a testing override surface for
unittest.mock().
The following test cases which use the function to override
the ArtifactCache behavior have been updated to use the new
overridable function name:
tests/artifactcache/cache_size.py
tests/artifactcache/expiry.py
|
| | |
| |
| |
| | |
We can streamline this call to os.statvfs() in a few places.
|
| |/
|
|
| |
The artifact cache emits messages, and we want to allow that in preflight.
|
| |\
| |
| |
| |
| | |
Cache management fixes
See merge request BuildStream/buildstream!1091
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
In order to test when core activities occur by parsing the stderr
in tests, we should make the messages conform more.
At the same time, this restores alignment of columns in core
messages with the element processing related messages.
Also, _scheduler/scheduler.py is updated to make it's activity names
conform to the (current) 5 character limit for the sake of alignment.
The tests/frontend/logging.py test gets it's regexes updated for
the log lines it checks for in stderr.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When running any session that has Queues which require Resource.CACHE,
check if our loaded estimated size exceeds the quota, and if so;
lock the Resource.CACHE resource exclusively right away and run
an exclusive initial cache size job.
This ensures we cleanup first before doing anything which might
add to the cache at startup time, if deemed needed.
This is a partial fix for issue #737
|
| |/
|
|
|
|
| |
Don't require the interest to be registered, just discard any interest,
this function just sets a bit in a mask, and is not intended to maintain
a balance like the reserve() function is.
|
| |\
| |
| |
| |
| | |
Add support for default targets
See merge request BuildStream/buildstream!925
|
| | | |
|
| | |
| |
| |
| |
| | |
Test that `bst build` does not fail in a project where the list of
default targets includes a junction (junctions cannot be built).
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
Junctions cannot be built, pulled, or pushed. Specifying a junction on
the command line for these commands will result in an error. However,
junctions may be in the list of default targets, so they need to be
ignored for build, pull, and push commands.
|
| | |
| |
| |
| | |
This filters out junctions from the list of targets.
|
| | |
| |
| |
| |
| |
| | |
_frontend/cli.py: Use new methods.
Based on patches by Phillip Smyth.
|
| | |
| |
| |
| | |
_versions.py: Bump format version
|
| |/
|
|
| |
There is no reason to disallow guess_element() for bst build --all.
|
