| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
This affects the cache key version (updated to 7) and introduces
a dependency on `ujson` which is BSD licenced as of the version
locked in `requirements.txt`
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Artifacts can be cached explicitly with an empty `build tree` when
built via the cli main options or user config for all or only
successful build artifacts. Default behaviour is to still create
and cache all expected buildtrees.
element.py: _cache_artifact() Check if context for cache_buildtrees
has been set to always or failure with a corresponding build
result, if not skip attempting to export the build-root. Element
types without a build-root are cached with an empty buildtree
regardless. Update _stage_sources_at() to warn the user that the
buildtree import is empty.
tests/integration: Add test to artifact.py for the optional caching
of buildtree content from bst build. Rename build-tree.py to
shellbuildtrees.py to reflect included test cases, add test for
empty buildtree warning and failure option.
NEWS: Add entry for new option.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
_context.py: Add cache_buildtrees global user context, the default
of which is set to by default to 'always' via the addition of
cache-buildtrees to userconfig.yaml cache group. 'failure' & 'never'
can be given as valid options.
app.py & cli.py: Add --cache-buildtrees as a bst main option, which
when passed with a valid option can override the default or user
defined context for cache_buildtrees.
tests/completions/completions.py: Update for the added flag.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Returning only empty directories leads to inconsistencies when computing
a manifest by combining results from multiple list_relative_paths()
calls as done by the compose plugin.
I.e., the same directory may be empty in one dependency and non-empty in
another dependency. The merged file list will still contain that
directory even though it's no longer empty.
This inconsistency causes problems when calculating differences between
manifests. Returning all directories fixes these inconsistencies.
This is a change in API behavior.
|
| |
|
|
|
| |
This is required to fix tests with the following commit that changes
list_relative_paths() to return all directories.
|
| |
|
|
|
|
|
|
|
| |
Most split rules already included the relevant directories themselves in
addition to the directory contents. Add the missing bin, sbin, and
libexec directories.
This is required to fix tests with the following commit that changes
list_relative_paths() to return all directories.
|
| |
|
|
|
|
|
|
| |
This commit removes the method ArtifactCache.get_artifact_fullname()
and replaces it with Element.get_artifact_name()
Given a key, we are now able to construct the full name of any of an
element's artifacts.
|
| |
|
|
| |
This was broken by 5ef19a0b31df84caed1e41719ef7ea5c6bd8a8bc.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At present, there isn't an easy way to print anything from `bst show`
that would give the users an idea of what the dependency graph looks
like. One could use things like `--deps build`, but that will just print
a list, without any information about the dependency edges.
Add `%{deps}`, `%{build-deps}` and `%{runtime-deps}` format strings to
`bst show` that would simply print the list of all dependencies, build
dependencies and runtime dependencies respectively.
Summary of changes:
* buildstream/_frontend/cli.py: Add help for new format symbols.
* buildstream/_frontend/widget.py: Add support for new format symbols
for dependencies.
* tests/frontend/show.py: Add tests for new format symbols.
|
| |
|
|
|
| |
Don't create and remove temp dirs unnecessarily when they are not used,
looks like these were just copy-pastes without intended side-effects.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, include a directory result in an error like this:
mydir is a directory. bst command expects a .bst file.
Note that the file containing the include was not mentioned.
Now we get an error like this instead:
element.bst [line 12 column 0]: Include block references a
directory instead of a file: 'mydir'.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, a missing include would result in an error like this:
Could not find file at not-a-file.include
Note that the file containing the include was not mentioned.
Now we get an error like this instead:
element.bst [line 7 column 5]: Include block references a file that
could not be found: 'not-a-file.include'.
|
| |
|
|
| |
Use the provenance of the include block, instead of the whole node.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We want external plugins to be able to make use of the core testing utils.
This commit exposes the basic utilities which are currently in use in
bst-external plugins. If necessary, more utilities could be exposed in the
future.
Moves the following files from tests/testutils/ to
buildstream/plugintestingutils/:
o runcli.py
o integration.py
As part of this, this commit makes the following changes to runcli.py
and integration.py:
o runcli.py: Fix linting errors
o runcli.py: Add user facing documentation
o Integration.py: Add user facing documentation
|
| |
|
|
| |
The 'wallclock-us' is a option that we thought we already had.
|
| |
|
|
|
|
| |
The unix specification for unix sockets only allocates ~100 characters
for the length of the path. This may be longer than the path for a CI
runner, or the path to a user's buildstream directory.
|
| | |
|
| |
|
|
|
|
| |
Providing a remote will limit build's pull/push remote actions to
the given remote specifically, ignoring those defined via user or
project configuration.
|
| |
|
|
|
|
|
|
|
| |
This patch also uncovered the fact that our test_filter_deps_ok() test
has been inaccurate. Thus the element built in this test
(deps-permitted.bst) has been modified so that it build depends on the
input.bst element, as it should.
tests/filter.py: Ensure deps_ok test passes
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We use output of `git --version` to determine if we can run some tests
that rely on features from newer git versions. Usually, we expect the
output to be like:
git version 2.17.2
On some platforms, like MacOS, there could be a suffix after the version
string, so that it looks something like:
git version 2.17.2 (Apple Git-113)
This causes things to fail like so:
ValueError: invalid literal for int() with base 10: '2 (Apple Git-113)\n'
Fix logic around `HAVE_OLD_GIT` such that we split the output of
`git --version` without limit on how many times we split. Previously we
used to split only twice so the suffixes like `(Apple Git-113)` are not
part of the parsed version.
|
| |
|
|
|
|
|
| |
Also bump the element's version so cached artifacts would be
invalidated.
Fixes #883
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provide bst shell --use-buildtree the ability to attempt to
acquire missing buildtrees, given respective option, user
pull-buildtree context and remote availability.
_frontend/cli.py: Refactor logic for determining --use-buildtree
option with given opportunity to attempt pulling a non-local
buildtree. Element loaded with artifact_config to allow remote
querying.
_stream.py: With given user option and element state, construct
PullQueue to fetch remote buildtree. Continue or Error without
buildtree if cannot be attained.
tests/integration/build-tree.py: Update to support new usecases
|
| |
|
|
|
| |
Now that 'bst track' is obsolete, change guidance to refer to the
replacement 'bst source track' instead.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead only rely on the headroom to be enough to protect against
out of space conditions. The headroom can become configurable as
a separate step is required.
The changes to achieve this are:
* Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().
* ArtifactCache.full() now also reports True if the available
space on the artifact cache volume is smaller than the headroom.
This ensures jobs get triggered to cleanup the cache when
reaching the end of the disk.
* When loading the artifact quota, it is now only an error if
the quota exceeds the overall disk space, not if it does not
fit in the available space.
It is still a warning if the quota does not fit in the
available space on the artifact cache volume.
* Updated scheduler.py and buildqueue.py for the API rename
* tests: Updated the artifactcache/expiry.py test for its
expectations in this regard.
Added a new test to test an error when quota was specified to
exceed total disk space, and adjusted the existing tests to
expect a warning when the quota does not fit in the available
space.
This fixes issue #733 and #869.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will benefit from a better UtilError being raised, and
and turns the artifact cache's local function into a one liner.
The loop which finds the first existing directory in the
given path has been removed, being meaningless due to the
call to os.makedirs() in ArtifactCache.__init__().
The local function was renamed to _get_cache_volume_size() and
no longer takes any arguments, which is more suitable for the
function as it serves as a testing override surface for
unittest.mock().
The following test cases which use the function to override
the ArtifactCache behavior have been updated to use the new
overridable function name:
tests/artifactcache/cache_size.py
tests/artifactcache/expiry.py
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to test when core activities occur by parsing the stderr
in tests, we should make the messages conform more.
At the same time, this restores alignment of columns in core
messages with the element processing related messages.
Also, _scheduler/scheduler.py is updated to make it's activity names
conform to the (current) 5 character limit for the sake of alignment.
The tests/frontend/logging.py test gets it's regexes updated for
the log lines it checks for in stderr.
|
| |
|
|
|
| |
Test that `bst build` does not fail in a project where the list of
default targets includes a junction (junctions cannot be built).
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Previously, there was no way of detecting whether fetching happened, as
an element with an open workspace will not be fetched.
|
| |
|
|
|
| |
Previously, it merely tested that buildstream did not fall other,
rather than whether it did anything useful.
|
| |
|
|
|
|
|
|
|
| |
This tests how BuildStream reacts when it fails to write the tracking
results to the element files or project.refs file, which is an operation
that plugins do not play a part in.
As such, removing the per repo kind parameterization from this test
as multiple runs are redundant here.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Instead of a single target, we can always provide a single target
in a list.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is not an error related to loading data, like a parse error
in the quota specification is, but a problem raised by the artifact
cache - this allows us to assert more specific machine readable
errors in test cases (instead of checking the string in stderr, which
this patch also fixes).
This also removes a typo from the error message in the said error.
* tests/artifactcache/cache_size.py
Updated test case to expect the artifact error, which consequently
changes the test case to properly assert a machine readable error
instead of asserting text in the stderr (which is the real, secret
motivation behind this patch).
* tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()
Now expect the artifact error for the tests which check configurations
which create caches too large to fit on the disk.
|
| |
|
|
| |
Instead of sitting mysteriously alone in internals/utils.py
|
| |
|
|
|
|
|
|
|
|
| |
This test has one test case which is marked as an integration test,
and the other is not an integration test, but was using the integration
cli. The integration cli does not work correctly if not run in
integration mode.
This was causing an error locally in conftest.py when trying to
create a tmpdir inside a nonexisting integration cache directory.
|
| |
|
|
|
|
|
|
|
|
|
| |
This commit marks 'bst checkout' as a 'hidden' command. If used,
the user will be prompted to use the new 'bst artifact checkout'
command.
All tests which used 'bst checkout' have been modified to use
the new artifact sub-command.
This partially solves #822.
|
| |
|
|
|
| |
This commit also ensures that if we try to use the 'old' commands,
BuildStream will fail and instruct the user to use the new command.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
test_track_invalid_submodule depends on being able to remove a submodule
by `git rm $submoduledir`, but old versions of git don't update .gitmodules
so BuildStream still thinks there's a submodule present.
For expediency the test is skipped rather than changed to manually remove
the entry from .gitmodules if git hasn't done it,
since in the common case git is new enough to do that itself.
test_git_describe expects --first-parent to find another tag,
but `bst track` will gracefully degrade if the option doesn't work
so a different history will be retained with old versions of git.
It's of marginal benefit to add additional cruft
to test for different output on old versions of git that won't persist forever.
|
