| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
| |
Calling pull() for a potentially unavailable artifact is no longer
considered an error.
|
| | |
|
| |
|
|
| |
This is in preparation for moving away from summary files.
|
| |
|
|
|
|
|
|
|
|
|
| |
Remotely cached artifacts matching the strict cache key take precedence
over locally cached artifacts matching only the weak cache key. However,
locally cached artifacts were excluded from the build plan, which means
that BuildStream never even checked whether the strict artifact is
available in the remote artifact cache.
This changes planning to keep cached elements in the build plan in
non-strict mode if a remote artifact cache is used.
|
| | |
|
| |
|
|
|
|
| |
Counter-intuitively, Python list += mutates the list. Use explicit
copy() and extend() instead of += to avoid adding project-specific
remotes to the global remote list.
|
| |
|
|
|
|
| |
For reference, this method was added in
commit c41f1093d83eb32606f81005227542699f756b0a, but since
commit 4912ed5f8922b47c1a16c3752eb92a3f1dea76b2 it has been unused.
|
| |
|
|
|
|
|
|
|
|
|
| |
Add `--soft` option to `bst workspace reset` which would allow uses to
reset workspace-related state without affecting its contents. This will
be useful in case when an user wants to re-run configure-commands for a
workspaced element.
Patch originally by Chandan Singh, rebased against recent refactor.
Fixes #375.
|
| |
|
|
|
|
| |
with cache.
Fixes #377.
|
| |
|
|
|
| |
We need the logging handler setup before creating the Stream(),
as the Stream() will expect logging to be already functional.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It makes not sense to type `bst fetch --track --deps all <targets>`,
because tracking will inevitably modify the build plan.
Stream initialization will not cope with this either, instead of
silently doing something which does not make any sense, we add an
assertion that this should not happen.
Unfortunately since `plan` is the default deps type for `bst fetch`,
this is likely to happen so it's important to correct.
This patch adds a warning in the case tracking of the build plan
elements is requested, and converts the request to track all elements
instead.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Here the pipeline becomes essentially stateless, some dangling
state remains to be factored out because of frontend accesses
which will be changed in a later commit.
Essentially, the Pipeline.load() method no longer has any knowledge
of the specific purposes of the loaded targets, and now takes
a list of target groups and returns a corresponding list of element
groups.
The Stream() business logic methods now use other pipeline helper
methods to create and filter lists from the loaded target elements.
The Stream() also finally absorbs the Scheduler frontend facing
APIs. However Queues are still exposed on the Stream object for
logging purposes and through callbacks such that the frontend can
retry elements.
|
| |
|
|
|
|
|
| |
This removes some additional initialization code from Pipeline().
Some symbols have changed here, the initialization is now called
from Stream(), and a test case was also adjusted for this.
|
| |
|
|
| |
There is no more need for this distinction.
|
| |
|
|
|
|
|
|
|
| |
This was previously decided in CLI, but knowledge of what to initialize
has been moved to Stream().
Now there is no more point to store this configuration in the Context,
we just have the Stream() decide it when asking the Pipeline() to
invoke the Loader().
|
| |
|
|
|
|
| |
This shifts the whole responsibility of interpreting command line
targets etc to the Stream() object itself. With this commit, the
Pipeline() truly becomes slaved to the Stream().
|
| |
|
|
|
|
|
|
| |
o This makes logging independent from the Pipeline()
o Removed size_request Widget() method, add context to Widget() initializer
o Make the Status() widget derive anything it needs through the Context()
|
| | |
|
| |
|
|
| |
Use Stream error for Stream errors.
|
| |
|
|
|
|
| |
This is the first part of the pipeline refactor, at this stage
all calling interfaces remain the same, except that invocation
of the scheduler has been moved from Pipline to Stream.
|
| |
|
|
| |
See: #373
|
| |
|
|
|
|
|
| |
Lazily parse the version of bwrap the first time the function is called.
On subsequent calls, used cached version info.
See: #373
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ensure that the strong cache key of each build dependency is available
before an element is built. Otherwise the strong cache key of the
element cannot be calculated and caching the artifact produces an
AssertionError.
In non-strict mode an element's strong cache key may not be available
yet even though an artifact is available in the local cache. This can
happen if the pull job is still pending as the remote cache may have an
artifact that matches the strict cache key, which is preferred over a
locally cached artifact with a weak cache key match.
Fixes #383.
|
| |
|
|
|
|
|
| |
changed encode_message push exception from
'Command must by GLib.Variant'
to
'Command must be PushCommand'
|
| |
|
|
| |
This matches the pull code path.
|
| | |
|
| |
|
|
| |
Fixes #325.
|
| |
|
|
|
|
|
|
| |
If we want to depend on being able to revision junction.refs and
project.refs separately, then we are better off just considering the
project.refs feature as available since the new version 8.
This will not harm projects which depended on it since version 5 instead.
|
| |
|
|
|
|
| |
Leave this error to be handled by preflight.
Updated test case to expect the new ElementError instead of a LoadError
|
| |
|
|
|
| |
Move this logic into the junction element itself, instead
of special case erroring for this in the loader.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Using setuptools_scm had a couple of bad problems:
o Unexpected versioning semantics, setuptools_scm would
increment the micro version by itself in the case that
we derive a version number from something that is not a tag,
making the assumption that we are "leading up to" the next
micro version.
People mostly dont expect this.
o When installing in developer mode, i.e. with `pip3 install --user -e .`,
then we were always picking the generated version at install time
and never again dynamically resolving it.
Many of our users install this way and update through git, so it's
important that we report more precise versions all the time.
This commit needs to make a series of changes at the same time:
o Adds versioneer.py to the toplevel, this is used by setup.py
for various activities.
This is modified only to inform the linter to skip
o Adds buildstream/_version.py, which is generated by versioneer
and gives us the machinery to automatically derive the correct version
This is modified only to inform the linter to skip
o Adds a .gitattributes file which informs git to substitute
the buildstream/_version.py file, this is just to ensure that
the versioning output would work if ever we used `git archive`
to release a tarball.
o Modifies setup.py and setup.cfg for versioneer
o Modifies utils.py and _frontend/cli.py such as to avoid importing
the derived version when running bash completion mode, we dont
derive the version at completion time because this can result
in running a subprocess (when running in developer install mode)
and is an undesirable overhead.
o Updates tests/frontend/version.py to expect changed version output
|
| |
|
|
|
|
|
|
|
|
|
|
| |
o _projectrefs.py: Additional constructor option to choose the base name
o _project.py: Load two ProjectRefs objects, one for the junctions
o source.py: Load and save junctioned source refs with the appropriate ProjectRefs object
o tests: Updated some tests to expect junctions to be stored in junction.refs
This fixes issue #361
|
| |
|
|
|
|
| |
This is needed so that Sources can derive whether they belong
to a junction or not, which is needed for separating where
junction refs are stored.
|
| |
|
|
| |
A late fix for issue #285
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The logic for determining which files were removed by integration
commands was broken when dealing with files staged within symlink
directories.
This rather weird scenario is only possible because of the way
BuildStream layers artifacts. If artifact 1 contains a symlink from
`/sbin` to `/usr/sbin`, and artifact 2 is staged on top and contains
a file `/sbin/init`, then the resulting filesystem contains a file at
`/usr/sbin/init`.
The manifest used by the compose element is generated from the contents
of the individual artifacts, so it lists the original paths such as
`/sbin/init`, but would would not contain `/usr/sbin/init` as nothing
has processed the symlinks.
The path `/sbin/init` is valid inside the composed tree, but filesystem
traversals that don't follow symlinks will not report that path in their
results. The compose plugin would look for `/sbin/init` in the results
of `utils.list_relative_paths()`, find it missing, and would act as if
some integration command had removed the file. This meant it would not
end up in the results.
To fix this, I have inverted the logic that processes the results of the
integration commands. We now work through every path in the manifest
and check it against the results of the integration commands, rather
than the other way around, and if any path from the manifest doesn't
appear in the snapshot we assume that it has staged in a different
location due to symlinks.
See: https://gitlab.com/BuildStream/buildstream/issues/270
|
| |
|
|
|
|
| |
This is an optimisation for the case where the compose element doesn't
do any splitting, and also brings 2 related code fragments closer to
each other.
|
| |
|
|
|
| |
There is no significant order for the lists of added, removed and
modified files, so use an unordered set() to store the data.
|
| |
|
|
|
|
| |
_artifactcache/pushreceive.py: Wrap OSTreeReceiver.do_run in a
profiling domain.
_profile.py: Add 'ARTIFACT_RECEIVE' domain.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This uses the existing messaging system to report errors in the main
application as a BUG type. It requires the use of a global_app variable
which isn't ideal; this may be replaced in future. This partially addresses
issue #197.
Theoretically, an exception could occur before Scheduler.loop is set up,
hence the check for it when terminating all jobs.
NOTE: This was originally submitted by Jim MacArthur, and manually
reapplied after some refactoring took place.
|
| | |
|
| |
|
|
|
| |
This is in a derived Process from the standard lib, we add this
here because we are unsure about the constructor signature.
|
| |
|
|
| |
in constructor
|
| | |
|
| | |
|
| |
|
|
|
| |
Plugins set their attributes in configure(), because the
constructor is not public API.
|
| | |
|
| | |
|
