| Commit message (Collapse) | Author | Age | Files | Lines |
|\
| |
| |
| |
| | |
All: Update all references to the mailing list.
See merge request BuildStream/buildstream!1980
|
|/
|
|
|
|
| |
Update any references to the mailing list to now point to:
https://lists.apache.org/list.html?dev@buildstream.apache.org
|
|\
| |
| |
| |
| | |
_artifactcache.py: Skip push only if server has identical artifact
See merge request BuildStream/buildstream!1976
|
| |
| |
| |
| |
| | |
This is a regression test to skip push only if server has identical
artifact.
|
|/
|
|
|
|
|
|
|
|
|
|
| |
The existing artifact on a server may refer to a previous build and the
referenced blobs may no longer exist. As we push blobs referenced by the
local artifact proto, we can skip pushing the artifact proto only if the
artifact proto on the server is identical.
It would be more efficient to always push the artifact proto without
checking what artifact proto is currently on the server. However, we
need the extra information to mark push jobs as skipped. This is clearer
to users and also required by some test cases.
|
|\
| |
| |
| |
| | |
bst-artifact-server: Fix error handling for requests forwarded to casd
See merge request BuildStream/buildstream!1972
|
| | |
|
|/ |
|
|\
| |
| |
| |
| |
| |
| | |
Explicit control of junction configuration overrides
Closes #1326
See merge request BuildStream/buildstream!1901
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit:
* Redocuments the `junction.py` element almost in it's entirity
* Describes the `overrides` feature
* Describes how to use `link` elements to derive junction configuration
from subprojects.
* Adds project.conf documentation for junction `duplicates`
* Adds project.conf documentation for junction `internal`
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit adds support for the `internal` list of the project's
`junctions` configuration section.
_loader/loadcontext.py is updated with this commit to consider
the internal markers and avoid raising errors for collisions with
internal junctions, as well as being enhanced to list internal
junctions in any conflicting junction error messages.
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This adds some measure of tolerance to duplicate projects by
accepting cases where multiple projects are loaded but whitelisted
to be loaded as duplicates by the project.conf.
This change does the following:
* _project.py: Add `junction_is_duplicated()` API and parse `duplicates`
The project now maintains a table of junctions which are considered
"duplicated" per project, and exposes an API for the loader to query
whether a given loader is explicitly listed as a duplicate (allowing
multiple "duplicated" instances of that project to coexist in the same
pipeline).
This new API searches for the passed Loader instance by way of it's
project relative search paths specified in the duplicates. Using the
`Loader.get_loader()` codepath ensures that any links get resolved
and that any overrides along the way will be considered correctly
while attempting to match the duplicate.
* _loader/loader.py: Few changes here.
- Allow Loader.get_loader() to operate in a mode which does not
load subprojects.
- Add new function Loader.ancestors(), this is a generator which yields
every ancestor loader which is capable of addressing the given loader,
including ancestors for which their junction definitions to this loader
have been overridden.
To this end, we now record `_alternative_parents` when searching for
overrides.
- Now implements __str__() in order to print an identifying string
for a loader in error messages
* _loader/loadcontext.py: Enhanced to now ask projects whether they are
marked as duplicates in order to avoid erroring out on project collisions
if those are explicitly marked to not be an error.
We now also split out the new function LoadContext.assert_loaders(), which
is to be called once at the end of loading the whole pipeline recursively.
This reduces the complexity of searching the loaded graph while loading
the graph, and also reports more complete errors (all instances of a
conflicting project will be reported in the error at once).
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* Test various scenarios of overriding junctions, including
deep paths as junctions to override, and as junctions to use
to override.
* Test conflicting junction configurations, ensuring that we
report both provenances of where the junctions were declared.
* Test circular references in element paths while declaring overrides,
for instance when trying to override a subproject using a deeper
definition of the same subproject.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch removes the functionality of coalescing junction
configurations on junction element names, and replaces it with
an explicit format which allows junction declarations to override
junctions in subprojects with junction declarations from the local
project.
Changes:
* plugins/elements/junction.py: Load the overrides dictionary
* exceptions.py: Added new CIRCULAR_REFERENCE to LoadErrorReason.
* _loader/loadcontext.py: Add new register_loader() function which
is called by the Loader to register all loaders with the context,
delegating the task of detecting conflicting junctions to the
load context.
* _loader/loader.py: When loading a junction, check if there is
an override from the parent project and use that instead.
Register with the LoadContext, and pass the LoadContext
and provenance along when creating a Project
Further, we now protect against circular element path references
with a new _loader_search_provenances table.
* _project.py: Pass new `provenance` member through the constructor
and onwards into the child loader.
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit refactors the junctions test to use more parameterization
and to remove copy_subproject(), using statically committed data as
much as possible.
This is because copy_subproject() causes data to be shared among tests
in such a way that when such data get's modified, it easily causes
unintended side effects on adjacent test cases, better to keep this
data separate.
Overview of changes:
* Remove copy_subproject()
* Split up test data into case specific directories, sometimes
reusing a directory among various related tests
* Use @pytest.mark.parameterize() as much as possible for better
coverage and more clearly expressed test cases
* Adds update_project() to modify a project.conf inline, as is
done in some other tests like tests/plugins/loading.py
* Removes tests related to junction name coalescing, this feature
will be removed later in this branch and other tests related
to junction overrides will replace these.
* Removes some redundant tests
* Removes a comment about how junction name coalescing can cause
errors when trying to open a junction but the project.conf is
missing. We continue to test missing project.conf files in a
variety of scenarios, but the comment will be rendered irrelevant
with the removal of junction name coalescing.
* Change the git related tests to use tar instead, this serves
the same purpose, but tar will remain a core plugin in BuildStream 2.
|
|/
|
|
|
|
|
|
|
|
|
|
| |
When a junction element has dependencies, we were not even specifying
which junction element was guilty of having dependencies and only
saying that such a thing would be forbidden (wherever it might be in
your project).
Fix this by using the provenance of the dependency in the junction,
specifying exactly where the invalid dependency exists.
This required carrying the provenance onto the other Dependency() object.
|
|\
| |
| |
| |
| | |
Small cleanups
See merge request BuildStream/buildstream!1966
|
| |
| |
| |
| |
| | |
Otherwise some of BuildStream's config will fail and it is therefore
impossible to just run `pytest tests/`
|
|/ |
|
|\
| |
| |
| |
| | |
_loader: Adding LoadContext
See merge request BuildStream/buildstream!1964
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of passing around many details though calling signatures
throughout the loader code, create a single LoadContext object
which holds any overall loading state along with any values which
are constant to a full load process.
Overall this patch does:
* _frontend/app.py: No need to pass Stream.fetch_subprojects() along anymore
* _loader/loadelement.pyx: collect_element_no_deps() no longer takes a task argument
* _loader/loader.py: Now the Loader has a `load_context` member, and no more
`_fetch_subprojects` member or `_context` members
Further, `rewritable` and `ticker` is no longer passed along through all
of the recursing calling signatures, and `ticker` itself is finally removed
because this has been replaced a long time ago with `Task` API from `State`.
* _pipeline.py: The load() function no longer has a `rewritable` parameter
* _project.py: The Project() is responsible for creating the toplevel
LoadContext() if one doesn't exist yet, and this is passed through
to the Loader() (and also passed to the Project() constructor by the
Loader() when instantiating subprojects).
* _stream.py: The `Stream._fetch_subprojects()` is now private and set
on the project when giving the Project to the Stream in `Stream.set_project()`,
also the Stream() sets the `rewritable` state on the `LoadContext` at the
earliest opportunity, as the `Stream()` is the one who decides this detail.
Further, some double underscore private functions are now regular single
underscores, there was no reason for this inconsistency.
* tests/internals/loader.py: Updated for API change
|
|\
| |
| |
| |
| | |
Completely abolish job pickling.
See merge request BuildStream/buildstream!1965
|
|/ |
|
| |
|
|\
| |
| |
| |
| | |
Remove a not very important message from the cli
See merge request BuildStream/buildstream!1963
|
| |
| |
| |
| |
| |
| | |
Add some comments clearly separating public and private methods,
and moving the _validate_node() private method into the private
section, renaming it _validate_toplevel_node() for better clarity.
|
| |
| |
| |
| | |
This is no longer needed.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This API was only used to issue a not very useful message in the CLI
when removing a workspace the user happened to have invoked BuildStream
from.
The API also didn't really make any sense, being documented as:
"Checks whether the workspace belonging to element_name is
required to load the project"
This in fact meant whether having that workspace open was required
for BuildStream to be invoked, due to the metadata encoded into the
workspace directory.
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When closing a workspace while the `workspace close` command was issued
from the closed workspace, we go to great lengths to inform the user
that they can no longer run bst commands from the closed workspace directory.
This is not worthwhile, the user will know this the next time they run
`bst` anyway.
|
|/
|
|
|
|
| |
Remove tests which check for a user message to be issued upon closing
a workspace who's metadata was used to launch BuildStream and find
the BuildStream project directory.
|
|\
| |
| |
| |
| | |
Reorginize some __init__() functions
See merge request BuildStream/buildstream!1961
|
| |
| |
| |
| |
| |
| |
| |
| | |
Put the private members in the private section and
the public members in public section.
The exception is `_context` which is resolved early, and
this is required by other code which runs in __init__().
|
|/
|
|
| |
Instead of mixing it up with the directories
|
|\
| |
| |
| |
| | |
.gitlab-ci.yml: Update BuildBox components to 0.0.9
See merge request BuildStream/buildstream!1962
|
|/ |
|
|\
| |
| |
| |
| | |
cascache.py: Fix file modes in checkout()
See merge request BuildStream/buildstream!1959
|
| |
| |
| |
| |
| | |
Do not copy file mode from casd object file. Do not change
non-executable mode bits if file is executable.
|
| | |
|
|/
|
|
| |
Catch errors raised by `shutil.copyfile()`.
|
|\
| |
| |
| |
| |
| |
| | |
Improved remote build log message
Closes #1336
See merge request BuildStream/buildstream!1955
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
| |
The log message which shows that the remote build is ongoing has been
extended to include the name of the element to which the remote build
corresponds.
Likewise the message indicating that an action result has been found in
the action cache will also no include the element name.
A new timed_activity() call has been added to show when missing blobs
are being determined and uploaded.
Fixed #1336
|
|\
| |
| |
| |
| | |
Make provenance mandatory in `Loader.get_loader()`
See merge request BuildStream/buildstream!1960
|
| |
| |
| |
| |
| |
| | |
This forces the `provenance` to `Loader.get_loader()` to be
a mandatory argument, ensuring that there are never any
callers which fail to provide provenance.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Instead of raising a customized error message which adds little
value to the provenance, just pass the provenance along.
This is important so that the Loader is aware of the provenance
of loaded junctions, so that it can more precisely report errors
about conflicting junctions when includes cause conflicts.
This commit also adjusts tests/format/includes.py
|
|/
|
|
|
|
|
|
|
|
| |
Change the error message to reflect that the file is a directory,
do not suggest that it should be a .bst file, it should not be
a .bst file.
A .bst file might be expected from the loader code, which can
feel free to express that, but other files can be loaded, for
example include files.
|
|\
| |
| |
| |
| | |
Support full paths
See merge request BuildStream/buildstream!1956
|