| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
| |
With sometimes very slow runners, this test has been found to timeout
more often after the recent refactoring.
Double the timeout to avoid erronously failing CI.
|
| |
|
|
|
|
| |
builds
The old one tested that retrying the failed build doesn't actually retry
|
| |
|
|
|
|
| |
Added tests to ensure that conditional statements don't get overwritten
when performing composition of one dictionary on top of another due to
include processing.
|
| |
|
|
|
|
| |
Check that lazy variable resolution allows using variables
in junction definitions which would not successfully resolve if
we needed to resolve synchronously.
|
| |
|
|
|
| |
Ensure that we get the expected provenance when expanding a variable
included in an overlap whitelist entry.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Test scenarios where a junction needs to resolve variables
in order to configure a subproject, but where some other variables
may be derived from the same subproject.
In this scenario we allow partial resolution of variables
for junction elements.
* Enhanced the undefined variables and circular reference tests
to also check for the expected provenances.
* Test for deep variable resolution
Test variable indirection with 50, 500 and 5000 variables:
* 50 - tests generally large indirections in the recursive algorithm,
which is limited to 200 recursions
* 500 - tests that the non-recursive algorithm works for successful
outcomes and not only for error resolution
* 5000 - tests that the iterative algorithm works and ensures it
is not discarded, as a recursive algorithm cannot be implemented
to support this depth with python (which limits itself to
merely 1000 stack frames).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Main enhancements here include:
* Support for deeply nested variable declarations, removing the
limitations of the recursive variable resolution algorithm.
We were unable to achieve equal performance with the iterative
resolution algorithm, so we now have the recursive approach as
the fast path and only support 200 recursions with this approach
before falling back on the iterative code path, which will support
deep variable resolution and take care of error reporting.
* Better error reporting for undefined variables.
Variables.subst() now requires a ScalarNode and not a simple `str`,
making it more difficult for the core to substitute an undefined
variable without providing the provenance of where that value
expression was declared.
Code changes:
* _variables.pyx: Complete rewrite
* exceptions.py: Added new LoadErrorReason.CIRCULAR_REFERENCE_VARIABLE
* element.py: Pass ScalarNode to Variable.subst() when substituting overlap
whitelists.
|
| |
|
|
|
| |
This fixes a bug where buildstream would ignore the opened workspace on a
cross-junction element.
|
| |
|
|
| |
boundaries
|
| |
|
|
|
| |
Use `bst source push` in a new test instead of manually deleting parts
of a remote cache.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
This is a regression test to skip push only if server has identical
artifact.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
* Test various scenarios of overriding junctions, including
deep paths as junctions to override, and as junctions to use
to override.
* Test conflicting junction configurations, ensuring that we
report both provenances of where the junctions were declared.
* Test circular references in element paths while declaring overrides,
for instance when trying to override a subproject using a deeper
definition of the same subproject.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit refactors the junctions test to use more parameterization
and to remove copy_subproject(), using statically committed data as
much as possible.
This is because copy_subproject() causes data to be shared among tests
in such a way that when such data get's modified, it easily causes
unintended side effects on adjacent test cases, better to keep this
data separate.
Overview of changes:
* Remove copy_subproject()
* Split up test data into case specific directories, sometimes
reusing a directory among various related tests
* Use @pytest.mark.parameterize() as much as possible for better
coverage and more clearly expressed test cases
* Adds update_project() to modify a project.conf inline, as is
done in some other tests like tests/plugins/loading.py
* Removes tests related to junction name coalescing, this feature
will be removed later in this branch and other tests related
to junction overrides will replace these.
* Removes some redundant tests
* Removes a comment about how junction name coalescing can cause
errors when trying to open a junction but the project.conf is
missing. We continue to test missing project.conf files in a
variety of scenarios, but the comment will be rendered irrelevant
with the removal of junction name coalescing.
* Change the git related tests to use tar instead, this serves
the same purpose, but tar will remain a core plugin in BuildStream 2.
|
| |
|
|
|
| |
Otherwise some of BuildStream's config will fail and it is therefore
impossible to just run `pytest tests/`
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of passing around many details though calling signatures
throughout the loader code, create a single LoadContext object
which holds any overall loading state along with any values which
are constant to a full load process.
Overall this patch does:
* _frontend/app.py: No need to pass Stream.fetch_subprojects() along anymore
* _loader/loadelement.pyx: collect_element_no_deps() no longer takes a task argument
* _loader/loader.py: Now the Loader has a `load_context` member, and no more
`_fetch_subprojects` member or `_context` members
Further, `rewritable` and `ticker` is no longer passed along through all
of the recursing calling signatures, and `ticker` itself is finally removed
because this has been replaced a long time ago with `Task` API from `State`.
* _pipeline.py: The load() function no longer has a `rewritable` parameter
* _project.py: The Project() is responsible for creating the toplevel
LoadContext() if one doesn't exist yet, and this is passed through
to the Loader() (and also passed to the Project() constructor by the
Loader() when instantiating subprojects).
* _stream.py: The `Stream._fetch_subprojects()` is now private and set
on the project when giving the Project to the Stream in `Stream.set_project()`,
also the Stream() sets the `rewritable` state on the `LoadContext` at the
earliest opportunity, as the `Stream()` is the one who decides this detail.
Further, some double underscore private functions are now regular single
underscores, there was no reason for this inconsistency.
* tests/internals/loader.py: Updated for API change
|
| | |
|
| |
|
|
|
|
| |
Remove tests which check for a user message to be issued upon closing
a workspace who's metadata was used to launch BuildStream and find
the BuildStream project directory.
|
| |
|
|
|
| |
Do not copy file mode from casd object file. Do not change
non-executable mode bits if file is executable.
|
| |
|
|
|
|
|
|
|
|
|
| |
Instead of raising a customized error message which adds little
value to the provenance, just pass the provenance along.
This is important so that the Loader is aware of the provenance
of loaded junctions, so that it can more precisely report errors
about conflicting junctions when includes cause conflicts.
This commit also adjusts tests/format/includes.py
|
| | |
|
| |
|
|
|
|
|
|
| |
* Test that we succeed to load links with full path targets
* Test that we get correct provenance information in errors when
linking to non-existing elements in existing subprojects, using
full paths.
|
| |
|
|
|
|
|
|
|
|
| |
Added tests to cover:
* Successfully loading plugins with full paths in junction origins
* Failing to load missing plugins in sub-junctions which are successfully loaded
* Failing to load missing plugins in sub-junctions which are invalid and couldnt load
|
| | |
|
| |
|
|
|
|
|
| |
buildbox-run is the only local sandbox and there are no plans to add
other sandboxing backends in the future. New platforms can be supported
by new buildbox-run implementations without requiring any changes in
BuildStream.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Verifying for variables (all used variables defined and no cycle) is
much faster than flattening them. It also uses much less memory. Only
`bst show -f "%{vars}"` requires to flatten all variables. For other
cases we can improve performance by only checking rather than
flattening.
Also flattening very nested variables could lead to very long lists of
empty strings. The memory usage for flattening could be a lot bigger
than the resulting value. Force flattening and caching every
variable definitions can improve intermediate memory consumption and
only consume memory of the size of the result.
Finally, an optimization allowed inifinite cycles leading to
segmentation fault. This was solved by simplifying the code.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This removes:
* The `target` feature from the junction plugin
* Special case code in the loader for the `target` feature
* The `target` related cases in tests/format/junctions.py
This also adjusts the `target` related documentation in
the `junction` element to suggest using a `link` element for
the purpose of using a subproject junction configuration to
access a common sub-subproject.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
While adding a mock task object to track progress in some loader
tests in commit 17144d84c2b63daf6e3aa9b42c6c773f134e8660, a new
`_NO_PROGRESS` object was added as an explicit marker to denote
that progress information should not be reported.
This complicates the code, None should be a sufficient value
for not reporting progress, while still permitting mock task
objects to capture progress if desired.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fixes a bug introduced by d7d18c1a2e454c507afd9e1d3f1358639dd43871,
where public data integration commands and others would never get their
data expanded and would thus fail to run.
The previous however revelead a previous bug, where variables values in
public data of elements would not be part of the cache key of an element
or it's reverse dependencies, and thus, on variable change, rebuilds
would not happen when they would have been needed.
This ensures that all the public data is always resolved and part of the
element's cache key. This however will bring a rebuild of an element
whenever its integration commands variables change, which is simpler to
handle than to try to push that responsibility on reverse dependencies,
since public data can contain plugin-defined values.
|
| | |
|
| | |
|
| |
|
|
| |
Make sure build-only dependencies are pulled only when necessary.
|
| | |
|
| |
|
|
|
| |
This ensures that when resolving variables, we do not overwrite the
values for a different element in the case we were using composition.
|
| |
|
|
|
|
|
|
| |
Use non-greedy search to ensure that we stop at the next closing brace.
Otherwise, for a string like `%{variable} {variable}`, the second
variable will also get substituted.
Fixes #1307.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
When we are not recursing, `Element.dependencies()` uses a much more
light weight codepath since it just needs to print the direct
dependencies. However, this simple codepath was not accounting for
duplicates, in case something is both a build time and run time
dependency.
One way this manifested itself was in `bst show --format %{deps}`, but
it would also affect anything that was using this method to iterate on
the dependencies.
Fixes #1308.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This can be quite handy when one is preparing to open a build shell.
This originally came up in
https://gitlab.com/BuildStream/buildstream/-/issues/685#note_105460896.
|
| | |
|
