| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
_classify_artifacts() no longer filters glob expressions by first
obtaining a list of ALL refs locally cached. We now only obtain refs
specified by the glob expression.
Furthermore, the Project.element_path is used to start searching for
globbed elements as opposed to the Project.directory.
|
| |
|
|
|
| |
This commit ensures that CASCache.list_refs(), and
ArtifactCache.list_artifacts(), can both handle glob expressions.
|
| |
|
|
| |
This method has been moved to Stream, where is it used there exclusively.
|
| |
|
|
|
| |
The loading of elements and the handling of artifacts does not belong
in this module. Such logic should be invoked using the Stream API
|
| |
|
|
|
|
|
| |
A CasBasedDirectory object of an artifacts logs can be obtained
with ArtifactCache.get_artifacts_log(). This ultimately calls
CASCache.get_top_level_dir() to obtain a CasBasedDirectory
of an artifact's subdirectory (or subdirectories).
|
| | |
|
| |
|
|
|
|
| |
* There is no need for this method to use a cas object.
* Search for artifact globs in the project's element path
* An artifact key is always 64 chars long
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This object should be used when we want to handle artifact refs directly
from the command line.
An ArtifactElementError has also been added to _exceptions.py
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
This commit removes the method ArtifactCache.get_artifact_fullname()
and replaces it with Element.get_artifact_name()
Given a key, we are now able to construct the full name of any of an
element's artifacts.
|
| |\
| |
| |
| |
| | |
buildstream/_cas/cascache.py: Set 0644 rights to pulled files
See merge request BuildStream/buildstream!1144
|
| |/
|
|
| |
This was broken by 5ef19a0b31df84caed1e41719ef7ea5c6bd8a8bc.
|
| |\
| |
| |
| |
| |
| |
| | |
Stage junctions into .bst instead of a tmpdir
Closes #895
See merge request BuildStream/buildstream!1134
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Since junction check-outs are now persistent, there is no need to clean
up temporary directories once finished.
Now, junction checkouts are stored within the top-level project dir and
will be cleaned-up when that directory is removed.
This involves changes in:
* loader.py: Remove cleanup logic and passing-around of tmpdir.
* _project.py: Remove passing around tmpdir and calling loader's
cleanup.
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| | |
Further optimisations
See merge request BuildStream/buildstream!1131
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
While get_strict() doesn't look expensive per-se, it is called so many
times that it is valuable to cache the result once computed. Since I
don't think it can change once it is computable, cache it immediately
that becomes possible and we save 20s in my test case.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
By re-using the isinstance replacements from earlier commits and
using a tuple of the string constants for checking for composition
markers, we reduce the cost of node_final_assertions by two thirds
in basic testing.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| | |
A non-trivial proportion of time pre-scheduler in `bst build` is spent
copying (or chain-copying) nodes. Approximately a quarter of the time
spent in that effort is in `isinstance()`. This removes that CPU load.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
We know that nodes are typically one of:
str, list, dict, bool, tuple, NoneType or our ChainMap
Of these, dict and ChainMap are Mapping, only list is list
and the rest are returned unchanged. We can reduce/defer our use
of isinstance here, dramatically, improving performance.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |/
|
|
|
|
|
|
|
|
| |
We were indiscriminately retrieving the node's provenance data in the
`node_get()` function which was accounting for approximately a third of
the total runtime of `node_get()` which dominates pre-scheduler time in
`bst build`. This change ameliorates that situation by only retrieving
the provenance data when it's actually needed.
Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Test with current fedora release: 29
See merge request BuildStream/buildstream!1137
|
| | |
| |
| |
| | |
Fedora 29 includes python 3.7 so this is not needed anymore
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| | |
_frontend: Allow printing dependencies using `bst show`
Closes #890
See merge request BuildStream/buildstream!1121
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At present, there isn't an easy way to print anything from `bst show`
that would give the users an idea of what the dependency graph looks
like. One could use things like `--deps build`, but that will just print
a list, without any information about the dependency edges.
Add `%{deps}`, `%{build-deps}` and `%{runtime-deps}` format strings to
`bst show` that would simply print the list of all dependencies, build
dependencies and runtime dependencies respectively.
Summary of changes:
* buildstream/_frontend/cli.py: Add help for new format symbols.
* buildstream/_frontend/widget.py: Add support for new format symbols
for dependencies.
* tests/frontend/show.py: Add tests for new format symbols.
|
| |\
| |
| |
| |
| | |
Increase read buffer size to improve performance
See merge request BuildStream/buildstream!1142
|
| | |
| |
| |
| |
| | |
Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
factor 4, according to a very simple benchmark.
|
| |/
|
|
|
| |
Increasing buffer size from 4 kB to 64 kB speeds up read() bandwidth by
factor 4, according to a very simple benchmark.
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: allow "test-wsl" to fail until runner problems are sorted out
See merge request BuildStream/buildstream!1145
|
| |/
|
|
| |
are sorted out
|
| |\
| |
| |
| |
| | |
More user-friendly reporting on include errors
See merge request BuildStream/buildstream!891
|
| | |
| |
| |
| |
| | |
Don't create and remove temp dirs unnecessarily when they are not used,
looks like these were just copy-pastes without intended side-effects.
|
| | |
| |
| |
| | |
Avoid an unnecessary call to os.path.join().
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, include a directory result in an error like this:
mydir is a directory. bst command expects a .bst file.
Note that the file containing the include was not mentioned.
Now we get an error like this instead:
element.bst [line 12 column 0]: Include block references a
directory instead of a file: 'mydir'.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, a missing include would result in an error like this:
Could not find file at not-a-file.include
Note that the file containing the include was not mentioned.
Now we get an error like this instead:
element.bst [line 7 column 5]: Include block references a file that
could not be found: 'not-a-file.include'.
|
| |/
|
|
| |
Use the provenance of the include block, instead of the whole node.
|
| |\
| |
| |
| |
| | |
contributing: snakeviz replaces pyflame+flamegraph
See merge request BuildStream/buildstream!1129
|
| |/
|
|
|
|
|
|
|
|
|
|
|
| |
Replace the instructions for pyflame+flamegraph with simpler ones for
snakeviz. For our general use-case this seems to be easier and better.
Usage of this tool was demonstrated at the 2019 BuildStream Gathering in
January by Daniel Silverstone, when presenting the aggregate results of
profiling on many target environments.
Here is the relevant mailing list thread:
"Profiling before the gathering"
https://mail.gnome.org/archives/buildstream-list/2019-January/msg00057.html
|
| |\
| |
| |
| |
| | |
Symlink fixes
See merge request BuildStream/buildstream!1138
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
This matches the change in utils.list_relative_paths() that now sorts
all symlinks as files, instead of following the broken behavior of
os.walk().
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
os.walk() resolves symlinks to check whether they point to a directory
even when followlinks is set to False. We already work around that
broken behavior by extracting symlinks from `dirnames`. However, the
sort order was still incorrect as we returned symlinks in dirnames
before files and other symlinks. This change fixes this, sorting all
files and symlinks in a single list.
|
