summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* tests/integration/interactive_build.py: Relax expect for buildbox-runjuerg/buildbox-run-testJürg Billeter2020-02-031-2/+1
| | | | | When commands are batched, the failure message can't include the command group label with buildbox-run.
* Merge branch 'juerg/buildbox' into 'master'Jürg Billeter2020-02-031-7/+7
|\ | | | | | | | | .gitlab-ci.yml: Update BuildBox components to 0.0.6 See merge request BuildStream/buildstream!1813
| * .gitlab-ci.yml: Update BuildBox components to 0.0.6Jürg Billeter2020-02-031-7/+7
|/
* Merge branch 'jjardon/overnigth_cargo' into 'master'Javier Jardón2020-02-011-1/+1
|\ | | | | | | | | | | | | .gitlab-ci.yml: build fdsdk needs cargo plugin and its dependencies Closes #1259 See merge request BuildStream/buildstream!1812
| * .gitlab-ci.yml: build fdsdk needs cargo plugin and its dependenciesJavier Jardón2020-02-011-1/+1
|/
* Merge branch 'coldtom/fix-overnight-test' into 'master'Javier Jardón2020-02-011-3/+2
|\ | | | | | | | | .gitlab-ci.yml: Bump bst-plugins-experimental See merge request BuildStream/buildstream!1811
| * .gitlab-ci.yml: Bump bst-plugins-experimentalcoldtom/fix-overnight-testThomas Coldrick2020-01-291-3/+2
|/ | | | | | | | | | | When we updated the entrypoints for the plugins, the bst-plugins-experimental version used for overnight tests was not bumped, which means that the plugins used in freedesktop-sdk don't get detected. This commit bumps this to the latest snapshot of bst-plugins-experimental, and downloads directly from PyPI rather than installing from git.
* Merge branch 'tmewett/git-source' into 'master'jjardon/fedora-30-removalBenjamin Schubert2020-01-293-74/+201
|\ | | | | | | | | Improvements to _GitSourceBase and _GitMirror See merge request BuildStream/buildstream!1808
| * Add Git source test for depth=1 fetch with exact tag reftmewett/git-sourceTom Mewett2020-01-241-0/+28
| |
| * Document my changes to the Git source plugin in NEWSTom Mewett2020-01-242-0/+17
| | | | | | | | | | This includes adding an entry for the recursive submodules changes from !1765, and adding myself to the authors of _gitsourcebase.py.
| * _gitsourcebase.py: Refactor latest_commit_with_tagsTom Mewett2020-01-241-19/+49
| | | | | | | | | | The functionality has been split into three new functions: to_commit, reachable_tags, and describe. No behaviour is changed.
| * _gitsourcebase.py: Strip git-describe tag info from cache keyTom Mewett2020-01-241-1/+10
| | | | | | | | | | | | | | | | | | | | | | The beginning parts of git-describe labels are completely arbitrary. They can be changed either manually or by a track (e.g. if a tag is moved, added or deleted) even if the referenced commit is the same. Hence, only the commit ID part of the label should factor into the cache key. This commit, of course, breaks cache keys for artifacts built with the 'git' source with git-describe refs.
| * _gitsourcebase.py: Fetch with depth=1 when an available tag is givenTom Mewett2020-01-241-16/+88
| |
| * _gitsourcebase.py: Refactor fetching and trackingTom Mewett2020-01-241-44/+15
|/ | | | | | | | | | This changes how the Git repo in the source cache is managed. The mirror is not created with an initial clone, but is set up manually from an empty repo. This is to support shallow cloning in future changes. To simplifiy code and Git calls, remotes are no longer used; instead URLs are passed on the command-line.
* Merge branch 'coldtom/testing-api' into 'master'Javier Jardón2020-01-2494-326/+373
|\ | | | | | | | | | | | | Stop plugin tests using private API Closes #1258 See merge request BuildStream/buildstream!1806
| * testing: Add functions to generate yaml filesThomas Coldrick2020-01-2410-136/+116
| | | | | | | | | | | | | | Adds functions to the `buildstream.testing` package to allow plugins to dump elements and projects on the fly. Before this plugins were just accessing the private yaml API for tests and loading/dumping directly. I also allow access to just `_yaml.load()` from testing.
| * exceptions: Expose ErrorDomain, ErrorLoadReasonThomas Coldrick2020-01-2390-190/+257
|/ | | | | | | Plugin tests are already accessing this API, but using imports from private modules. For motivation for this to be exposed publicly, note that ErrorDomain is an argument for most things in runcli.py, and LoadErrorReason may be another.
* Merge branch 'tpollard/shellbuildtree' into 'master'Tom Pollard2020-01-212-30/+89
|\ | | | | | | | | | | | | _frontend/cli.py: Make show() --use-buildtree respect pull semantics Closes #1151 See merge request BuildStream/buildstream!1767
| * _frontend/cli.py: Make show() --use-buildtree respect pull semanticsTom Pollard2020-01-212-30/+89
|/ | | | | | | Ensure that if a buildtree isn't cached locally, it's only fetched if --pull and pull-buildtrees config are set. Also, only attempt to fetch if it's plausible that it could be pulled, with appropriate messaging based on local cached state.
* Merge branch 'juerg/assemble_done' into 'master'Jürg Billeter2020-01-183-9/+24
|\ | | | | | | | | element.py: Optimize assemble_done() See merge request BuildStream/buildstream!1802
| * element.py: Optimize assemble_done()Jürg Billeter2020-01-183-9/+24
|/ | | | | After a successful build we know that the artifact is cached. Avoid querying buildbox-casd and the filesystem.
* Merge branch 'chandan/1.93.0' into 'master'1.93.0Chandan Singh2020-01-171-0/+5
|\ | | | | | | | | NEWS: Add mark for 1.93.0 See merge request BuildStream/buildstream!1804
| * NEWS: Add mark for 1.93.0chandan/1.93.0Chandan Singh2020-01-171-0/+5
|/ | | | Prepare for next release.
* Merge branch 'chandan/plugin-namespace-fix' into 'master'Chandan Singh2020-01-176-8/+19
|\ | | | | | | | | Plugins: Update entrypoint group for pip origin See merge request BuildStream/buildstream!1803
| * doc/sample_plugin: Update name of entry point groupChandan Singh2020-01-171-4/+1
| | | | | | | | | | | | | | | | Update sample plugin documentation based on the recent change to entry point group for plugins. While I'm here, also remove the unnecessary dependency on `setuptools` from the sample plugin.
| * NEWS: Announce new entry point groups for pluginsChandan Singh2020-01-171-0/+4
| |
| * Plugins: Update entrypoint group for pip originChandan Singh2020-01-174-4/+14
|/ | | | | | | | | | | | | | | As discussed in https://mail.gnome.org/archives/buildstream-list/2020-January/msg00001.html, separate the entry point groups for elements and sources. Previously, both used `buildstream.plugins` which meant that it wasn't possible to have an element and a source plugin with the same name, when using `pip` origin. This a breaking change for all plugins, but will only affect projects using the `pip` plugin origin. As part of this, also bump the version of bst-plugins-experimental used in our tests to avoid CI breakages.
* Merge branch 'jjardon/move_deb_source' into 'master'Javier Jardón2020-01-1733-304/+9
|\ | | | | | | | | Remove deb surce plugin, it has beem moved to bst-plugins-experimental See merge request BuildStream/buildstream!1729
| * Remove "deb" surce plugin, it has beem moved to bst-plugins-experimentaljjardon/move_deb_sourceJavier Jardón2020-01-1733-304/+9
|/
* Merge branch 'bschubert/optimize-consistency' into 'master'Benjamin Schubert2020-01-1635-285/+288
|\ | | | | | | | | | | | | Optimize consistency and state handling Closes #1126 See merge request BuildStream/buildstream!1739
| * NEWS: Add news information about Consistency breakagebschubert/optimize-consistencyBenjamin Schubert2020-01-161-0/+21
| |
| * element.py: Stop asserting we are cached after fetch and mark us as suchBenjamin Schubert2020-01-161-9/+4
| | | | | | | | | | | | This assert would only be run in development mode and would actually not mark the source as downloaded. Therefore forcefully mark them as fetched and remove the assert. Our test suite will catch if there is a problem.
| * types.py: Completely remove 'Consistency'Benjamin Schubert2020-01-162-43/+13
| | | | | | | | | | 'Consistency' is not needed anymore, now that we have everything in place to not use it
| * element.py: Optimize _should_fetch conditionBenjamin Schubert2020-01-161-6/+3
| | | | | | | | | | By looking at the flag first, we can avoid expensive checks on whether the element is cached or not.
| * element.py: Compute whether an element is cached only on `is_cached`Benjamin Schubert2020-01-163-23/+20
| | | | | | | | | | | | | | | | | | | | | | This removes the early call to get whether sources are locally cached in `_update_source_state` by delegating it to the call of `is_cached`. Once it is cached, the element is assumed to stay that way for the duration of the pipeline, we can therefore cache the result once it is true. Also remove `Consistency.IS_CACHED`, which is not used anywhere else.
| * source.py: Remove 'get_consistency' completelyBenjamin Schubert2020-01-1622-163/+32
| | | | | | | | | | This is not needed now that we have 'is_resolved' and 'is_cached'. We can therefore drop all calling places and implementations of it.
| * source.py: Remove the reliance on consistency to get whether a source is cachedBenjamin Schubert2020-01-1618-12/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes the need to use consistency in Sources, by asking explicitely whether the source is cached or not. This introduces a new public method on source: `is_cached` that needs implementation and that should return whether the source has a local copy or not. - On fetch, also reset whether the source was cached or set if as cached when we know it was. - Validate the cache's source after fetching it This doesn't need to be run in the scheduler's process and can be offloaded to the child, which will allow better multiprocessing
| * source.py: Add a new 'is_resolved' to get whether a source is resolved.Benjamin Schubert2020-01-1612-16/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `get_consistency` is coarse grained and hard to optimize, in addition to being un-userfriendly. This adds a new `is_resolved` that has for default implementation `get_ref() is not None`, which is true for most sources in BuildStream. Sources for which this is not true can override the method to give a more accurate description. Checking for this before looking whether the source is cached can reduce the amount of work necessary in some pipeline and opens the door for more optimizations and the removal of the source state check.
| * element.py: Remove _get_consistency and introduce explicit methodsBenjamin Schubert2020-01-165-30/+33
| | | | | | | | | | | | | | This replaces the _get_consistency method by two methods: `_has_all_sources_resolved` and `_has_all_sources_cached` which allows a more fine grained control on what information is needed.
| * element.py: Rename '_source_cached' to '_has_all_sources_in_source_cache'Benjamin Schubert2020-01-169-27/+27
| | | | | | | | | | | | '_source_cached' is not explicit enough as it doesn't distinguishes between sources in their respective caches and sources in the global sourcecache.
| * source.py: Introduce methods to query state instead of get_consistencyBenjamin Schubert2020-01-163-9/+19
| | | | | | | | | | | | | | | | `get_consistency` doesn't allow being fine grained and asking only for a specific bit of information. This introduces methods `is_cached` and `is_resolved` which will be more flexible for refactoring.
| * tox.ini: Bump version of bst-plugins-experimental to 0.14.0Benjamin Schubert2020-01-161-2/+2
|/
* Merge branch 'chandan/remove-bst-experimental-unused' into 'master'Chandan Singh2020-01-161-6/+0
|\ | | | | | | | | tests/integration/project.conf: Remove unused external plugins See merge request BuildStream/buildstream!1799
| * tests/integration/project.conf: Remove unused external pluginschandan/remove-bst-experimental-unusedChandan Singh2020-01-151-6/+0
|/ | | | | There are no tests that rely on these plugins anymore, so remove the corresponding declarations from project.conf.
* Merge branch 'tpollard/removeassert' into 'master'Tom Pollard2020-01-151-10/+0
|\ | | | | | | | | | | | | element.py: Remove unused __staged_sources_directory & assert Closes #1249 See merge request BuildStream/buildstream!1798
| * element.py: Remove unused __staged_sources_directory & asserttpollard/removeassertTom Pollard2020-01-151-10/+0
|/
* Merge branch 'tpollard/messagerate' into 'master'Tom Pollard2020-01-154-13/+62
|\ | | | | | | | | Make message & status rendering be tick driven by default See merge request BuildStream/buildstream!1745
| * userconfig.yaml: Add bool for disabling console output throttlingtpollard/messagerateTom Pollard2020-01-154-3/+33
| | | | | | | | Also highlight the change & configuration in NEWS
| * _frontend/app.py: Make message printing tick drivenTom Pollard2020-01-151-12/+31
|/
* Merge branch 'chandan/gitlab-ci-toxenvs' into 'master'Chandan Singh2020-01-141-5/+2
|\ | | | | | | | | .gitlab-ci.yml: Simplify test environment declaration See merge request BuildStream/buildstream!1795