summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* types.py: Completely remove 'Consistency'bschubert/optimize-consistency-no-fetch-optimizedBenjamin Schubert2019-12-092-39/+9
| | | | | 'Consistency' is not needed anymore, now that we have everything in place to not use it
* element.py: Optimize _should_fetch conditionBenjamin Schubert2019-12-093-6/+27
| | | | | By looking at the flag first, we can avoid expensive checks on whether the element is cached or not.
* element.py: Compute whether an element is cached only on `is_cached`Benjamin Schubert2019-12-093-49/+19
| | | | | | | | | | | This removes the early call to get whether sources are locally cached in `_update_source_state` by delegating it to the call of `is_cached`. Once it is cached, the element is assumed to stay that way for the duration of the pipeline, we can therefore cache the result once it is true. Also remove `Consistency.IS_CACHED`, which is not used anywhere else.
* source.py: Remove 'get_consistency' completelyBenjamin Schubert2019-12-0922-164/+47
| | | | | This is not needed now that we have 'is_resolved' and 'is_cached'. We can therefore drop all calling places and implementations of it.
* source.py: Remove the reliance on consistency to get whether a source is cachedBenjamin Schubert2019-12-0914-2/+59
| | | | | | | | | This removes the need to use consistency in Sources, by asking explicitely whether the source is cached or not. This introduces a new public method on source: `is_cached` that needs implementation and that should return whether the source has a local copy or not.
* source.py: Add a new 'is_resolved' to get whether a source is resolved.Benjamin Schubert2019-12-0912-15/+52
| | | | | | | | | | | | | | | `get_consistency` is coarse grained and hard to optimize, in addition to being un-userfriendly. This adds a new `is_resolved` that has for default implementation `get_ref() is not None`, which is true for most sources in BuildStream. Sources for which this is not true can override the method to give a more accurate description. Checking for this before looking whether the source is cached can reduce the amount of work necessary in some pipeline and opens the door for more optimizations and the removal of the source state check.
* element.py: Remove _get_consistency and introduce explicit methodsBenjamin Schubert2019-12-095-30/+33
| | | | | | | This replaces the _get_consistency method by two methods: `_has_all_sources_resolved` and `_has_all_sources_cached` which allows a more fine grained control on what information is needed.
* element.py: Rename '_source_cached' to '_has_all_sources_in_source_cache'Benjamin Schubert2019-12-099-27/+27
| | | | | | '_source_cached' is not explicit enough as it doesn't distinguishes between sources in their respective caches and sources in the global sourcecache.
* source.py: Introduce methods to query state instead of get_consistencyBenjamin Schubert2019-12-093-9/+19
| | | | | | | | `get_consistency` doesn't allow being fine grained and asking only for a specific bit of information. This introduces methods `is_cached` and `is_resolved` which will be more flexible for refactoring.
* Merge branch ↵Tristan Maat2019-12-091-4/+4
|\ | | | | | | | | | | | | | | | | '1211-stop-using-non-posix-features-in-small-test-suite-tests-to-avoid-fake-test-failures' into 'master' Resolve "Stop using non-POSIX features in small test suite tests to avoid fake test failures" Closes #1211 See merge request BuildStream/buildstream!1725
| * tests/internals/cascache.py: Stop using non-posix shell featuresTristan Maat2019-12-091-4/+4
|/
* Merge branch 'tlater/update-workspace-manpages' into 'master'Tristan Maat2019-12-091-2/+2
|\ | | | | | | | | man/bst-workspace-reset.1: Update workspace re-open help message See merge request BuildStream/buildstream!1723
| * man/bst-workspace-reset.1: Update workspace re-open help messageTristan Maat2019-12-091-2/+2
|/ | | | This was missed in !1710.
* Merge branch 'traveltissues/remove-unused-functions' into 'master'Darius Makovsky2019-12-0912-296/+2
|\ | | | | | | | | remove unused functions 1/2 See merge request BuildStream/buildstream!1753
| * _remote: ignore unused argsDarius Makovsky2019-12-091-1/+1
| |
| * _profile: ignore unused argsDarius Makovsky2019-12-091-1/+1
| |
| * resources: remove [un]register_exclusive_interest()Darius Makovsky2019-12-091-50/+0
| |
| * _project: remove create_artifact_element()Darius Makovsky2019-12-091-13/+0
| |
| * _pipeline: remove subtract_elements()Darius Makovsky2019-12-091-15/+0
| |
| * _pipeline: remove targets_include()Darius Makovsky2019-12-091-17/+0
| |
| * _context: remove set_artifact_directories_optional()Darius Makovsky2019-12-091-10/+0
| |
| * casserver: remove _digest_from_*_resource_name()Darius Makovsky2019-12-091-45/+0
| |
| * casserver: remove ArtifactStatus()Darius Makovsky2019-12-091-4/+0
| |
| * cascache: remove update_tree_mtime()Darius Makovsky2019-12-091-4/+0
| |
| * _artifactcache: remove _reachable_digests()Darius Makovsky2019-12-091-18/+0
| |
| * _artifactcache: remove _reachable_directories()Darius Makovsky2019-12-091-18/+0
| |
| * _artifactcache: remove get_artifact_logs()Darius Makovsky2019-12-091-16/+0
| |
| * _basecache: remove has_open_grpc_channels()Darius Makovsky2019-12-091-12/+0
| |
| * _workspaces: remove get_key()Darius Makovsky2019-12-091-38/+0
| |
| * _workspaces: remove invalidate_key()Darius Makovsky2019-12-091-8/+0
| |
| * element: remove _is_required()/__requiredDarius Makovsky2019-12-091-8/+0
| |
| * element: Remove unused method (__reset_cache_data)Darius Makovsky2019-12-091-18/+0
|/
* Merge branch 'bschubert/stricter-asyncio-handling' into 'master'Benjamin Schubert2019-12-072-27/+15
|\ | | | | | | | | Fixes for asyncio loops were we don't follow the documentation See merge request BuildStream/buildstream!1756
| * scheduler.py: Only run thread-safe code in callbacks from watchersbschubert/stricter-asyncio-handlingBenjamin Schubert2019-12-072-2/+12
| | | | | | | | | | | | | | | | Per https://docs.python.org/3/library/asyncio-policy.html#asyncio.AbstractChildWatcher.add_child_handler, the callback from a child handler must be thread safe. Not all our callbacks were. This changes all our callbacks to schedule a call for the next loop iteration instead of executing it directly.
| * job.py: Only start new jobs in a `with watcher:` blockBenjamin Schubert2019-12-071-26/+5
| | | | | | | | | | | | | | | | The documentation (https://docs.python.org/3/library/asyncio-policy.html#asyncio.AbstractChildWatcher) is apparently missing this part, but the code mentions that new processes should only ever be called inside a with block: https://github.com/python/cpython/blob/99eb70a9eb9493602ff6ad8bb92df4318cf05a3e/Lib/asyncio/unix_events.py#L808
| * job.py: Remove '_watcher' attribute, it is not neededBenjamin Schubert2019-12-071-3/+2
|/ | | | We don't need to keep a reference to the watcher, let's remove it.
* Merge branch 'bschubert/optimize-scheduling' into 'master'Benjamin Schubert2019-12-071-9/+18
|\ | | | | | | | | scheduler.py: Optimize scheduling by not calling it unnecessarily See merge request BuildStream/buildstream!1755
| * scheduler.py: Optimize scheduling by not calling it unnecessarilyBenjamin Schubert2019-12-071-9/+18
|/ | | | | | | | | | | | This delays the call to the re-scheduling of jobs until the current event loop as terminated. This is in order to reduce the number of time we call this method per loop, which should reduce the pressure on the loop and allow faster event handling Since the call is now delayed, also ensure we only call it once per loop iteration.
* Merge branch 'chandan/fix-manifest' into 'master'Chandan Singh2019-12-051-10/+6
|\ | | | | | | | | Minor packaging-related fixups See merge request BuildStream/buildstream!1754
| * MANIFEST.in: Minor fixups and improvementsChandan Singh2019-12-051-10/+6
|/ | | | | | | | | | | | | | | This patch contains the following changes: * Remove include directive for MAINTAINERS file, that does not exist anymore. Instead, add the COMMITTERS.rst file, which is the logical replacement for the MAINTAINERS file. * Recursively include all requirements files instead of listing them one by one. This also fixes and issue where the cov-requirements files were not included in the source distribution. * Simplify `recursive-include DIR *` to `graft DIR`. These two forms are functionally equivalent, but the latter is a bit easier to understand.
* Merge branch 'chandan/interactive-tests' into 'master'Chandan Singh2019-12-056-0/+167
|\ | | | | | | | | Add tests for interactive BuildStream operations See merge request BuildStream/buildstream!1706
| * Add tests for failed build in interactive modeChandan Singh2019-12-052-0/+115
| |
| * Add tests for interactive `bst init`Chandan Singh2019-12-052-0/+49
| | | | | | | | | | Add tests for interactive `bst init` command using [pexpect](https://pexpect.readthedocs.io).
| * requirements: Add pexpect for running interactive testsChandan Singh2019-12-052-0/+3
|/
* Merge branch 'juerg/cas-directory-reset' into 'master'Jürg Billeter2019-12-053-10/+8
|\ | | | | | | | | _sandboxreapi.py: Reset CasBasedDirectory instead of recreating it See merge request BuildStream/buildstream!1749
| * sandbox.py: Remove unused _set_virtual_directory() methodJürg Billeter2019-12-051-6/+0
| |
| * _sandboxreapi.py: Use CasBasedDirectory._reset()Jürg Billeter2019-12-051-3/+2
| | | | | | | | | | | | Calling _reset() instead of completely replacing the object fixes element plugins that use a virtual directory object across Sandbox.run() calls such as the compose plugin with integration commands.
| * _casbaseddirectory.py: Add _reset() methodJürg Billeter2019-12-051-1/+6
|/ | | | This reinitializes a CASBasedDirectory object from a directory digest.
* Merge branch 'juerg/runcli-unused-parameter' into 'master'Jürg Billeter2019-12-051-34/+5
|\ | | | | | | | | testing/runcli.py: Remove unused configure parameter from run() methods See merge request BuildStream/buildstream!1748
| * testing/runcli.py: Remove unused configure parameter from run() methodsJürg Billeter2019-12-051-34/+5
|/