summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Remove cleanup functionality from loaderjonathan/junction-no-tmpdirJonathan Maw2019-02-112-41/+6
| | | | | | | | | | | | Since junction check-outs are now persistent, there is no need to clean up temporary directories once finished. Now, junction checkouts are stored within the top-level project dir and will be cleaned-up when that directory is removed. This involves changes in: * loader.py: Remove cleanup logic and passing-around of tmpdir. * _project.py: Remove passing around tmpdir and calling loader's cleanup.
* loader.py: Load junction from workspace if one's openJonathan Maw2019-02-081-1/+6
|
* loader.py: Stage junctions into .bst instead of a tmpdirJonathan Maw2019-02-081-4/+7
|
* Merge branch 'willsalmon/log_formating' into 'master'Will Salmon2019-02-072-4/+19
|\ | | | | | | | | Add more log formatting options See merge request BuildStream/buildstream!1125
| * Add more log formatting optionsWilliam Salmon2019-02-072-4/+19
|/ | | | The 'wallclock-us' is a option that we thought we already had.
* Merge branch 'danielsilverstone-ct/other-roaring' into 'master'Benjamin Schubert2019-02-064-22/+18
|\ | | | | | | | | | | | | loadelement.py: Switch from roaringbitmap to pyroaring Closes #899 See merge request BuildStream/buildstream!1133
| * Update testing images to include c++ compilerBenjamin Schubert2019-02-061-13/+13
| |
| * loadelement.py: Switch from roaringbitmap to pyroaringDaniel Silverstone2019-02-063-9/+5
|/ | | | | | | | The licence for roaringbitmap was not compatible. pyroaring is a similar project which is MIT licensed and is slightly better on memory too. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'jonathan/wsl-tests' into 'master'Jonathan Maw2019-02-0622-86/+118
|\ | | | | | | | | | | | | Add pre-merge tests that use a WSL runner Closes #852 See merge request BuildStream/buildstream!1108
| * tests: Fix test failures caused by unix sockets' path length limitjonathan/wsl-testsJonathan Maw2019-02-061-1/+8
| | | | | | | | | | | | The unix specification for unix sockets only allocates ~100 characters for the length of the path. This may be longer than the path for a CI runner, or the path to a user's buildstream directory.
| * tests: Skip tests that use sandboxes on WSLJonathan Maw2019-02-0620-85/+96
| |
| * gitlab-ci: Add commands to be run by WSL runnersJonathan Maw2019-02-061-0/+14
|/
* Merge branch 'danielsilverstone-ct/roaring-bitmaps' into 'master'Benjamin Schubert2019-02-053-3/+18
|\ | | | | | | | | Switch to roaring bitmaps for the loader dependency caches See merge request BuildStream/buildstream!1128
| * loadelement.py: Use roaring bitmaps in dep cacheDaniel Silverstone2019-02-051-3/+12
| | | | | | | | | | | | | | | | | | | | | | The dependency cache in LoadElement causes the peak RAM consumption of the loader to be exceedingly large, upwards of 25GB for a test of a Debian stack. By switching from the old dict cache to a roaring bitmap based cache, we reduce the cost of the loader cache in RAM terms to around 5.5GB for the same stack, which makes it plausible for the short term. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * requirements: Add roaringbitmap to the requirementsDaniel Silverstone2019-02-052-0/+6
|/ | | | | | | In order to support use of roaring bitmaps in the loader, we need to depend on it here. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'tpollard/buildremote' into 'master'Tom Pollard2019-02-056-6/+107
|\ | | | | | | | | Add --remote, -r option to bst build, inline with pull & push See merge request BuildStream/buildstream!1119
| * Add --remote, -r option to bst build, inline with pull & pushTom Pollard2019-02-056-6/+107
|/ | | | | | Providing a remote will limit build's pull/push remote actions to the given remote specifically, ignoring those defined via user or project configuration.
* Merge branch 'jennis/warn_for_nonexistent_domains' into 'master'James Ennis2019-02-045-4/+49
|\ | | | | | | | | Fail when we explictly try to include/exclude non-existent domains in a filter element See merge request BuildStream/buildstream!1117
| * filter.py: Fail if declared domains do not exist in the parent elementjennis/warn_for_nonexistent_domainsJames Ennis2019-02-045-4/+49
|/ | | | | | | | | This patch also uncovered the fact that our test_filter_deps_ok() test has been inaccurate. Thus the element built in this test (deps-permitted.bst) has been modified so that it build depends on the input.bst element, as it should. tests/filter.py: Ensure deps_ok test passes
* Merge branch 'jennis/filter-docs' into 'master'James Ennis2019-02-012-21/+124
|\ | | | | | | | | | | | | Improve our filter documentation Closes #278 See merge request BuildStream/buildstream!1112
| * filter.py: Add an example to the documentationjennis/filter-docsJames Ennis2019-02-011-3/+105
| |
| * filter.py/filter.yaml: Documentation improvementsJames Ennis2019-02-012-21/+22
|/
* Merge branch 'phil/cache-key-stability-test' into 'master'Phil Dawson2019-02-016-0/+66
|\ | | | | | | | | tests/cachekey: Test cache keys are independent of target elements See merge request BuildStream/buildstream!1123
| * tests/cachekey: Test cache keys are independent of target elementsphil/cache-key-stability-testPhil Dawson2019-02-016-0/+66
|/
* Merge branch 'bschubert/loader' into 'master'Benjamin Schubert2019-02-012-116/+86
|\ | | | | | | | | Cleanup loader by linking LoadElements sooner See merge request BuildStream/buildstream!1122
| * Pass element directly to collect_elementbschubert/loaderBenjamin Schubert2019-01-311-13/+9
| | | | | | | | This simplifies the loading
| * Add LoadElement in dependency list for LoadElement idrectlyBenjamin Schubert2019-01-312-106/+80
|/ | | | | | This removes the need for the 'Dependency' list to then be matched with the corresponding LoadElement and will allow iterating the graph more easily
* Merge branch 'chandan/fix-git-version-mac' into 'master'Chandan Singh2019-01-311-1/+1
|\ | | | | | | | | testutils/site.py: Support parsing more exotic git versions See merge request BuildStream/buildstream!1118
| * testutils/site.py: Support parsing more exotic git versionsChandan Singh2019-01-311-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | We use output of `git --version` to determine if we can run some tests that rely on features from newer git versions. Usually, we expect the output to be like: git version 2.17.2 On some platforms, like MacOS, there could be a suffix after the version string, so that it looks something like: git version 2.17.2 (Apple Git-113) This causes things to fail like so: ValueError: invalid literal for int() with base 10: '2 (Apple Git-113)\n' Fix logic around `HAVE_OLD_GIT` such that we split the output of `git --version` without limit on how many times we split. Previously we used to split only twice so the suffixes like `(Apple Git-113)` are not part of the parsed version.
* Merge branch 'chandan/min-version-click-man' into 'master'Chandan Singh2019-01-311-1/+1
|\ | | | | | | | | tox.ini: Specify minimum version of click-man See merge request BuildStream/buildstream!1120
| * tox.ini: Specify minimum version of click-manChandan Singh2019-01-311-1/+1
|/ | | | | | | | | | | `click-man` versions < 0.3.0 do not properly support multiple entrypoints. Since this was added to `tox` after `0.3.0` was released, `tox` should never be pulling older versions. But, let's add it here for documentation purposes. See https://gitlab.com/BuildStream/buildstream/merge_requests/1107#note_135187046 for some background on this.
* Merge branch 'doraskayo/filter-indirect-deps-fix' into 'master'James Ennis2019-01-304-1/+42
|\ | | | | | | | | | | | | filter.py: don't recurse when staging dependencies Closes #883 See merge request BuildStream/buildstream!1110
| * filter.py: don't recurse when staging dependenciesDor Askayo2019-01-304-1/+42
|/ | | | | | | Also bump the element's version so cached artifacts would be invalidated. Fixes #883
* Merge branch 'aevri/are_you_sure' into 'master'Angelos Evripiotis2019-01-294-46/+5
|\ | | | | | | | | BREAK:remove unconditional 'are you sure?' prompts See merge request BuildStream/buildstream!1061
| * BREAK:remove unconditional 'are you sure?' promptsAngelos Evripiotis2019-01-294-46/+5
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a breaking change, as it affects behaviour that people might be relying on. An entry has been added to NEWS. As proposed on the mailing list, this change removes the unconditional prompts on: o: bst workspace reset o: bst workspace close --remove-dir If interactive, these commands would always interrupt you with a prompt like this: This will remove all your changes, are you sure? This seems like it may just save someone's work some time. It may also condition folks to hit 'y' quickly without thinking. This change also makes the non-interactive behaviour consistent with the interactive behaviour in the default case. There is also the case of the prompt configured by 'really-workspace-close-project-inaccessible', which may be tackled in later work. This change also removes the new config options to suppress those prompts, and their associated news entry. The relevant bit of the mailing list conversation is here: https://mail.gnome.org/archives/buildstream-list/2018-December/msg00106.html The issue to make interactive and non-interactive behaviour consistent is here: https://gitlab.com/BuildStream/buildstream/issues/744
* Merge branch 'valentindavid/make_cache_dir' into 'master'Jürg Billeter2019-01-291-0/+1
|\ | | | | | | | | | | | | Make sure testing cache directory exists Closes #873 See merge request BuildStream/buildstream!1092
| * Make sure testing cache directory existsValentin David2019-01-291-0/+1
|/ | | | Fixes #873
* Merge branch 'valentindavid/wrong_type_in_status_code' into 'master'Jürg Billeter2019-01-291-3/+3
|\ | | | | | | | | | | | | Fix type of error codes in CAS server Closes #882 See merge request BuildStream/buildstream!1099
| * Fix type of error codes in CAS serverValentin David2019-01-291-3/+3
|/ | | | Fixes #882.
* Merge branch 'valentindavid/crash_in_scheduler_857' into 'master'Jürg Billeter2019-01-281-1/+1
|\ | | | | | | | | | | | | Fix crash when spawned job completes very fast Closes #857 See merge request BuildStream/buildstream!1095
| * Fix crash when spawned job completes very fastvalentindavid/crash_in_scheduler_857Valentin David2019-01-281-1/+1
|/ | | | | | | | | | | | | | | | Job can complete before we return from `Job.span()` to `Scheduler._spawn_job()`, so that `_active_jobs` would not yet contain the job. This would print a stack on the console and try to run a second time the job which can have unexpected effects. In order to reproduce the issue, in `buildstream/_scheduler/jobs/job.py`, in `Job.spawn`, add a call to `time.sleep()` right before call to `asyncio.get_child_watcher()`. This fixes issue #857.
* Merge branch 'abderrahim/protobuf-version' into 'master'Javier Jardón2019-01-281-2/+2
|\ | | | | | | | | | | | | requirements/requirements.in: update minimum versions Closes #884 See merge request BuildStream/buildstream!1114
| * requirements/requirements.in: require Click >= 7.0abderrahim/protobuf-versionAbderrahim Kitouni2019-01-281-1/+1
| | | | | | | | This is needed since 629a6e524, and was lost in the conversion to requirements.in
| * requirements/requirements.in: require protobuf >= 3.6Abderrahim Kitouni2019-01-281-1/+1
|/ | | | This is needed since 0f2bc3754
* Merge branch 'tpollard/829' into 'master'Tom Pollard2019-01-283-30/+94
|\ | | | | | | | | | | | | Download buildtrees on demand for bst shell --use-buildtree Closes #829 See merge request BuildStream/buildstream!1050
| * Download buildtrees on demand for bst shell --use-buildtreetpollard/829Tom Pollard2019-01-283-29/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Provide bst shell --use-buildtree the ability to attempt to acquire missing buildtrees, given respective option, user pull-buildtree context and remote availability. _frontend/cli.py: Refactor logic for determining --use-buildtree option with given opportunity to attempt pulling a non-local buildtree. Element loaded with artifact_config to allow remote querying. _stream.py: With given user option and element state, construct PullQueue to fetch remote buildtree. Continue or Error without buildtree if cannot be attained. tests/integration/build-tree.py: Update to support new usecases
| * _stream.py: Add use_artifact_config opt arg for load_selection()Tom Pollard2019-01-281-2/+5
|/ | | | | use_artifact_config added as an optional default arg, allowing for loading of given elements artifact remote config.
* Merge branch 'aevri/bst_track_guidance' into 'master'Angelos Evripiotis2019-01-2813-18/+18
|\ | | | | | | | | Fixup refs to 'bst track' and 'bst fetch' See merge request BuildStream/buildstream!1086
| * Fixup refs to 'bst fetch'Angelos Evripiotis2019-01-283-4/+4
| | | | | | | | | | Now that 'bst fetch' is obsolete, change guidance to refer to the replacement 'bst source fetch' instead.
| * Fixup refs to 'bst track'Angelos Evripiotis2019-01-2812-13/+13
| | | | | | | | | | Now that 'bst track' is obsolete, change guidance to refer to the replacement 'bst source track' instead.
View Lorry Controller Status