summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* contributing: snakeviz replaces pyflame+flamegraphsnakevizAngelos Evripiotis2019-02-111-18/+5
| | | | | | | | | | | | | Replace the instructions for pyflame+flamegraph with simpler ones for snakeviz. For our general use-case this seems to be easier and better. Usage of this tool was demonstrated at the 2019 BuildStream Gathering in January by Daniel Silverstone, when presenting the aggregate results of profiling on many target environments. Here is the relevant mailing list thread: "Profiling before the gathering" https://mail.gnome.org/archives/buildstream-list/2019-January/msg00057.html
* Merge branch 'juerg/symlinks' into 'master'Jürg Billeter2019-02-115-38/+43
|\ | | | | | | | | Symlink fixes See merge request BuildStream/buildstream!1138
| * tests/sources/local.py: Add directory symlink testJürg Billeter2019-02-111-0/+20
| |
| * _casbaseddirectory.py: Do not mimic os.walk() in list_relative_paths()Jürg Billeter2019-02-111-14/+1
| | | | | | | | | | | | This matches the change in utils.list_relative_paths() that now sorts all symlinks as files, instead of following the broken behavior of os.walk().
| * utils.py: Fix sorting of symlinks to directoriesJürg Billeter2019-02-111-13/+13
| | | | | | | | | | | | | | | | | | os.walk() resolves symlinks to check whether they point to a directory even when followlinks is set to False. We already work around that broken behavior by extracting symlinks from `dirnames`. However, the sort order was still incorrect as we returned symlinks in dirnames before files and other symlinks. This change fixes this, sorting all files and symlinks in a single list.
| * utils.py: Remove list_dirs parameter from list_relative_paths()Jürg Billeter2019-02-112-9/+7
| | | | | | | | | | | | | | | | list_dirs was always True in the BuildStream code base. There was also a bug in the list_dirs=False code path as it did not return symlinks in `dirnames`. This is an API break, however, there are no known external callers.
| * sandbox/sandbox.py: Do not follow symlinks in _has_command()Jürg Billeter2019-02-111-2/+2
| | | | | | | | This is required to ensure symlinks are not resolved on the host.
| * local.py: Do not follow symlinks in local directoriesJürg Billeter2019-02-111-3/+3
|/ | | | | isdir() follows symlinks on the host, resulting in potential host contamination. This change reorders the file checks to avoid this issue.
* Merge branch 'juerg/import' into 'master'Jürg Billeter2019-02-111-0/+4
|\ | | | | | | | | | | | | import.py: Validate config node Closes #662 See merge request BuildStream/buildstream!1141
| * import.py: Validate config nodejuerg/importJürg Billeter2019-02-101-0/+4
|/ | | | Fixes #662.
* Merge branch 'jjardon/fedora_27' into 'master'Jürg Billeter2019-02-091-6/+1
|\ | | | | | | | | .gitlab-ci.yml: Remove testing on Fedora 27 See merge request BuildStream/buildstream!1136
| * .gitlab-ci.yml: Remove testing on Fedora 27Javier Jardón2019-02-091-6/+1
|/ | | | | | Fedora 27 is EOL since 2018-11-30 See https://fedoraproject.org/wiki/End_of_life
* Merge branch 'abderrahim/artifact-cache-junction' into 'master'Javier Jardón2019-02-082-6/+11
|\ | | | | | | | | | | | | Use artifact cache specs from the parent project before those defined in junctions Closes #618 and #401 See merge request BuildStream/buildstream!1113
| * _project.py: use artifact caches from the parent project for junctionsabderrahim/artifact-cache-junctionAbderrahim Kitouni2019-02-082-5/+10
| | | | | | | | | | | | | | This makes a junction use the artifact cache of the parent project before the ones defined for the junction Fixes #401
| * _artifactcache.py: don't leak the project specific remote cachesAbderrahim Kitouni2019-02-081-1/+1
|/ | | | | | | the code for initializing remotes added the project specific remote caches to the global list instead of making a copy. Fixes #618
* Merge branch 'phil/plugin-testing-api' into 'master'Phil Dawson2019-02-08101-159/+274
|\ | | | | | | | | | | | | Expose basic api for testing external plugins. Closes #847 See merge request BuildStream/buildstream!1075
| * Expose basic api for testing external plugins.phil/plugin-testing-apiPhil Dawson2019-02-08101-159/+274
|/ | | | | | | | | | | | | | | | | | We want external plugins to be able to make use of the core testing utils. This commit exposes the basic utilities which are currently in use in bst-external plugins. If necessary, more utilities could be exposed in the future. Moves the following files from tests/testutils/ to buildstream/plugintestingutils/: o runcli.py o integration.py As part of this, this commit makes the following changes to runcli.py and integration.py: o runcli.py: Fix linting errors o runcli.py: Add user facing documentation o Integration.py: Add user facing documentation
* Merge branch 'willsalmon/log_formating' into 'master'Will Salmon2019-02-072-4/+19
|\ | | | | | | | | Add more log formatting options See merge request BuildStream/buildstream!1125
| * Add more log formatting optionsWilliam Salmon2019-02-072-4/+19
|/ | | | The 'wallclock-us' is a option that we thought we already had.
* Merge branch 'danielsilverstone-ct/other-roaring' into 'master'Benjamin Schubert2019-02-064-22/+18
|\ | | | | | | | | | | | | loadelement.py: Switch from roaringbitmap to pyroaring Closes #899 See merge request BuildStream/buildstream!1133
| * Update testing images to include c++ compilerBenjamin Schubert2019-02-061-13/+13
| |
| * loadelement.py: Switch from roaringbitmap to pyroaringDaniel Silverstone2019-02-063-9/+5
|/ | | | | | | | The licence for roaringbitmap was not compatible. pyroaring is a similar project which is MIT licensed and is slightly better on memory too. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'jonathan/wsl-tests' into 'master'Jonathan Maw2019-02-0622-86/+118
|\ | | | | | | | | | | | | Add pre-merge tests that use a WSL runner Closes #852 See merge request BuildStream/buildstream!1108
| * tests: Fix test failures caused by unix sockets' path length limitjonathan/wsl-testsJonathan Maw2019-02-061-1/+8
| | | | | | | | | | | | The unix specification for unix sockets only allocates ~100 characters for the length of the path. This may be longer than the path for a CI runner, or the path to a user's buildstream directory.
| * tests: Skip tests that use sandboxes on WSLJonathan Maw2019-02-0620-85/+96
| |
| * gitlab-ci: Add commands to be run by WSL runnersJonathan Maw2019-02-061-0/+14
|/
* Merge branch 'danielsilverstone-ct/roaring-bitmaps' into 'master'Benjamin Schubert2019-02-053-3/+18
|\ | | | | | | | | Switch to roaring bitmaps for the loader dependency caches See merge request BuildStream/buildstream!1128
| * loadelement.py: Use roaring bitmaps in dep cacheDaniel Silverstone2019-02-051-3/+12
| | | | | | | | | | | | | | | | | | | | | | The dependency cache in LoadElement causes the peak RAM consumption of the loader to be exceedingly large, upwards of 25GB for a test of a Debian stack. By switching from the old dict cache to a roaring bitmap based cache, we reduce the cost of the loader cache in RAM terms to around 5.5GB for the same stack, which makes it plausible for the short term. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * requirements: Add roaringbitmap to the requirementsDaniel Silverstone2019-02-052-0/+6
|/ | | | | | | In order to support use of roaring bitmaps in the loader, we need to depend on it here. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* Merge branch 'tpollard/buildremote' into 'master'Tom Pollard2019-02-056-6/+107
|\ | | | | | | | | Add --remote, -r option to bst build, inline with pull & push See merge request BuildStream/buildstream!1119
| * Add --remote, -r option to bst build, inline with pull & pushTom Pollard2019-02-056-6/+107
|/ | | | | | Providing a remote will limit build's pull/push remote actions to the given remote specifically, ignoring those defined via user or project configuration.
* Merge branch 'jennis/warn_for_nonexistent_domains' into 'master'James Ennis2019-02-045-4/+49
|\ | | | | | | | | Fail when we explictly try to include/exclude non-existent domains in a filter element See merge request BuildStream/buildstream!1117
| * filter.py: Fail if declared domains do not exist in the parent elementjennis/warn_for_nonexistent_domainsJames Ennis2019-02-045-4/+49
|/ | | | | | | | | This patch also uncovered the fact that our test_filter_deps_ok() test has been inaccurate. Thus the element built in this test (deps-permitted.bst) has been modified so that it build depends on the input.bst element, as it should. tests/filter.py: Ensure deps_ok test passes
* Merge branch 'jennis/filter-docs' into 'master'James Ennis2019-02-012-21/+124
|\ | | | | | | | | | | | | Improve our filter documentation Closes #278 See merge request BuildStream/buildstream!1112
| * filter.py: Add an example to the documentationjennis/filter-docsJames Ennis2019-02-011-3/+105
| |
| * filter.py/filter.yaml: Documentation improvementsJames Ennis2019-02-012-21/+22
|/
* Merge branch 'phil/cache-key-stability-test' into 'master'Phil Dawson2019-02-016-0/+66
|\ | | | | | | | | tests/cachekey: Test cache keys are independent of target elements See merge request BuildStream/buildstream!1123
| * tests/cachekey: Test cache keys are independent of target elementsphil/cache-key-stability-testPhil Dawson2019-02-016-0/+66
|/
* Merge branch 'bschubert/loader' into 'master'Benjamin Schubert2019-02-012-116/+86
|\ | | | | | | | | Cleanup loader by linking LoadElements sooner See merge request BuildStream/buildstream!1122
| * Pass element directly to collect_elementbschubert/loaderBenjamin Schubert2019-01-311-13/+9
| | | | | | | | This simplifies the loading
| * Add LoadElement in dependency list for LoadElement idrectlyBenjamin Schubert2019-01-312-106/+80
|/ | | | | | This removes the need for the 'Dependency' list to then be matched with the corresponding LoadElement and will allow iterating the graph more easily
* Merge branch 'chandan/fix-git-version-mac' into 'master'Chandan Singh2019-01-311-1/+1
|\ | | | | | | | | testutils/site.py: Support parsing more exotic git versions See merge request BuildStream/buildstream!1118
| * testutils/site.py: Support parsing more exotic git versionsChandan Singh2019-01-311-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | We use output of `git --version` to determine if we can run some tests that rely on features from newer git versions. Usually, we expect the output to be like: git version 2.17.2 On some platforms, like MacOS, there could be a suffix after the version string, so that it looks something like: git version 2.17.2 (Apple Git-113) This causes things to fail like so: ValueError: invalid literal for int() with base 10: '2 (Apple Git-113)\n' Fix logic around `HAVE_OLD_GIT` such that we split the output of `git --version` without limit on how many times we split. Previously we used to split only twice so the suffixes like `(Apple Git-113)` are not part of the parsed version.
* Merge branch 'chandan/min-version-click-man' into 'master'Chandan Singh2019-01-311-1/+1
|\ | | | | | | | | tox.ini: Specify minimum version of click-man See merge request BuildStream/buildstream!1120
| * tox.ini: Specify minimum version of click-manChandan Singh2019-01-311-1/+1
|/ | | | | | | | | | | `click-man` versions < 0.3.0 do not properly support multiple entrypoints. Since this was added to `tox` after `0.3.0` was released, `tox` should never be pulling older versions. But, let's add it here for documentation purposes. See https://gitlab.com/BuildStream/buildstream/merge_requests/1107#note_135187046 for some background on this.
* Merge branch 'doraskayo/filter-indirect-deps-fix' into 'master'James Ennis2019-01-304-1/+42
|\ | | | | | | | | | | | | filter.py: don't recurse when staging dependencies Closes #883 See merge request BuildStream/buildstream!1110
| * filter.py: don't recurse when staging dependenciesDor Askayo2019-01-304-1/+42
|/ | | | | | | Also bump the element's version so cached artifacts would be invalidated. Fixes #883
* Merge branch 'aevri/are_you_sure' into 'master'Angelos Evripiotis2019-01-294-46/+5
|\ | | | | | | | | BREAK:remove unconditional 'are you sure?' prompts See merge request BuildStream/buildstream!1061
| * BREAK:remove unconditional 'are you sure?' promptsAngelos Evripiotis2019-01-294-46/+5
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a breaking change, as it affects behaviour that people might be relying on. An entry has been added to NEWS. As proposed on the mailing list, this change removes the unconditional prompts on: o: bst workspace reset o: bst workspace close --remove-dir If interactive, these commands would always interrupt you with a prompt like this: This will remove all your changes, are you sure? This seems like it may just save someone's work some time. It may also condition folks to hit 'y' quickly without thinking. This change also makes the non-interactive behaviour consistent with the interactive behaviour in the default case. There is also the case of the prompt configured by 'really-workspace-close-project-inaccessible', which may be tackled in later work. This change also removes the new config options to suppress those prompts, and their associated news entry. The relevant bit of the mailing list conversation is here: https://mail.gnome.org/archives/buildstream-list/2018-December/msg00106.html The issue to make interactive and non-interactive behaviour consistent is here: https://gitlab.com/BuildStream/buildstream/issues/744
* Merge branch 'valentindavid/make_cache_dir' into 'master'Jürg Billeter2019-01-291-0/+1
|\ | | | | | | | | | | | | Make sure testing cache directory exists Closes #873 See merge request BuildStream/buildstream!1092
View Lorry Controller Status