summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * source-bundle: Enable --except optionFrancisco Redondo Marchena2018-08-132-2/+5
|/ | | | | | | Before this option was listed in help but was ignored when creating the source-bundle. Issue #468
* Merge branch 'valentindavid/bst_workspace_open_force_does_nothing-1.2' into ↵Valentin David2018-08-132-1/+57
|\ | | | | | | | | | | | | 'bst-1.2' _stream.py: Added functionality for workspace open -f See merge request BuildStream/buildstream!651
| * _stream.py: Added functionality for workspace open -fPhillip Smyth2018-08-132-1/+57
|/ | | | tests/frontend/workspace.py: Added tests
* Merge branch 'valentindavid/python3.7-1.2' into 'bst-1.2'Valentin David2018-08-131-2/+5
|\ | | | | | | | | Python 3.7 support See merge request BuildStream/buildstream!649
| * _artifactcache/cascache.py: Fix for PEP 479 / Python 3.7Jürg Billeter2018-08-131-2/+5
|/ | | | | | Do not rely on `StopIteration` bubbling up. https://www.python.org/dev/peps/pep-0479/
* Merge branch 'valentindavid/fix-debug-crash-1.2' into 'bst-1.2'Valentin David2018-08-131-2/+3
|\ | | | | | | | | Fix crash when --debug is passed See merge request BuildStream/buildstream!648
| * Fix crash when --debug is passedSam Thursfield2018-08-131-2/+3
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I hit the following backtrace running `bst --debug push` and `bst --debug build`: pid:16736 id:000[--:--:--][][] START Push pid:16736 id:000[--:--:--][][] START Loading pipeline pid:16736 id:000[00:00:00][][] SUCCESS Loading pipeline pid:16736 id:000[--:--:--][][] START Resolving pipeline pid:16736 id:000[--:--:--][][] BUG 'MesonElement' object has no attribute '_Element__cache_key' Traceback (most recent call last): File "/home/sam/.local/bin/bst", line 8, in <module> sys.exit(cli()) File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 162, in override_main standalone_mode=standalone_mode, **extra) File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/sam/.local/lib/python3.6/site-packages/click/core.py", line 535, in invoke return callback(*args, **kwargs) File "/home/sam/.local/lib/python3.6/site-packages/click/decorators.py", line 27, in new_func return f(get_current_context().obj, *args, **kwargs) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 471, in push app.stream.push(elements, selection=deps, remote=remote) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 341, in push fetch_subprojects=True) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_stream.py", line 814, in _load fetch_subprojects=fetch_subprojects) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in load for meta in meta_elements File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_pipeline.py", line 119, in <listcomp> for meta in meta_elements File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 894, in _new_from_meta element = project.create_element(artifacts, meta) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_project.py", line 184, in create_element element = self._element_factory.create(self._context, self, artifacts, meta) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_elementfactory.py", line 57, in create return element_type(context, project, artifacts, meta, default_config) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 196, in __init__ super().__init__(meta.name, context, project, meta.provenance, "element") File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 171, in __init__ self.debug("Created: {}".format(self)) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 407, in debug self.__message(MessageType.DEBUG, brief, detail=detail) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/plugin.py", line 652, in __message self.__context.message(message) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_context.py", line 338, in message self._message_handler(message, context=self) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/app.py", line 665, in _message_handler text = self.logger.render(message) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 585, in render return self._render(message) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 620, in _render text += widget.render(message) File "/home/sam/.local/lib/python3.6/site-packages/buildstream/_frontend/widget.py", line 228, in render _, key, missing = plugin._get_display_key() File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 1138, in _get_display_key cache_key = self._get_cache_key() File "/home/sam/.local/lib/python3.6/site-packages/buildstream/element.py", line 991, in _get_cache_key return self.__cache_key AttributeError: 'MesonElement' object has no attribute '_Element__cache_key' The issue here is that when `--debug` is enabled, Plugin.__init__() tries to log a message like this: pid:16929 id:001[--:--:--][????????][ main:tracker.bst ] DEBUG Created: meson element at tracker.bst [line 1 column 0] The log formatter is trying to get the cache key of the element in order to show it in the log, but the attribute hasn't yet been set because we've not got to the Element() constructor.
* Merge branch 'valentindavid/ruamel-version-1.2' into 'bst-1.2'Valentin David2018-08-131-1/+1
|\ | | | | | | | | Set version of ruaml.yaml to at least 0.15.41 but strictly less than 0.15.52. See merge request BuildStream/buildstream!647
| * Set version of ruaml.yaml to strictly less than 0.15.52.Valentin David2018-08-131-1/+1
|/ | | | | | | This allows to use version 0.15.51 rather than 0.15.0 which is required for Python 3. Fixes #571.
* Merge branch 'valentindavid/fix_included_junction_track-1.2' into 'bst-1.2'Valentin David2018-08-132-2/+37
|\ | | | | | | | | Fix tracking of junctions used in project.conf. See merge request BuildStream/buildstream!645
| * Fix tracking of junctions used in project.conf.Valentin David2018-08-132-2/+37
|/ | | | | | | | | | | | | | Stream._load() now returns early without resolving build pipeline when only tracking. Resolving track pipelines does not require to fully load project configurations when when elements to track are only junctions. However build pipelines require to fully load project configurations. This might not be possible in the case a project configuration includes a file from a junction that yet needs to be tracked. Fixes #565.
* Merge branch 'valentindavid/deterministic-source-1.2' into 'bst-1.2'Tristan Van Berkom2018-08-1231-30/+249
|\ | | | | | | | | Deterministic staging See merge request BuildStream/buildstream!634
| * Bump BST_CORE_ARTIFACT_VERSION for deterministic source pluginsValentin David2018-08-1226-26/+27
| |
| * Add some integration tests for source plugin determinism.Valentin David2018-08-121-0/+155
| |
| * buildstream/plugins/sources/remote.py: Make staging deterministic.Valentin David2018-08-121-0/+2
| |
| * buildstream/plugins/sources/zip.py: Fix non-determism in staging.Valentin David2018-08-121-2/+30
| | | | | | | | | | Staging could end-up with file with different rights depending on the umask. The extracted files need to get their access rights fixed.
| * Use deterministic umask when staging sources.Valentin David2018-08-121-1/+10
| | | | | | | | | | | | This fix is applied to plugins bzr, git, patch. Fixes #543 #544 #555.
| * buildstream/plugins/sources/local.py: Make staging deterministic.Valentin David2018-08-121-1/+25
|/ | | | | | | | | | | | | | Instead of copying metadata on files staged by local, we manually set mode to 0755 or 0644 depending on whether user execution was enabled on source file. This makes file modes deterministic independently on the way source was distributed. Non-deterministic mode copying all metadata can still be enabled by disable 'deterministic' Boolean configuration on the plugin. Fixes #527.
* setup.py: fix previous commitJavier Jardón2018-08-121-3/+2
|
* dev-requirements.txt: Remove obsolete commentsJavier Jardón2018-08-122-5/+1
| | | | | This also fixes an error with pylint in setup.py: setup.py:226:19: R1718: Consider using a set comprehension (consider-using-set-comprehension)
* setup.py: Specify minimum required version of pytest-cov pluginTristan Van Berkom2018-08-121-1/+1
| | | | | | This causes the new artifact tests to pass (unless you happened to already have a recent enough version of pytest-cov, in which case you didn't notice the breakage).
* Revert "Restrict version of pylint"Jürg Billeter2018-08-121-1/+0
| | | | | | | | | This reverts commit 4f168b9b6a02216e2fae24d758ae6b778e545869. The latest version of pytest_pylint works fine with pylint 2, which means there is no longer a reason to restrict the pylint version. pylint 2 is required for Python 3.7.
* .gitlab-ci.yml: Use the same version of the docker image for all jobsJavier Jardón2018-08-121-4/+4
|
* .gitlab-ci.yml: Run tests in fedora28 imageJavier Jardón2018-08-121-0/+5
|
* .gitlab-ci-yml: Add ubuntu 18.04 testPhil Dawson2018-08-121-0/+3
|
* Merge branch 'chandan/dev-reqs-1.2' into 'bst-1.2'Javier Jardón2018-08-123-14/+27
|\ | | | | | | | | Move development reqirements to dev-requirements.txt See merge request BuildStream/buildstream!642
| * Move development reqirements to dev-requirements.txtChandan Singh2018-08-123-14/+27
|/ | | | | | | | | | | | | | | | | | | | | | | | | | This is backport of !637 to bst-1.2. There were a couple of differences between the requirements listed in the `master` branch and the `bst-1.2` branch so I have left them untouched. For reference, here are the differences between them: ```diff --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -2,8 +2,9 @@ # random crashes with 4.4.2 coverage == 4.4.0 pep8 +pylint >= 1.8 , < 2 pytest >= 3.1.0 -pytest-cov >= 2.5.0 +pytest-cov pytest-datafiles pytest-env pytest-pep8 ```
* Merge branch 'valentindavid/498_bwrap_environment-1.2' into 'bst-1.2'Valentin David2018-08-101-3/+7
|\ | | | | | | | | Set environment in bwrap command line instead of its environment See merge request BuildStream/buildstream!632
| * Set environment in bwrap command line instead of its environmentValentin David2018-08-101-3/+7
|/ | | | Fixes #498
* Merge branch 'tristan/pin-ruamel-version-1.2' into 'bst-1.2'Tristan Van Berkom2018-08-101-1/+1
|\ | | | | | | | | Backport fix for ruamel.yaml version to 1.2 See merge request BuildStream/buildstream!631
| * setup.py: Pin ruamel.yaml version to <= 0.15Jim MacArthur2018-08-101-1/+1
|/
* Merge branch 'jjardon/max-jobs_1.2' into 'bst-1.2'Javier Jardón2018-08-091-2/+4
|\ | | | | | | | | [1.2] Backport of !620: Restrict max-jobs See merge request BuildStream/buildstream!629
| * buildstream/_project.py: Restrict max-jobsJavier Jardón2018-08-091-1/+4
| | | | | | | | | | | | even if the machine have mores cores available Patch taken from YBD: https://gitlab.com/baserock/ybd/blob/master/ybd/app.py#L227
| * buildstream/_project.py:Javier Jardón2018-08-091-2/+1
|/ | | | | | | | | Multiprocessing with n_jobs given by multiprocessing.cpu_count() is not optimal for systems where not all CPU can be used (in particular, Docker, CI etc). As suggested in the multiprocessing docs (http://docs.python.org/3/library/multiprocessing.html#multiprocessing.cpu_count) using len(os.sched_getaffinity(0)) might be better.
* Merge branch 'valentindavid/fuse-create-flags-1.2' into 'bst-1.2'Valentin David2018-08-092-3/+7
|\ | | | | | | | | Keep original flags for create in SafeHardlinks. See merge request BuildStream/buildstream!625
| * Keep original flags for create in SafeHardlinks.Valentin David2018-08-092-3/+7
|/ | | | | | | | | | When open(2) is used with flags O_CREAT|O_RDWR, the file descriptor must be readable. Unfortunately O_RDWR was not passed which made read fail with EBADF and mmap to signal SIGBUS. This issue happened with man-db for example. Fixes #143.
* Merge branch 'tiagogomes/issue-520-bp' into 'bst-1.2'Tiago Gomes2018-08-093-8/+10
|\ | | | | | | | | cascache: move tmp directory one level up See merge request BuildStream/buildstream!623
| * cascache: move tmp directory one level upTiago Gomes2018-08-093-8/+10
|/ | | | | | | | | | | | | The CAS uses a temp directory while manipulating the cache, temporary files can be added and removed while adding artifacts to the cache here. Since calculation of the cache size happens in parallel to artifact cache additions, this causes race conditions in the size calculation job, as we end up calling `stat` on temporary files which are being removed in parallel. Handle this by moving the temporary directory out of the way, and avoid considering the tmp directory when calculating the cache size
* Merge branch 'tiagogomes/issue-550-backport' into 'bst-1.2'Tiago Gomes2018-08-062-3/+3
|\ | | | | | | | | PullQueue: fix resource used See merge request BuildStream/buildstream!612
| * PullQueue: fix resource usedTiago Gomes2018-08-062-3/+3
|/ | | | | | Improve fetchers, builders and pushers documentation. Closes #550.
* Merge branch 'Qinusty/backport-skipped-562' into 'bst-1.2'Qinusty2018-08-063-3/+12
|\ | | | | | | | | Backport SKIPPED MessageType and duplicate skipped message fix See merge request BuildStream/buildstream!609
| * cascache.py: Display SKIPPED message for each remote skippedQinusty/backport-skipped-562Josh Smith2018-08-061-3/+10
| |
| * message.py: MessagesType.SKIPPED addedJosh Smith2018-08-062-0/+2
|/
* Merge branch 'Qinusty/553-backport-1.2' into 'bst-1.2'Qinusty2018-08-062-18/+24
|\ | | | | | | | | Backport cascache push/pull messages See merge request BuildStream/buildstream!608
| * cascache.py: Add pull info detailing which remote is being pulled fromQinusty/553-backport-1.2Josh Smith2018-08-061-0/+2
| | | | | | | | Work towards fixing issue #275
| * cascache.py: Add push info detailing which remote is being pushed toJosh Smith2018-08-061-0/+4
| | | | | | | | Work towards fixing issue #275
| * element.py: Make Element._get_brief_display_key() available to Internal APIJosh Smith2018-08-061-18/+18
|/ | | | | This change will allow for more detailed info messages throughout the CASCache.
* man: Updating man pages for the upcoming 1.1.5 release1.1.5Tristan Van Berkom2018-08-0617-59/+61
|
* NEWS: Adding news entry for the include directive work.Tristan Van Berkom2018-08-061-0/+3
|
* Merge branch 'Qinusty/563-cache-quota-restriction' into 'bst-1.2'Tristan Van Berkom2018-08-062-17/+64
|\ | | | | | | | | Backport cache quota restriction to 1.2 See merge request BuildStream/buildstream!607