summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* _profile.py: Update copyright statementjennis/add_new_profile_topicJames Ennis2019-01-251-0/+2
|
* _profile.py: Added a new profiling topic, load-selectionJames Ennis2019-01-252-0/+8
| | | | | profile_start() and profile_end() calls have been added to Stream.load_selection()
* _profile.py: Added a new profiling topic, schedulerJames Ennis2019-01-252-0/+7
| | | | | profile_start() and profile_end() calls have been incorporated into Scheduler.run()
* Merge branch 'tristan/cache-management-logging' into 'master'Tristan Van Berkom2019-01-2410-63/+260
|\ | | | | | | | | Cache management logging enhancements See merge request BuildStream/buildstream!1105
| * _artifactcache.py: Correcting API documenting comment for remove()Tristan Van Berkom2019-01-241-2/+1
| | | | | | | | | | | | This seems to have been copy/pasted from cascache, and documents the function to possibly return None if defer_prune was specified, but this function does not expose defer_prune.
| * _scheduler/jobs/cleanupjob.py: Update cache size while processingTristan Van Berkom2019-01-241-1/+14
| | | | | | | | | | | | Updates the known cache size in the main process while the cleanup process is ongoing, so that the status indicators update live while the cleanup happens.
| * _scheduler/jobs/job.py: Allow subclasses to message the frontendTristan Van Berkom2019-01-241-19/+50
| |
| * _artifactcache.py: Added client progress callback to ArtifactCache.clean()Tristan Van Berkom2019-01-241-1/+12
| |
| * _artifactcache.py: Add status messages in cache management operationsTristan Van Berkom2019-01-241-5/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | Added some useful status messages when: * Calculating a new artifact cache usage size * Starting a cleanup * Finishing a cleanup Also enhanced messaging about what was cleaned up so far when aborting a cleanup.
| * _frontend/status.py: Added Cache size usage indicator to status barTristan Van Berkom2019-01-241-7/+35
| | | | | | | | | | This also adds some comments around the main status bar heading rendering function.
| * _frontend/widget.py: Added cache usage entry in the startup headingTristan Van Berkom2019-01-241-0/+1
| |
| * _context.py: Added get_artifact_cache_usage()Tristan Van Berkom2019-01-241-1/+11
| | | | | | | | | | | | | | | | A frontend facing API for obtaining usage statistics. I would have put this on Stream instead, but the Context seems to be the de facto place for looking up the artifact cache in general so let's put it here.
| * _artifactcache.py: Added ArtifactCacheUsage()Tristan Van Berkom2019-01-241-0/+35
| | | | | | | | | | A simple object which creates a snapshot of current usage statistics for easy reporting in the frontend.
| * _artifactcache.py: Refactored to use utils._get_volume_size()Tristan Van Berkom2019-01-243-20/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will benefit from a better UtilError being raised, and and turns the artifact cache's local function into a one liner. The loop which finds the first existing directory in the given path has been removed, being meaningless due to the call to os.makedirs() in ArtifactCache.__init__(). The local function was renamed to _get_cache_volume_size() and no longer takes any arguments, which is more suitable for the function as it serves as a testing override surface for unittest.mock(). The following test cases which use the function to override the ArtifactCache behavior have been updated to use the new overridable function name: tests/artifactcache/cache_size.py tests/artifactcache/expiry.py
| * utils.py: Add _get_volume_size()Tristan Van Berkom2019-01-241-2/+22
| | | | | | | | We can streamline this call to os.statvfs() in a few places.
| * _frontend/app.py: Initialize logging before preflighting the artifact cacheTristan Van Berkom2019-01-241-5/+7
|/ | | | The artifact cache emits messages, and we want to allow that in preflight.
* Merge branch 'tristan/cache-management' into 'master'Tristan Van Berkom2019-01-245-24/+141
|\ | | | | | | | | Cache management fixes See merge request BuildStream/buildstream!1091
| * tests/artifactcache/expiry.py: Test that expiry happens firsttristan/cache-managementTristan Van Berkom2019-01-241-0/+64
| |
| * _frontend/widget.py: Render core messages more like other messagesTristan Van Berkom2019-01-243-21/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to test when core activities occur by parsing the stderr in tests, we should make the messages conform more. At the same time, this restores alignment of columns in core messages with the element processing related messages. Also, _scheduler/scheduler.py is updated to make it's activity names conform to the (current) 5 character limit for the sake of alignment. The tests/frontend/logging.py test gets it's regexes updated for the log lines it checks for in stderr.
| * _scheduler/scheduler.py: Run cache size exclusively at startupTristan Van Berkom2019-01-241-2/+59
| | | | | | | | | | | | | | | | | | | | | | | | When running any session that has Queues which require Resource.CACHE, check if our loaded estimated size exceeds the quota, and if so; lock the Resource.CACHE resource exclusively right away and run an exclusive initial cache size job. This ensures we cleanup first before doing anything which might add to the cache at startup time, if deemed needed. This is a partial fix for issue #737
| * _scheduler/resources.py: Dont error out in unregister_exclusive_interest()Tristan Van Berkom2019-01-241-1/+1
|/ | | | | | Don't require the interest to be registered, just discard any interest, this function just sets a bit in a mask, and is not intended to maintain a balance like the reserve() function is.
* Merge branch 'issue-638-validate-all-files' into 'master'Jürg Billeter2019-01-2432-45/+519
|\ | | | | | | | | Add support for default targets See merge request BuildStream/buildstream!925
| * NEWS: Add entry for default target featurePhillip Smyth2019-01-241-0/+4
| |
| * tests/frontend/buildcheckout.py: Add default target test with junctionJürg Billeter2019-01-244-0/+56
| | | | | | | | | | Test that `bst build` does not fail in a project where the list of default targets includes a junction (junctions cannot be built).
| * tests/frontend/pull.py: Add default target test for bst push/pullJürg Billeter2019-01-241-0/+50
| |
| * tests/frontend/fetch.py: Add default target test for bst source fetchJürg Billeter2019-01-241-0/+35
| |
| * tests/frontend: Add default target tests for bst show and buildPhillip Smyth2019-01-2419-0/+179
| |
| * format_project.rst: Add documentation for default targetsJürg Billeter2019-01-241-0/+38
| |
| * _frontend/cli.py: Ignore junctions in default targets where appropriateJürg Billeter2019-01-241-2/+17
| | | | | | | | | | | | | | Junctions cannot be built, pulled, or pushed. Specifying a junction on the command line for these commands will result in an error. However, junctions may be in the list of default targets, so they need to be ignored for build, pull, and push commands.
| * _stream.py: Add ignore_junction_targets parameterJürg Billeter2019-01-241-0/+15
| | | | | | | | This filters out junctions from the list of targets.
| * _project.py: Add get_default_target() and get_default_targets() methodsJürg Billeter2019-01-243-41/+115
| | | | | | | | | | | | _frontend/cli.py: Use new methods. Based on patches by Phillip Smyth.
| * projectconfig.yaml: Add key for default targetsPhillip Smyth2019-01-242-1/+9
| | | | | | | | _versions.py: Bump format version
| * _frontend/cli.py: Guess element also for bst build --allJürg Billeter2019-01-241-1/+1
|/ | | | There is no reason to disallow guess_element() for bst build --all.
* Merge branch 'jonathan/test-missing-workspace-guessing' into 'master'Jürg Billeter2019-01-241-25/+46
|\ | | | | | | | | Add tests to cover reinstated support for guessing targets See merge request BuildStream/buildstream!1042
| * tests: Test that fetching an open workspace will fetch its dependenciesJonathan Maw2019-01-241-21/+34
| | | | | | | | | | Previously, there was no way of detecting whether fetching happened, as an element with an open workspace will not be fetched.
| * Test that tracking in workspaces actually worksJonathan Maw2019-01-241-4/+12
|/ | | | | Previously, it merely tested that buildstream did not fall other, rather than whether it did anything useful.
* Merge branch 'tristan/track-test-reduce' into 'master'Tristan Van Berkom2019-01-241-4/+3
|\ | | | | | | | | test_track_error_cannot_write_file() fixup See merge request BuildStream/buildstream!1104
| * tests/frontend/track.py: test_track_error_cannot_write_file() fixupTristan Van Berkom2019-01-241-4/+3
|/ | | | | | | | | This tests how BuildStream reacts when it fails to write the tracking results to the element files or project.refs file, which is an operation that plugins do not play a part in. As such, removing the per repo kind parameterization from this test as multiple runs are redundant here.
* Merge branch 'tristan/test-element-states' into 'master'Tristan Van Berkom2019-01-248-68/+94
|\ | | | | | | | | Reduce number of calls to `bst show` in tests See merge request BuildStream/buildstream!1103
| * tests/frontend/workspace.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-12/+16
| |
| * tests/frontend/track.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-8/+12
| |
| * tests/frontend/push.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-6/+9
| |
| * tests/frontend/pull.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-4/+4
| |
| * tests/sources/remote.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-6/+13
| |
| * tests/elements/filter.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-4/+6
| |
| * tests/artifactcache/expiry.py: Refactored to use get_element_states()Tristan Van Berkom2019-01-241-23/+30
| |
| * tests/testutils/runcli.py: Make get_element_states() take a list of targetsTristan Van Berkom2019-01-242-5/+4
|/ | | | | Instead of a single target, we can always provide a single target in a list.
* Merge branch 'tristan/insufficient-storage-error' into 'master'Tristan Van Berkom2019-01-244-58/+60
|\ | | | | | | | | Tristan/insufficient storage error See merge request BuildStream/buildstream!1102
| * _artifactcache.py: Raise ArtifactError() when quota size exceeds disk space.tristan/insufficient-storage-errorTristan Van Berkom2019-01-233-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is not an error related to loading data, like a parse error in the quota specification is, but a problem raised by the artifact cache - this allows us to assert more specific machine readable errors in test cases (instead of checking the string in stderr, which this patch also fixes). This also removes a typo from the error message in the said error. * tests/artifactcache/cache_size.py Updated test case to expect the artifact error, which consequently changes the test case to properly assert a machine readable error instead of asserting text in the stderr (which is the real, secret motivation behind this patch). * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota() Now expect the artifact error for the tests which check configurations which create caches too large to fit on the disk.
| * tests: Migrated cache quota test into artifactcache/cache_size.pyTristan Van Berkom2019-01-232-34/+28
|/ | | | Instead of sitting mysteriously alone in internals/utils.py