<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/buildstream.git/buildstream/_artifactcache.py, branch tpollard/workspacebuildtree</title>
<subtitle>gitlab.com: buildstream/buildstream.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/'/>
<entry>
<title>WIP: Opening a workspace with a cached build</title>
<updated>2019-02-12T12:13:57+00:00</updated>
<author>
<name>Tom Pollard</name>
<email>tom.pollard@codethink.co.uk</email>
</author>
<published>2018-10-01T09:56:09+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=05fc1bf443a91cd2515dcfb4df80864fa8060e66'/>
<id>05fc1bf443a91cd2515dcfb4df80864fa8060e66</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>_project.py: use artifact caches from the parent project for junctions</title>
<updated>2019-02-08T16:42:23+00:00</updated>
<author>
<name>Abderrahim Kitouni</name>
<email>akitouni@gnome.org</email>
</author>
<published>2019-01-28T10:01:08+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=24c0de16faec2b8b9bd6a03504ce951dc49afbe2'/>
<id>24c0de16faec2b8b9bd6a03504ce951dc49afbe2</id>
<content type='text'>
This makes a junction use the artifact cache of the parent project
before the ones defined for the junction

Fixes #401
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This makes a junction use the artifact cache of the parent project
before the ones defined for the junction

Fixes #401
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: don't leak the project specific remote caches</title>
<updated>2019-02-08T16:42:23+00:00</updated>
<author>
<name>Abderrahim Kitouni</name>
<email>akitouni@gnome.org</email>
</author>
<published>2019-01-27T10:43:18+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=1ee4a4ba3a91e27515967d5e2daaedf664f10c78'/>
<id>1ee4a4ba3a91e27515967d5e2daaedf664f10c78</id>
<content type='text'>
the code for initializing remotes added the project specific remote
caches to the global list instead of making a copy.

Fixes #618
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
the code for initializing remotes added the project specific remote
caches to the global list instead of making a copy.

Fixes #618
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Don't require the quota to be available on disk.</title>
<updated>2019-01-25T18:35:36+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-24T21:02:34+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=22b3a0c182916b8eacb666aa7bc1ea829cd92ca4'/>
<id>22b3a0c182916b8eacb666aa7bc1ea829cd92ca4</id>
<content type='text'>
Instead only rely on the headroom to be enough to protect against
out of space conditions. The headroom can become configurable as
a separate step is required.

The changes to achieve this are:

  * Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().

  * ArtifactCache.full() now also reports True if the available
    space on the artifact cache volume is smaller than the headroom.

    This ensures jobs get triggered to cleanup the cache when
    reaching the end of the disk.

  * When loading the artifact quota, it is now only an error if
    the quota exceeds the overall disk space, not if it does not
    fit in the available space.

    It is still a warning if the quota does not fit in the
    available space on the artifact cache volume.

  * Updated scheduler.py and buildqueue.py for the API rename

  * tests: Updated the artifactcache/expiry.py test for its
    expectations in this regard.

    Added a new test to test an error when quota was specified to
    exceed total disk space, and adjusted the existing tests to
    expect a warning when the quota does not fit in the available
    space.

This fixes issue #733 and #869.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Instead only rely on the headroom to be enough to protect against
out of space conditions. The headroom can become configurable as
a separate step is required.

The changes to achieve this are:

  * Rename ArtifactCache.has_quota_exceeded() to ArtifactCache.full().

  * ArtifactCache.full() now also reports True if the available
    space on the artifact cache volume is smaller than the headroom.

    This ensures jobs get triggered to cleanup the cache when
    reaching the end of the disk.

  * When loading the artifact quota, it is now only an error if
    the quota exceeds the overall disk space, not if it does not
    fit in the available space.

    It is still a warning if the quota does not fit in the
    available space on the artifact cache volume.

  * Updated scheduler.py and buildqueue.py for the API rename

  * tests: Updated the artifactcache/expiry.py test for its
    expectations in this regard.

    Added a new test to test an error when quota was specified to
    exceed total disk space, and adjusted the existing tests to
    expect a warning when the quota does not fit in the available
    space.

This fixes issue #733 and #869.
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Correcting API documenting comment for remove()</title>
<updated>2019-01-24T18:02:08+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-23T14:31:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=9c33107f48948523737ab3419a08d0fbe5b9474e'/>
<id>9c33107f48948523737ab3419a08d0fbe5b9474e</id>
<content type='text'>
This seems to have been copy/pasted from cascache, and
documents the function to possibly return None if defer_prune
was specified, but this function does not expose defer_prune.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This seems to have been copy/pasted from cascache, and
documents the function to possibly return None if defer_prune
was specified, but this function does not expose defer_prune.
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Added client progress callback to ArtifactCache.clean()</title>
<updated>2019-01-24T18:02:08+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-22T16:57:06+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=49c11bc8398f79c8a97806db75978c0c425f5244'/>
<id>49c11bc8398f79c8a97806db75978c0c425f5244</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Add status messages in cache management operations</title>
<updated>2019-01-24T18:02:08+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-21T19:22:06+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=8074ebf4a704cacfe344d4df58076dd144bef636'/>
<id>8074ebf4a704cacfe344d4df58076dd144bef636</id>
<content type='text'>
Added some useful status messages when:

  * Calculating a new artifact cache usage size

  * Starting a cleanup

  * Finishing a cleanup

Also enhanced messaging about what was cleaned up so far when
aborting a cleanup.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Added some useful status messages when:

  * Calculating a new artifact cache usage size

  * Starting a cleanup

  * Finishing a cleanup

Also enhanced messaging about what was cleaned up so far when
aborting a cleanup.
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Added ArtifactCacheUsage()</title>
<updated>2019-01-24T18:02:08+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-20T19:36:51+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=7ee0c579d5c6c2e3f4a70f80e3414a03d3a18cca'/>
<id>7ee0c579d5c6c2e3f4a70f80e3414a03d3a18cca</id>
<content type='text'>
A simple object which creates a snapshot of current
usage statistics for easy reporting in the frontend.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A simple object which creates a snapshot of current
usage statistics for easy reporting in the frontend.
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Refactored to use utils._get_volume_size()</title>
<updated>2019-01-24T18:02:08+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-20T17:52:16+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=9fd9fbb99fe54fda7599c39c014727a157c9ee2a'/>
<id>9fd9fbb99fe54fda7599c39c014727a157c9ee2a</id>
<content type='text'>
This will benefit from a better UtilError being raised, and
and turns the artifact cache's local function into a one liner.

The loop which finds the first existing directory in the
given path has been removed, being meaningless due to the
call to os.makedirs() in ArtifactCache.__init__().

The local function was renamed to _get_cache_volume_size() and
no longer takes any arguments, which is more suitable for the
function as it serves as a testing override surface for
unittest.mock().

The following test cases which use the function to override
the ArtifactCache behavior have been updated to use the new
overridable function name:

   tests/artifactcache/cache_size.py
   tests/artifactcache/expiry.py
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This will benefit from a better UtilError being raised, and
and turns the artifact cache's local function into a one liner.

The loop which finds the first existing directory in the
given path has been removed, being meaningless due to the
call to os.makedirs() in ArtifactCache.__init__().

The local function was renamed to _get_cache_volume_size() and
no longer takes any arguments, which is more suitable for the
function as it serves as a testing override surface for
unittest.mock().

The following test cases which use the function to override
the ArtifactCache behavior have been updated to use the new
overridable function name:

   tests/artifactcache/cache_size.py
   tests/artifactcache/expiry.py
</pre>
</div>
</content>
</entry>
<entry>
<title>_artifactcache.py: Raise ArtifactError() when quota size exceeds disk space.</title>
<updated>2019-01-23T13:57:53+00:00</updated>
<author>
<name>Tristan Van Berkom</name>
<email>tristan.vanberkom@codethink.co.uk</email>
</author>
<published>2019-01-19T15:41:57+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/buildstream.git/commit/?id=9be316419923997932c2f176b592de4d18210604'/>
<id>9be316419923997932c2f176b592de4d18210604</id>
<content type='text'>
This is not an error related to loading data, like a parse error
in the quota specification is, but a problem raised by the artifact
cache - this allows us to assert more specific machine readable
errors in test cases (instead of checking the string in stderr, which
this patch also fixes).

This also removes a typo from the error message in the said error.

  * tests/artifactcache/cache_size.py

    Updated test case to expect the artifact error, which consequently
    changes the test case to properly assert a machine readable error
    instead of asserting text in the stderr (which is the real, secret
    motivation behind this patch).

  * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()

    Now expect the artifact error for the tests which check configurations
    which create caches too large to fit on the disk.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is not an error related to loading data, like a parse error
in the quota specification is, but a problem raised by the artifact
cache - this allows us to assert more specific machine readable
errors in test cases (instead of checking the string in stderr, which
this patch also fixes).

This also removes a typo from the error message in the said error.

  * tests/artifactcache/cache_size.py

    Updated test case to expect the artifact error, which consequently
    changes the test case to properly assert a machine readable error
    instead of asserting text in the stderr (which is the real, secret
    motivation behind this patch).

  * tests/artifactcache/expiry.py: Reworked test_invalid_cache_quota()

    Now expect the artifact error for the tests which check configurations
    which create caches too large to fit on the disk.
</pre>
</div>
</content>
</entry>
</feed>
