Add ElementSourcesCache

Sources have been cached in CAS individually, except for sources that
transform other sources, which have been cached combined with all
previous sources of the element. This caching structure may be confusing
as sources are specified in the element as a list and this is not a good
fit for #1274 where we want to support caching individual sources in a
Remote Asset server with a BuildStream-independent URI (especially the
`directory` configuration would be problematic).

This replaces the combined caching of 'previous' sources with an
element-level source cache, which caches all sources of an element
staged together. Sources that don't depend on previous sources are still
cached individually.

This also makes it possible to add a list of all element sources to the
source proto used by the element-level source cache.

1 files changed, 3 insertions, 0 deletions

diff --git a/src/buildstream/_stream.py b/src/buildstream/_stream.py
index bd203b0a0..fe641d03c 100644
--- a/src/buildstream/_stream.py
+++ b/src/buildstream/_stream.py
@@ -83,6 +83,7 @@ class Stream:
         #
         self._context = context
         self._artifacts = None
+        self._elementsourcescache = None
         self._sourcecache = None
         self._project = None
         self._pipeline = None
@@ -104,6 +105,7 @@ class Stream:
     #
     def init(self):
         self._artifacts = self._context.artifactcache
+        self._elementsourcescache = self._context.elementsourcescache
         self._sourcecache = self._context.sourcecache
 
     # cleanup()
@@ -1176,6 +1178,7 @@ class Stream:
 
         # Connect to remote caches, this needs to be done before resolving element state
         self._artifacts.setup_remotes(use_config=use_artifact_config, remote_url=artifact_url)
+        self._elementsourcescache.setup_remotes(use_config=use_source_config, remote_url=source_url)
         self._sourcecache.setup_remotes(use_config=use_source_config, remote_url=source_url)
 
     # _load_tracking()