_loader: Adding LoadContexttristan/load-context

Instead of passing around many details though calling signatures
throughout the loader code, create a single LoadContext object
which holds any overall loading state along with any values which
are constant to a full load process.

Overall this patch does:

  * _frontend/app.py: No need to pass Stream.fetch_subprojects() along anymore

  * _loader/loadelement.pyx: collect_element_no_deps() no longer takes a task argument

  * _loader/loader.py: Now the Loader has a `load_context` member, and no more
    `_fetch_subprojects` member or `_context` members

    Further, `rewritable` and `ticker` is no longer passed along through all
    of the recursing calling signatures, and `ticker` itself is finally removed
    because this has been replaced a long time ago with `Task` API from `State`.

  * _pipeline.py: The load() function no longer has a `rewritable` parameter

  * _project.py: The Project() is responsible for creating the toplevel
    LoadContext() if one doesn't exist yet, and this is passed through
    to the Loader() (and also passed to the Project() constructor by the
    Loader() when instantiating subprojects).

  * _stream.py: The `Stream._fetch_subprojects()` is now private and set
    on the project when giving the Project to the Stream in `Stream.set_project()`,
    also the Stream() sets the `rewritable` state on the `LoadContext` at the
    earliest opportunity, as the `Stream()` is the one who decides this detail.

    Further, some double underscore private functions are now regular single
    underscores, there was no reason for this inconsistency.

  * tests/internals/loader.py: Updated for API change

1 files changed, 66 insertions, 0 deletions

diff --git a/src/buildstream/_loader/loadcontext.py b/src/buildstream/_loader/loadcontext.py
new file mode 100644
index 000000000..161be913b
--- /dev/null
+++ b/src/buildstream/_loader/loadcontext.py
@@ -0,0 +1,66 @@
+#
+#  Copyright (C) 2020 Codethink Limited
+#
+#  This program is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 2 of the License, or (at your option) any later version.
+#
+#  This library is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	 See the GNU
+#  Lesser General Public License for more details.
+#
+#  You should have received a copy of the GNU Lesser General Public
+#  License along with this library. If not, see <http://www.gnu.org/licenses/>.
+#
+#  Authors:
+#        Tristan Van Berkom <tristan.vanberkom@codethink.co.uk>
+
+
+# LoaderContext()
+#
+# An object to keep track of overall context during the load process.
+#
+# Args:
+#    context (Context): The invocation context
+#
+class LoadContext:
+    def __init__(self, context):
+
+        # Keep track of global context required throughout the recursive load
+        self.context = context
+        self.rewritable = False
+        self.fetch_subprojects = None
+        self.task = None
+
+    # set_rewritable()
+    #
+    # Sets whether the projects are to be loaded in a rewritable fashion,
+    # this is used for tracking and is slightly more expensive in load time.
+    #
+    # Args:
+    #   task (Task): The task to report progress on
+    #
+    def set_rewritable(self, rewritable):
+        self.rewritable = rewritable
+
+    # set_task()
+    #
+    # Sets the task for progress reporting.
+    #
+    # Args:
+    #   task (Task): The task to report progress on
+    #
+    def set_task(self, task):
+        self.task = task
+
+    # set_fetch_subprojects()
+    #
+    # Sets the task for progress reporting.
+    #
+    # Args:
+    #   task (callable): The callable for loading subprojects
+    #
+    def set_fetch_subprojects(self, fetch_subprojects):
+        self.fetch_subprojects = fetch_subprojects