From 208250cb59581579299bf353e502d065ac386064 Mon Sep 17 00:00:00 2001 From: Tristan Van Berkom Date: Fri, 16 Mar 2018 17:39:06 +0900 Subject: Documenting project.refs o Adding explanation of the `ref-storage` option in project.conf o Adjusting some adjacent relevant documentation o Adding a new projectrefs.rst section for the project.refs format itself, and linking it from the main authoring page. --- doc/source/formatintro.rst | 4 ++- doc/source/main_authoring.rst | 13 ++++++-- doc/source/projectconf.rst | 74 ++++++++++++++++++++++++++++++++----------- doc/source/projectrefs.rst | 73 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 143 insertions(+), 21 deletions(-) create mode 100644 doc/source/projectrefs.rst diff --git a/doc/source/formatintro.rst b/doc/source/formatintro.rst index a1f4e8f26..f3e7a2874 100644 --- a/doc/source/formatintro.rst +++ b/doc/source/formatintro.rst @@ -28,11 +28,13 @@ A BuildStream project is a directory consisting of: * A project configuration file * BuildStream element files -* User defined Plugins +* Optional user defined plugins +* An optional project.refs file A typical project structure may look like this:: myproject/project.conf + myproject/project.refs myproject/elements/element1.bst myproject/elements/element2.bst myproject/elements/... diff --git a/doc/source/main_authoring.rst b/doc/source/main_authoring.rst index f23bfb7c1..14d051276 100644 --- a/doc/source/main_authoring.rst +++ b/doc/source/main_authoring.rst @@ -34,14 +34,23 @@ Project format * :ref:`public` + * :ref:`public_integration` + * :ref:`public_split_rules` + * :ref:`public_overlap_whitelist` + +* :ref:`projectrefs` + + * :ref:`projectrefs_basics` + * :ref:`projectrefs_format` + Plugins ------- Plugins provide their own individual plugin specific YAML configurations, The element ``.bst`` files can specify plugin specific configuration in the :ref:`config section `, while sources declared on a -given element specify their plugin specific configuration -:ref:`directly on their main dictionary `. +given element specify their plugin specific configuration directly +:ref:`in their source declarations `. Elements diff --git a/doc/source/projectconf.rst b/doc/source/projectconf.rst index 0db06fad8..248f140ab 100644 --- a/doc/source/projectconf.rst +++ b/doc/source/projectconf.rst @@ -21,6 +21,8 @@ Essentials ---------- +.. _project_format_name: + Project Name ~~~~~~~~~~~~ The first thing to setup in your ``project.conf`` should be the name @@ -74,6 +76,60 @@ Note that elements are referred to by their relative paths, whenever elements are referred to in a ``.bst`` file or on the command line. +.. _project_format_ref_storage: + +Ref Storage +~~~~~~~~~~~ +By default, BuildStream expects to read and write source references +directly in the :ref:`source declaration `, but this +can be inconvenient and prohibitive in some workflows. + +Alternatively, BuildStream allows source references to be stored +centrally in a :ref:`project.refs file ` in the toplevel +:ref:`project directory `. + +This can be controlled with the ``ref-storage`` option, which is +allowed to be configured with the following values: + +* ``inline`` + + Source references are stored directly in the + :ref:`source declaration ` + +* ``project.refs`` + + Source references are stored in the ``project.refs`` file. + +To enable storing of source references in ``project.refs``, add the +following to your ``project.conf``: + +.. code:: yaml + + ref-storage: project.refs + +.. note:: + + The ``ref-storage`` configuration is available since :ref:`format version 5 ` + + +Fail on Overlaps +~~~~~~~~~~~~~~~~ +When multiple elements are staged, there's a possibility that different +elements will try and stage different versions of the same file. + +When ``fail-on-overlap`` is true, if an overlap is detected +that hasn't been allowed by the element's +:ref:`overlap whitelist`, +then an error will be raised and the build will fail. + +otherwise, a warning will be raised indicating which files had overlaps, +and the order that the elements were overlapped. + +.. code:: yaml + + fail-on-overlap: true + + Source Aliases ~~~~~~~~~~~~~~ In order to abstract the download location of source code and @@ -108,24 +164,6 @@ You can also specify a list of caches here; earlier entries in the list will have higher priority than later ones. -Fail on Overlaps -~~~~~~~~~~~~~~~~ -When multiple elements are staged, there's a possibility that different -elements will try and stage different versions of the same file. - -When ``fail-on-overlap`` is true, if an overlap is detected -that hasn't been allowed by the element's -:ref:`overlap whitelist`, -then an error will be raised and the build will fail. - -otherwise, a warning will be raised indicating which files had overlaps, -and the order that the elements were overlapped. - -.. code:: yaml - - fail-on-overlap: true - - .. _project_plugins: External Plugins diff --git a/doc/source/projectrefs.rst b/doc/source/projectrefs.rst new file mode 100644 index 000000000..557f8c5f3 --- /dev/null +++ b/doc/source/projectrefs.rst @@ -0,0 +1,73 @@ +:orphan: + + +.. _projectrefs: + +The project.refs File +===================== +If one has elected to store source references in a single ``project.refs`` +file, then it will be stored at the toplevel of your project directory +adjacent to ``project.conf``. This can be configured in your project +using the :ref:`ref-storage configuration ` + + +.. _projectrefs_basics: + +Basic Behavior +-------------- +When a ``project.refs`` file is in use, any source references found +in the :ref:`inline source declarations ` are considered +invalid and will be ignored, and a warning will be emitted for them. + +When ``bst track`` is run for your project, the ``project.refs`` file +will be updated instead of the inline source declarations. In the absence +of a ``project.refs`` file, ``bst track`` will create one automatically +with the tracking results. + +An interesting property of ``project.refs`` is that it allows for +*cross junction tracking*. This is to say that it is possible to override +the *ref* of a given source in a project that your project depends on via +a :mod:`junction `, without actually modifying the +junctioned project. + + +.. _projectrefs_format: + +Format +------ +The ``project.refs`` uses the same YAML format used throughout BuildStream, +and supports the same :ref:`directives ` which apply to +``project.conf`` and element declaration files (i.e. *element.bst* files). + +The ``project.refs`` file format itself is very simple, it contains a single ``projects`` +key at the toplevel, which is a dictionary of :ref:`project names `. +Each *project name* is a dictionary of *element names*, and each *element name* holds +a list of dictionaries corresponding to the element's :ref:`sources `. + + +**Example** + +.. code:: yaml + + # The local project's name is "core" + core: + + # A dictionary of element names + base/automake.bst: + + # A list of sources corresponding to the element + # in the same order in which they were declared. + # + # The values of this list are dictionaries of the + # symbolic "ref" portion understood by the given + # source plugin implementation. + # + - ref: af6ba39142220687c500f79b4aa2f181d9b24e4... + + # The "core" project depends on the "bootstrap" project, + # here we are allowed to override the refs for the projects + # we depend on through junctions. + bootstrap: + + zlib.bst: + - ref: 4ff941449631ace0d4d203e3483be9dbc9da4540... -- cgit v1.2.1