summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSam Thursfield <sam.thursfield@codethink.co.uk>2017-05-23 15:00:23 +0100
committerSam Thursfield <sam.thursfield@codethink.co.uk>2017-08-24 13:11:39 +0000
commit1d45bf651f58fce1ddb27ec7db77f81d429b4bf5 (patch)
treee2dc0becec1f3ef959c5a11f83ec68e39b9b9a55
parent820352ca1e47256b4bbc9f124bfe1135607a05a3 (diff)
downloaddefinitions-1d45bf651f58fce1ddb27ec7db77f81d429b4bf5.tar.gz
Add 'convert' script and update README
-rw-r--r--README66
-rwxr-xr-xconvert65
2 files changed, 95 insertions, 36 deletions
diff --git a/README b/README
index ec38f9df..4cacb647 100644
--- a/README
+++ b/README
@@ -8,52 +8,46 @@ These are some example definitions for use with Baserock tooling. You can fork
this repo and develop your own systems directly within it, or use it as a
reference point when developing your own set of definitions.
-These definitions follow the Baserock definitions format, which is defined in
-spec.git repository (http://git.baserock.org/cgit/baserock/baserock/spec.git).
-
-The spec is readable online at <http://docs.baserock.org/spec>.
-
-The systems listed in the systems/ directory are example systems
+The systems listed in the elements/systems/ directory are example systems
that build and run at some point. The only ones we can be sure
that still build in current master of definitions are the ones that
we keep building in our ci system; they are listed in
http://git.baserock.org/cgit/baserock/baserock/definitions.git/tree/clusters/ci.morph
-Keeping up to date
-------------------
+BuildStream conversions
+-----------------------
+
+It is possible to use BuildStream to build Baserock definitions. The
+BuildStream build tool uses a different definitions format, so a conversion
+needs to be done first.
+
+Run the `convert` script from the root of the repository and you will be
+rewarded with an autogenerated set of .bst files in the elements/ subdirectory
+which should be buildable using BuildStream.
-The Baserock definitions format is evolving. A set of automated migrations is
-provided in the migrations/ directory of spec.git, for use when the format has
-changed and you want to bring your definitions up to date.
+To run `convert`, you will need defs2bst and ybd. The following commands, run
+from the root of the repository, should be enough to do a conversion:
-Before running the migrations, you can use the 'migrations/indent' tool to
-format the definitions in the specific style that the migrations expect.
-The migrations use the 'ruamel.yaml' Python library for editing the .morph
-files. This library preserves comments, ordering and some of the formatting
-when it rewrites a .morph file. However, it does impose a certain line width
-and indent style.
+ git clone https://gitlab.com/BuildStream/defs2bst/ ../defs2bst
+ git clone https://gitlab.com/baserock/ybd/ ../ybd
+ ./convert
-It makes a lot of sense to run the migrations with a *clean Git working tree*,
-so you can clearly see what changes they made, and can then choose to either
-commit them, tweak them, or revert them with `git reset --hard` and write an
-angry email.
+You can then build e.g. a devel-system using BuildStream:
-The suggested workflow is to run this from within your definitions.git clone:
+ bst build systems/devel-system-content.bst
- git clone git://git.baserock.org/baserock/baserock/spec ../spec.git
+Some things are not supported by the BuildStream conversion tool, and will need
+to be manually dealt with.
- git status # ensure a clean Git tree
- ../spec/migrations/indent
- git diff # check for any spurious changes
- git commit -a -m "Fix formatting"
- ../spec/migrations/run-all
- git diff # check the results
- git commit -a -m "Migrate to version xx of Baserock definitions format"
+ * the build-essential stratum isn't automatically converted, the
+ elements/gnu-toolchain.bst element is its equivalent and needs to be
+ manually kept in sync with any changes
-If you are working in a fork of the Baserock definitions.git repo, you can
-also keep to date with using changes in 'master' using `git merge`. In general,
-we recommend first running the migrations, committing any changes they make,
-*then* merging in changes using `git merge`. This should minimise the number of
-merge conflicts, although merge conflicts are still possible.
+ * .configure and .write extensions need to be rewritten as BuildStream
+ elements
-See migrations/GUIDELINES for information on how to write new migrations.
+ * non-x86_64 systems aren't automatically converted; BuildStream supports
+ "arch conditionals" which should make it easier for a single tree of .bst
+ files to support many platforms, but it's not easy to automatically achieve
+ this in places like linux.bst where Baserock currently has many different
+ equivalents
diff --git a/convert b/convert
new file mode 100755
index 00000000..32ea03c7
--- /dev/null
+++ b/convert
@@ -0,0 +1,65 @@
+#!/usr/bin/bash
+# Baserock definitions -> BuildStream conversion script.
+#
+# This script aims to be the "canonical" conversion of the Baserock definitions
+# repository for use with the BuildStream build tool.
+
+set -eu
+
+# This must point to a clone of https://gitlab.com/BuildStream/defs2bst/
+CONVERTER=../defs2bst/defs2bst.py
+
+# This must point to a clone of https://gitlab.com/baserock/ybd/
+YBD=../ybd/ybd.py
+
+# Only the following systems will be converted.
+systems="
+ systems/base-system-x86_64-generic.morph
+ systems/build-system-x86_64.morph
+ systems/devel-system-x86_64-generic.morph
+ systems/minimal-system-x86_64-generic.morph
+ gnome/systems/gnome-system-x86_64.morph
+ genivi/systems/genivi-demo-platform-x86_64-generic.morph
+ ivi/systems/ivi-system-x86_64.morph
+ trove/systems/trove-system-x86_64.morph
+ weston/systems/weston-system-x86_64-generic.morph
+ weston/systems/weston-qt5-system-x86_64.morph
+ unmaintained/systems/openstack-system-x86_64.morph"
+
+mkdir -p elements/
+mkdir -p elements/systems/
+
+for system_morph in $systems; do
+ # Run the defs2bst script for $system_morph. This will produce a .bst
+ # file for every chunk involved in the build, and will produce a file named
+ # something like XX-system-x86_64-content.bst from the system .morph file.
+ $CONVERTER --ybd $YBD \
+ --definitions ./ \
+ --output ./elements \
+ --rebase strata/build-essential.morph gnu-toolchain.bst \
+ $system_morph x86_64
+
+ # Figure out the .bst file defs2bst will have produced for $system_morph.
+ system_bst_initial=$(echo $system_morph | sed -e 's@systems/@@' -e 's@\.morph$@-content.bst@')
+
+ # Now we remove x86_64 from the name; BuildStream allows architecture
+ # conditionals so we don't need to duplicate everything per platform.
+ #
+ # We take the opportunity to remove -generic as well if present.
+ system_bst_final=$(basename $system_bst_initial | sed -e 's@-x86_64@@' -e 's@-generic@@')
+
+ # All systems go in an elements/systems/ subdirectory, rather than being
+ # spread around different dirs like they are currently.
+ mv elements/$system_bst_initial elements/systems/$system_bst_final
+
+ echo
+ echo "convert: Created elements/systems/$system_bst_final from elements/$system_bst_initial"
+ echo
+done
+
+# Manual fix for libffi build instructions. The output of $(gcc -dumpmachine)
+# is not the same as the output of $(sh ./config.guess) when building with
+# the BuildStream gnu-toolchain definitions.
+#
+# We cannot fix this in definitions.git master, see: https://gitlab.com/baserock/definitions/merge_requests/49
+sed -i elements/core/libffi.bst -e 's@\$(gcc -dumpmachine)@$(sh ./config.guess)@'