From 4e5cfa4f2a8343e0f68ac22ab393d9bdc9fdea82 Mon Sep 17 00:00:00 2001 From: Richard Maw Date: Wed, 29 Feb 2012 11:52:54 +0000 Subject: README: amend some obsolete info, add bootstrap Amend the config portion, some parts now require a remote url, new sections need to be added. Nearly all of it is now optional, make that known in the README. Quote shell variables in some example morphs. Add a section of my understanding of the bootstrap process. --- README | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 98 insertions(+), 6 deletions(-) (limited to 'README') diff --git a/README b/README index 2c717dc4..5aefaf71 100644 --- a/README +++ b/README @@ -40,9 +40,11 @@ something like this: cachedir = /home/username/baserock/cache log = /home/username/baserock/morph.log log-max = 200M - git-base-url = file:///home/username/baserock/gits/ + git-base-url = git://gitorious.org/baserock-morphs + bundle-server = http://roadtrain.codethink.co.uk/bundles -You should set at least the above settings. +All of the above settings apart from `log` are the defaults, so may be +omitted. Morphology file syntax @@ -54,7 +56,7 @@ JSON is used for the morphology syntax. For example, to build a chunk: "name": "foo", "kind": "chunk", "configure-commands": [ - "./configure --prefix=$PREFIX" + "./configure --prefix=\"$PREFIX\"" ], "build-commands": [ "make" @@ -63,7 +65,7 @@ JSON is used for the morphology syntax. For example, to build a chunk: "make check" ], "install-commands": [ - "make DESTDIR=$DESTDIR install" + "make DESTDIR=\"$DESTDIR\" install" ] } @@ -77,8 +79,8 @@ For chunks, use the following fields: * `build-system`: if the program is built using a build system known to `morph`, you can set this field and avoid having to set the various - `*-commands` fields; only `autotools` is currently known and that - requires the `configure` script to have been committed into git; + `*-commands` fields; only `autotools` is currently known; the commands + that the build system specifies can be overridden; optional * `configure-commands`: a list of shell commands to run at the configuraiton phase of a build; optional @@ -258,6 +260,96 @@ and check out the `cmdtest` utility (from ). Run the checks before submitting a patch, please. +The Bootstrap Process +--------------------- + +The goal of the bootstrap process is to create an environment that has +only been built with morph. It is a long boring process that can take +a few hours. + +It can be started by running `./run-bootstrap-in-chroot $workingdir` +with some environment variables set. + +The variables are: +* `DEBIAN_MIRROR`; which must be a path suitable for passing to + debootstrap; which at the time of writing allowed http://, + file:// or ssh:// urls; **required** +* `GIT_BUNDLES`; which must be a url to a bundles directory; + morph will try to fetch bundles from here before trying the + git-base-url; **required** +* `LFS_MIRROR`; which must point to a directory containing all + the tarballs mentioned in `wget-list`; **required** +* `snapshot`; which should be `false` (or any executable that + exits unsuccessfully) if creating a snapshot after each stage + of the bootstrap is not desirable; optional + +If snapshot is true or omitted then after each stage of the bootstrap +a tarball of the working directory is created, so that if a later stage +fails it can start from the last successful stage. + +The mirror variables allow the bootstrap to be run more quickly if the +required files are available locally. + +The mirror variables being required and snapshotting defaulting +to true are artefacts of the bootstrap process' development, where +only re-running the step that failed and using local mirrors were +essential for making fixes. + + +Bootstrap stages +---------------- + +The stages required for the bootstrap are to create a debian squeeze +chroot; build a skeleton system using the linux from scratch specified +tarballs; chroot into this system and build a system using morph with +bootstrap mode; then build the system again but instead using morph's +--staging-chroot mode and with the linux from scratch system removed. + +The squeeze chroot (stage 0) is used so that the bootstrap process does +not need to be supported on every available host system, but every host +system that can run debootstrap can be indirectly supported. + +In this stage debootstrap creates a chroot with all the packages required +to build the linux from scratch system, from then onwards every stage is +built by running baserock-bootstrap and optionally making a snapshot. + +Stage 1 builds everything in a subdirectory of the squeeze-chroot called +tree, and all the packages are built with --prefix=/tools, so the full +path for `cat` would be `$workingdir/squeeze-chroot/tree/tools/bin/cat`. + +This is done so that we have a known good system to work from, that won't +become entangled with the final system, which installed with --prefix=/usr. + +Stage 2 uses morph to build a system, while chrooted into +`$workingdir/squeeze-chroot/tree`, this naturally required PATH to include +/tools/bin. + +Morph is used to build the `foundation.morph` and `devel-bootstrap.morph` +strata in the `morphs` repository. `devel-bootstrap.morph` may not use the +same morphs as `devel.morph` because the bootstrap requires some hacks that +aren't required when a fully bootstrapped system is available. + +Morph also uses the `--bootstrap` flag to make the chunks be unpacked after +they have been built, this is required for build-dependencies to be met as +prerequisites are expected to be available on the system. + +After morph has finished a reasonably complete system should be available, +so `$workingdir/squeeze-chroot/tree/tools` is removed to save space and +make it obvious if stage 2 is missing anything needed for stage 3. + +Stage 3 builds `devel.morph` which does not need all the hacks required +in building during the bootstrap, so it should be much more like what +morph will finally build. + +Rather than using `--bootstrap`, stage 3 uses `--staging-chroot`, which +builds and extracts into a chroot, rather than using anything on the host +system. + +For this to be able to build anything `--staging-filler` is used, which +allows for specifying the environment that chunks are built in. Stage 3 +uses the strata that were build in stage 2. + + Legalese -------- -- cgit v1.2.1