From e858cd8d7a88a6ceb00ac9a9b7b2b6efeb388ef8 Mon Sep 17 00:00:00 2001 From: Sam Thursfield Date: Tue, 1 Mar 2016 13:18:41 +0000 Subject: Document the DEFAULTS file, and rewrite description of chunks Change-Id: Ie4e693fe37252860110b4d5ff0b7d172c7dae415 --- spec.mdwn | 226 ++++++++++++++++++++++++++++++++++++++------------------------ 1 file changed, 139 insertions(+), 87 deletions(-) diff --git a/spec.mdwn b/spec.mdwn index c7574d7..30366dc 100644 --- a/spec.mdwn +++ b/spec.mdwn @@ -62,6 +62,10 @@ uses. A tool should refuse to process a version that it doesn't support, to avoid unpredictable errors. See the "Versioning" heading above for more detail on versions. +The top directory of the repo can also contain a file named `DEFAULTS`. This +holds repo-wide 'build-system' and 'split-rules' information. See the +'Defaults' section below. + To find all the Baserock definition files in the repo, tooling can recursively scan the contents of the repo for files matching the glob pattern "\*.morph". @@ -101,82 +105,75 @@ The structure of a 'chunk' definition is described using [JSON-Schema] in the [spec.git] repo: -The fields mean the following: +The build sequence consists of four phases: + +1. configure +2. build +3. install +4. strip + +You can define one or more commands for each phase, or none. Here is an example +chunk: + + name: glibc + kind: chunk + description: GNU C library (example) + + configure-commands: + - mkdir o + - cd o && ../libc/configure --prefix=/usr + + build-commands: + - cd o && make + + install-commands: + - cd o && make install_root="$DESTDIR" install + +A chunk can optionally make use of a repo-wide build system defined in +'DEFAULTS', by using the `build-system` field. In this case, the +command sequences defined in DEFAULTS are used. + +Any predefined command sequence can be overridden by specifying a new value. +You can also extend a predefined build system with the `pre-` or `post-` +fields. For example, `pre-configure-commands` are run directly before +`configure-commands`, and all `post-configure-commands` are run directly after. + +For example: -* `build-system`: if the program is built using a build system known to - `morph`, you can set this field and avoid having to set the various - `*-commands` fields; the commands that the build system specifies can - be overridden; the following build-systems are known: - - - `autotools` - - `python-distutils` - - `cpan` - - `cmake` - - `qmake` - - optional - -* `pre-configure-commands`: a list of shell commands to run at - the configuration phase of a build, before the list in `configure-commands`; - optional -* `configure-commands`: a list of shell commands to run at the configuraiton - phase of a build; optional -* `post-configure-commands`: a list of shell commands to run at - the configuration phase of a build, after the list in `configure-commands`; - optional - -* `pre-build-commands`: a list of shell commands to run at - the build phase of a build, before the list in `build-commands`; - optional -* `build-commands`: a list of shell commands to run to build (compile) the - project; optional -* `post-build-commands`: a list of shell commands to run at - the build phase of a build, after the list in `build-commands`; - optional - -* `pre-test-commands`: a list of shell commands to run at - the test phase of a build, before the list in `test-commands`; - optional -* `test-commands`: a list of shell commands to run unit tests and other - non-interactive tests on the built but un-installed project; optional -* `post-test-commands`: a list of shell commands to run at - the test phase of a build, after the list in `test-commands`; - optional - -* `pre-install-commands`: a list of shell commands to run at - the install phase of a build, before the list in `install-commands`; - optional -* `install-commands`: a list of shell commands to ; optional -* `post-install-commands`: a list of shell commands to run at - the install phase of a build, after the list in `strip-commands`; - optional - -* `pre-strip-commands`: a list of shell commands to run at - the strip phase of a build, before the list in `strip-commands`; - optional -* `strip-commands`: a list of shell commands to strip debug symbols from binaries; - this should strip binaries in the directory named in the `DESTDIR` environment - variable, not the actual system; optional -* `post-strip-commands`: a list of shell commands to run at - the strip phase of a build, after the list in `install-commands`; - optional - -* `max-jobs`: a string to be given to `make` as the argument to the `-j` - option to specify the maximum number of parallel jobs; the only sensible - value is `"1"` (including the quotes), to prevent parallel jobs to run - at all; parallel jobs are only used during the `build-commands` phase, - since the other phases are often not safe when run in parallel; `morph` - picks a default value based on the number of CPUs on the host system; - optional - -* `chunks`: a key/value map of lists of regular expressions; - the key is the name - of a binary chunk, the regexps match the pathnames that will be - included in that chunk; the patterns match the pathnames that get installed - by `install-commands` (the whole path below `DESTDIR`); every file must - be matched by at least one pattern; by default, a single chunk gets - created, named according to the definition, and containing all files; - optional + name: git + kind: chunk + description: Git version control tool (example) + + build-system: autotools + + # This command will run before the normal 'configure' command sequence. + pre-configure-commands: + - make configure + + # This command overrides the normal 'build' command sequence. + build-commands: + - make all + +If a chunk doesn't need to override anything from 'DEFAULTS', you can avoid +having a chunk .morph file altogether, and just set 'build-system' when +referring to the chunk from the stratum. + +The `max-jobs` field can be used to pass a custom value for --jobs to Make, +via the `MAKEFLAGS` environment variable. This is useful for Makefiles which +don't work in parallel: you can set `max-jobs: 1` to work around the problem. +Parallel jobs are only used during the `build-commands` phase, since the other +phases are often not safe when run in parallel; `morph` picks a default value +based on the number of CPUs on the host system. + +Built chunks are split up into multiple artifacts. The default splitting rules +for a chunk are defined in 'DEFAULTS'. You can use the `chunks` field of a +chunk to override these. The `chunks` field is a key/value map of lists of +regular expressions; the key is the name of a binary chunk, the regexps match +the pathnames that will be included in that chunk; the patterns match the +pathnames that get installed by `install-commands` (the whole path below +`DESTDIR`); every file must be matched by at least one pattern; by default, a +single chunk gets created, named according to the definition, and containing +all files. #### Strata @@ -234,18 +231,6 @@ The fields mean the following: - `name`: name of the artifact when the stratum is build - `morph`: path to a stratum .morph file relative to the top of the containing repo -#### Example chunk (simplified commands): - - name: eglibc - kind: chunk - configure-commands: - - mkdir o - - cd o && ../libc/configure --prefix=/usr - build-commands: - - cd o && make - install-commands: - - cd o && make install_root="$DESTDIR" install - #### Example stratum: name: foundation @@ -387,6 +372,73 @@ There are two repo aliases that *must* be defined: Baserock tools should allow changing these values. The main benefit of this compact URI scheme is that definitions are not tied to a specific Git server, or protocol. You can build against a mirror of the original Git server, or change the protocol that is used, just by altering the repo-alias configuration. +### Defaults + +A definitions repository can contain a file named DEFAULTS. This file sets up +repo-wide configuration. + +The structure of the DEFAULTS file is described using [JSON-Schema] in +the [spec.git] repo: + + +#### Build systems + +You can define common command sequences using the 'build-systems' key. These +can then be referenced in chunk (and stratum) definitions, to avoid repeating +common patterns. + +Here is an example DEFAULTS file that defines the python-distutils build +system: + + build-systems: + python-distutils: + configure-commands: [] + build-commands: + - python setup.py build + install-commands: + - python setup.py install --prefix "$PREFIX" --root "$DESTDIR" + +#### Splitting rules + +One 'source' in Baserock can produce multiple binary 'artifacts'. This allows +you to separate programs, libraries, documentation, debugging information, and +whatever else into different artifacts. You can then choose to exclude some of +these artifacts from your final system. + +Split rules are defined separately for chunks, and for strata. + +Chunks define a list of artifacts, and a series of regular expression patterns +that are matched against filenames to define what to include. The list of +artifacts is evaluated *in order*, so if two artifacts match the same pattern, +those files will go into whichever artifact is defined first. + +Stratum matches work similarly, except the stratum artifacts are a series of +patterns that match the *names of chunk artifacts*. + +Here is an example DEFAULTS file that sets up some simple splitting rules. + + split-rules: + chunk: + - artifact: -devel + include: + - (usr/)?include/.* + - (usr/)?share/man/.* + - artifact: -runtime + include: + - .* + + stratum: + - artifact: -devel + include: + - .*-devel + - artifact: -runtime + include: + - .*-runtime + +You could use these rules to keep header files (/usr/include/\*) and manual +pages (/usr/share/man/\*) out of your systems, by only including the -runtime +stratum artifacts. + Build environment ----------------- -- cgit v1.2.1