From b25eacf0670c007d17942a0cf792651b373a4175 Mon Sep 17 00:00:00 2001 From: Sam Thursfield Date: Tue, 1 Mar 2016 11:42:41 +0000 Subject: Move deployment into a separate page Deployment is pretty unrelated to the definitions format, it's mostly about how to run programs in the extensions/ directory. So I think it makes sense to describe it in a separate page. This helps shorten the 'Definitions format' page, which is pretty long right now. Change-Id: I13684a3cc0bc8bdd7c9c043dd2a9ee2addac0d6a --- deployment.mdwn | 120 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 120 insertions(+) create mode 100644 deployment.mdwn (limited to 'deployment.mdwn') diff --git a/deployment.mdwn b/deployment.mdwn new file mode 100644 index 0000000..a7505fd --- /dev/null +++ b/deployment.mdwn @@ -0,0 +1,120 @@ +Deploying Baserock systems +========================== + +Building systems is quite a well-defined operation: source code goes in, a tree +of binary files comes out. + +Deployment is rather open-ended. Since a rootfs tarball isn't often useful +in itself, Baserock tools can follow the approach presented below to get a set +of built systems actually running somewhere. This might involve copying it to +a disk, creating an OpenStack instance or some other kind of virtual machine, +using the [Baserock upgrade +mechanism](http://wiki.baserock.org/guides/upgrades/) to upgrade an existing +deployment, or anything else. + +Within this document, consider "deployment" to be a process of first +post-processing a filesystem tree with one or more 'configure extensions', then +performing an operation to convert and/or transfer the filesystem tree using a +'write extension'. + +### Deployment sequence + +A Baserock deployment tool should, for each labelled deployment in the +'cluster', follow this sequence. + +1. Run the program '$type.check' (where $type is the value of the 'type' + field), if it exists. If it fails, exit now with an error. + + The .check extensions exist only to work around the fact that (2), (3) and + (4) may be slow, because programs that wait several minutes just to raise an + error that could have been detected right away are very annoying. + +2. Create a writeable temporary directory containing the contents of the system + artifact. + +3. For each 'subsystem' specified, recursively run steps 1, 3, 4 and 5. The + intention is that the result of the subsystem deployments end up *inside* + the temporary directory created in step 2 for the 'outer' deployment. + +4. For each path in the `configuration-extensions` field of the corresponding + system morphology, run `$path.configure`. The order of running these is, + sadly, unspecified. + +5. Run the .write extension for that deployment 'type'. + +The deployment of that system and its subsystems is now considered complete, +and temporary data can be removed. + +### Locating and running extensions + +The deployment extensions must live in the definitions repository. They can be +in a subdirectory, we recommend `extensions/`. + +> Previously extensions could and did live in the build tool's code. This was a +bad idea because we needed to define a new VERSION every time we changed +anything in the API of any deployment extension that the build tool provide, +and every new build tool that wanted to do deployment needed to reimplement or +duplicate every such extension. + +An extension must be executable using the [POSIX `exec()` system call]. We +encourage writing them as Python scripts and using a `#!/usr/bin/env python` +[hashbang], but any executable code is permitted. + +The 'execution environment' for an extension is sadly unspecified at the moment. +[Morph] runs extensions without any kid of chroot environment, but each one +is run in a separate [mount namespace]. Running extensions chrooted into the +system being deployed does not make sense, as it may not contain the right +dependencies for the extensions to work, and it may expect to run on a +different, incompatible architecture to the system running the deployment in +any case. + +A .check extension must be passed one commandline argument: the value of the +`location` or `upgrade-location` field. + +A .configure extension must be passed one commandline argument: the path to +the unpacked filesystem tree, which it can modify in some way. It is expected +that a .configure extension will do nothing unless it is enabled using a +configuration variable, but it is up to the code in the .configure extension +to do this, and this behaviour is not currently enforced. + +A .write extension must be passed two commandline arguments: the value of the +`location` or `upgrade-location` field, then the path to the unpacked +filesystem tree (which it could modify in some way). + +All key/value pairs from the 'cluster' definition for the given labelled +deployment of a system must be passed to all extensions, as environment +variables. The `type`, `location`, `upgrade-type` and `upgrade-location` fields +do not need to be passed in. + +Extensions are expected to set exit code 0 on success, and a non-zero value on +failure. If any extension returns non-zero, the deployment tool should abort +the deployment. + +Extensions are expected to write status information to 'stdout', and any error +messages to 'stderr'. This is for user feedback ONLY, deployment tools should +not do anything with this data other than log and/or display it. + +[Morph] sets an extra environment variable `MORPH_LOG_FD` which is a file +descriptor that the extension can write log messages to, in order for them to +appear in morph.log but not on stdout or stderr. + +[POSIX `exec()` system call]: http://pubs.opengroup.org/onlinepubs/009695399/functions/exec.html +[hashbang]: https://en.wikipedia.org/wiki/Shebang_%28Unix%29 +[mount namespace]: https://stackoverflow.com/questions/22889241/linux-understanding-the-mount-namespace-clone-clone-newns-flag#22889401 + +### Help for extensions + +The .configure and .write extensions may provide .help files as documentation. +The .help file should be valid [YAML], containing a dict with the key `help` +and a string value. + +For example, the `tar.write` extension could provide `tar.write.help` with the +following content: + + help: | + Deploy a system as a .tar file. + +### Common configuration parameters used by extensions + +Fill me in! + -- cgit v1.2.1