diff options
Diffstat (limited to 'yarns/deployment.yarn')
| -rw-r--r-- | yarns/deployment.yarn | 330 |
1 files changed, 330 insertions, 0 deletions
diff --git a/yarns/deployment.yarn b/yarns/deployment.yarn new file mode 100644 index 00000000..0782c7c1 --- /dev/null +++ b/yarns/deployment.yarn @@ -0,0 +1,330 @@ +Morph Deployment Tests +====================== + + SCENARIO deploying a non-cluster morphology + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + AND the user attempts to deploy the system systems/test-system.morph in branch master + THEN morph failed + AND the deploy error message includes the string "morph deployment commands are only supported for cluster morphologies" + FINALLY the git server is shut down + + SCENARIO deploying a cluster morphology as a tarfile + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar + AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar + WHEN the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster test-cluster.morph in branch master + THEN morph succeeded + FINALLY the git server is shut down + +Some deployment types support upgrades, but some do not and Morph needs to make +this clear. + + SCENARIO attempting to upgrade a tarfile deployment + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar + AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar + WHEN the user builds the system systems/test-system.morph in branch master + AND the user attempts to upgrade the cluster test-cluster.morph in branch master + THEN morph failed + FINALLY the git server is shut down + +The rawdisk write extension supports both initial deployment and subsequent +upgrades. Note that the rawdisk upgrade code needs bringing up to date to use +the new Baserock OS version manager tool. Also, the test deploys an identical +base OS as an upgrade. While pointless, this is permitted and does exercise +the same code paths as a real upgrade. + + SCENARIO deploying a cluster morphology as rawdisk and then upgrading it + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: rawdisk + AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar + WHEN the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.DISK_SIZE=20M test-system.VERSION_LABEL=test1 + THEN morph succeeded + WHEN the user attempts to upgrade the cluster test-cluster.morph in branch master with options test-system.VERSION_LABEL=test2 + THEN morph succeeded + FINALLY the git server is shut down + +Nested deployments +================== + +For the use-cases of: + +1. Installer CD/USB +2. NFS/VM host +3. System with multiple containerised applications +4. System with a toolchain targetting the sysroot of another target +5. Any nested combination of the above + +It is convenient to be able to deploy one system inside another. + + SCENARIO deploying a cluster morphology with nested systems + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar + +After the usual setup, we also add a subsystem to the cluster. + + GIVEN a subsystem in cluster test-cluster.morph in branch master called test-system.sysroot + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment type: sysroot + +We specify the location as a file path, this is relative to the parent +system's extracted rootfs, before it is configured. + + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment location: var/lib/sysroots/test-system + WHEN the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" + THEN morph succeeded + +Morph succeeding alone is not sufficient to check whether it actually +worked, since if it ignored the subsystems field, or got the location +wrong for the subsystem. To actually test it, we have to check that our +deployed system contains the other. Since the baserock directory is in +every system, we can check for that. + + AND tarball test.tar contains var/lib/sysroots/test-system/baserock + FINALLY the git server is shut down + +Initramfs deployments +===================== + +There's a few ways of creating an initramfs. We could: +1. Build a sysroot and: + 1. Have a chunk turn that into a cpio archive, written into /boot. + 2. Embed it in the Linux kernel image, having the initramfs as part + of the BSP. +2. Deploy an existing system as a cpio archive + 1. As a stand-alone system, without a rootfs + 2. Nested inside another system + +1.1 and 1.2 require system engineering work, so won't be mentioned here. + + SCENARIO deploying a system with an initramfs + ASSUMING there is space for 5 512M disk images + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + GIVEN a cluster called C.morph in system branch master + AND a system in cluster C.morph in branch master called S + +2.2 needs a nested system that is deployed with the initramfs write +extension. + + GIVEN a subsystem in cluster C.morph in branch master called S.I + AND subsystem S.I in cluster C.morph in branch master builds systems/test-system.morph + AND subsystem S.I in cluster C.morph in branch master has deployment type: initramfs + +The nested system needs to be placed somewhere in the parent. The +traditional place for an initramfs is `/boot`. + + AND subsystem S.I in cluster C.morph in branch master has deployment location: boot/initramfs.gz + +1.1 and 2.2 need the write extension to configure the boot-loader to +use the produced initramfs. Only write extensions that involve creating a disk image care, so we'll use `rawdisk.write`. + + GIVEN system S in cluster C.morph in branch master builds systems/test-system.morph + AND system S in cluster C.morph in branch master has deployment type: rawdisk + AND system S in cluster C.morph in branch master has deployment location: test.img + AND system S in cluster C.morph in branch master has deployment variable: DISK_SIZE=512M + +Initramfs support is triggered by the `INITRAMFS_PATH` variable. It could have been made automatic, triggering the behaviour if `/boot/initramfs.gz` exists, but: + +1. There are a bunch of possible names, some of which imply different formats. +2. If we decide on one specific name, how do we pick. +3. If we allow multiple possible names, how do we handle multiple being possible. +4. We may need to pick a non-standard name: e.g. We have a deployment + where the system loads a kernel and initramfs from a disk, then boots + the target in KVM, so the bootloader we want to use for the guest is + `initramfs.gz`, while the host's initramfs is `hyp-initramfs.gz`. +5. We may have the initramfs come from a chunk the system built, but + for speed, we want this particular deployment not to use an initramfs, + even though we have a generic image that may support one. + +For all these reasons, despite there being redundancy in some cases, +we're going to set `INITRAMFS_PATH` to the same as the nested deployment's +location. + + GIVEN system S in cluster C.morph in branch master has deployment variable: INITRAMFS_PATH=boot/initramfs.gz + +Fully testing that the system is bootable requires a lot more time, +infrastructure and dependencies, so we're just going to build it and +inspect the result of the deployment. + + WHEN the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster C.morph in branch master + THEN morph succeeded + AND file workspace/master/test/morphs/test.img exists + +If the initramfs write extension works, the rootfs image should contain +`boot/initramfs.gz`. + + WHEN disk image workspace/master/test/morphs/test.img is mounted at mnt + THEN file mnt/systems/default/run/boot/initramfs.gz exists + +If the `rawdisk` write extension worked, then the bootloader config file +will mention the initramfs, and the UUID of the disk. + + AND file mnt/extlinux.conf matches initramfs + AND file mnt/extlinux.conf matches root=UUID= + FINALLY mnt is unmounted + AND the git server is shut down + +Partial deployments +=================== + +Deploy part of a cluster +------------------------ + +Starting from the well-defined position of having a cluster morphology +with only one definition. + + SCENARIO partially deploying a cluster morphology + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + AND the user builds the system systems/test-system.morph in branch master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar + AND system test-system in cluster test-cluster.morph in branch master has deployment location: test-system.tar + +It is useful to group related deployments together, so we support adding +another deployment to the same cluster morphology. + + GIVEN a system in cluster test-cluster.morph in branch master called second-system + AND system second-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system second-system in cluster test-cluster.morph in branch master has deployment type: tar + AND system second-system in cluster test-cluster.morph in branch master has deployment location: second-system.tar + +When we don't tell `morph deploy` which system we want to deploy, all +of the systems in the cluster are deployed. Here a successful deployment +will have morph exit sucessfully and in the case of tarball deployments, +the tarballs for both the systems will be created. + + WHEN the user attempts to deploy the cluster test-cluster.morph in branch master + THEN morph succeeded + AND file workspace/master/test/morphs/test-system.tar exists + AND file workspace/master/test/morphs/second-system.tar exists + +However, we don't need to deploy every system defined in a cluster at +once. This is useful for cases such as having a cluster morphology for +deploying a whole distbuild network, and re-deploying only nodes that +have failed. + + GIVEN the files workspace/master/test/morphs/test-system.tar and workspace/master/test/morphs/second-system.tar are removed + WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master + +A successful deployment will have morph exit successfully, and in the +case of tarball deployments, only the tarball for the system we asked +for will be created. + + THEN morph succeeded + AND file workspace/master/test/morphs/test-system.tar exists + AND file workspace/master/test/morphs/second-system.tar does not exist + +Cluster morphs can contain "nested systems", i.e. systems which have +subsystems to deploy as part of them. + +We need to add a subsystem to the cluster to test this. + + GIVEN a subsystem in cluster test-cluster.morph in branch master called test-system.sysroot + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment type: sysroot + +We specify the location as a file path, this is relative to the parent +system's extracted rootfs, before it is configured. + + AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment location: var/lib/sysroots/test-system + +The system which contains a nested system is deployed the same as +before, we don't need to mention the nested deployment. + + AND the file workspace/master/test/morphs/test-system.tar is removed + WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master + THEN morph succeeded + AND file workspace/master/test/morphs/test-system.tar exists + AND tarball workspace/master/test/morphs/test-system.tar contains var/lib/sysroots/test-system/baserock + +Morph will abort deployment if the system to deploy that is specified +on the command line is not defined in the morphology. + + WHEN the user attempts to deploy not-a-system from cluster test-cluster.morph in branch master + THEN morph failed + +It is not valid to deploy a nested system on its own. If it becomes +desirable to deploy a system that is identical to a system that already +exists but is nested in another, it should be redefined as a top-level +deployment. + + WHEN the user attempts to deploy test-system.sysroot from cluster test-cluster.morph in branch master + THEN morph failed + FINALLY the git server is shut down + +Deploying branch-from-image produced systems +============================================ + +We have this nifty subcommand called branch-from-image, which can be +used to build the same thing as an existing image. + +There's no special requirements for making the image reproducible. + + SCENARIO reproducing systems + GIVEN a workspace + AND a git server + WHEN the user checks out the system branch called master + AND the user builds the system systems/test-system.morph in branch master + GIVEN a cluster called test-cluster.morph in system branch master + AND a system in cluster test-cluster.morph in branch master called test-system + AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch master has deployment type: sysroot + AND system test-system in cluster test-cluster.morph in branch master has deployment location: test-system + WHEN the user attempts to deploy the cluster test-cluster.morph in branch master + THEN morph succeeded + AND file workspace/master/test/morphs/test-system exists + +To reproduce an existing image, do a checkout with the extracted root +filesystem's /baserock directory as the `--metadata-dir` argument. + + WHEN the user attempts to check out the system branch from workspace/master/test/morphs/test-system called mybranch + THEN morph succeeded + AND the system branch mybranch is checked out + +After it is checked-out, the system can be rebuilt. + + WHEN the user attempts to build the system systems/test-system.morph in branch mybranch + THEN morph succeeded + +Once it is rebuilt, it can be deployed. + + GIVEN a cluster called test-cluster.morph in system branch mybranch + AND a system in cluster test-cluster.morph in branch mybranch called test-system + AND system test-system in cluster test-cluster.morph in branch mybranch builds systems/test-system.morph + AND system test-system in cluster test-cluster.morph in branch mybranch has deployment type: tar + AND system test-system in cluster test-cluster.morph in branch mybranch has deployment location: test-system.tar + WHEN the user attempts to deploy the cluster test-cluster.morph in branch mybranch + THEN morph succeeded + AND file workspace/mybranch/test/morphs/test-system.tar exists |