From 36b312b50a272daac9b2906f7af85b26dcc11608 Mon Sep 17 00:00:00 2001 From: Richard Ipsum Date: Sat, 21 Nov 2015 14:36:02 +0000 Subject: yarns: drop use of workspace This should help clear the way for the eventual removal of deprecated workspace code and related commands: morph checkout etc Change-Id: I66298c4f9f790848962bd70ab434bef96cf96cb9 --- yarns/architecture.yarn | 15 ++- yarns/branches-workspaces.yarn | 24 ++--- yarns/building.yarn | 66 ++++++------- yarns/deployment.yarn | 158 ++++++++++--------------------- yarns/implementations.yarn | 210 ++++++++++++++++++----------------------- yarns/regression.yarn | 45 ++------- yarns/splitting.yarn | 17 ++-- 7 files changed, 208 insertions(+), 327 deletions(-) diff --git a/yarns/architecture.yarn b/yarns/architecture.yarn index b558e378..faa0f291 100644 --- a/yarns/architecture.yarn +++ b/yarns/architecture.yarn @@ -2,10 +2,9 @@ Morph Cross-Building Tests ========================== SCENARIO building a system for a different architecture - GIVEN a workspace - AND a git server + GIVEN a git server AND a system called base-system-testarch.morph for the test architecture in the git server - WHEN the user checks out the system branch called master + WHEN the user clones definitions AND the user attempts to build the system base-system-testarch.morph in branch master THEN morph failed AND the build error message includes the string "Are you trying to cross-build?" @@ -16,10 +15,9 @@ Morph Cross-Bootstrap Tests =========================== SCENARIO cross-bootstrapping a system for a different architecture - GIVEN a workspace - AND a git server + GIVEN a git server AND a system called base-system-testarch.morph for the test architecture in the git server - WHEN the user checks out the system branch called master + WHEN the user clones definitions THEN the user cross-bootstraps the system base-system-testarch.morph in branch master of repo test:morphs to the arch testarch FINALLY the git server is shut down @@ -27,10 +25,9 @@ Architecture validation Tests ============================= SCENARIO building a system with no architecture - GIVEN a workspace - AND a git server + GIVEN a git server AND a system called base-system-noarch.morph with no architecture in the git server - WHEN the user checks out the system branch called master + WHEN the user clones definitions AND the user attempts to build the system base-system-testarch.morph in branch master THEN morph failed FINALLY the git server is shut down diff --git a/yarns/branches-workspaces.yarn b/yarns/branches-workspaces.yarn index 98265934..42ff1226 100644 --- a/yarns/branches-workspaces.yarn +++ b/yarns/branches-workspaces.yarn @@ -14,13 +14,13 @@ we use. Note that we only require a definitions repo here to allow us to obtain a list of shas so we can verify each sha has been anchored. - GIVEN a definitions repo + WHEN the user clones definitions The `morph anchor` command must be given a semantic name, to make the anchor refs less opaque to humans, and allow anchor ref cleanup by semantic name, such as when a given release is no longer supported. - WHEN the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master + AND the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master THEN morph succeeded After the `morph anchor` command completes, there are branches on the git @@ -148,9 +148,10 @@ Manifests Generating a manifest works SCENARIO morph generates a manifest - GIVEN a workspace + GIVEN a git server AND a system artifact - WHEN morph generates a manifest + WHEN the user clones definitions + AND morph generates a manifest THEN the manifest is generated A CSV manifest can also be generated. @@ -166,22 +167,21 @@ Definition diffing ------------------ SCENARIO diff reporting changes - GIVEN a workspace - AND a git server + GIVEN a git server - WHEN the user checks out the system branch called master + WHEN the user clones definitions AND chunk repository test-chunk is re-tagged as test-tag - AND chunk test-chunk in stratum strata/core.morph in system branch master is updated to use HEAD from chunk repository test-chunk - AND the user commits all changes in system branch master - AND the user requests a definition diff on all systems on branches HEAD and HEAD^ in system branch master + AND chunk test-chunk in stratum strata/core.morph in branch master is updated to use HEAD from chunk repository test-chunk + AND the user commits all changes in branch master + AND the user requests a definition diff on all systems on branches HEAD and HEAD^ in branch master THEN morph succeeded AND morph output test-tag ### diff implementations - IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+) in system branch (\S+) - repo="$DATADIR/workspace/$MATCH_3/test/morphs" + IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+) in branch (\S+) + repo="$DATADIR/definitions" attempt_morph diff "$repo" "$MATCH_1" - "$repo" "$MATCH_2" - Certify diff --git a/yarns/building.yarn b/yarns/building.yarn index 8a98e5d9..363d496a 100644 --- a/yarns/building.yarn +++ b/yarns/building.yarn @@ -2,38 +2,34 @@ Morph Building Tests ====================== SCENARIO attempting to build a system morphology which has never been committed - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master + GIVEN a git server + WHEN the user clones definitions + AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in branch master AND the user attempts to build the system systems/base-system.morph in branch master THEN morph succeeded FINALLY the git server is shut down SCENARIO build system with relative path - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master - AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system base-system.morph + GIVEN a git server + WHEN the user clones definitions + AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in branch master + AND from the directory definitions/systems the user attempts to morph build the system base-system.morph THEN morph succeeded FINALLY the git server is shut down SCENARIO build system with relative path (second variant) - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master - AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system ../systems/base-system.morph + GIVEN a git server + WHEN the user clones definitions + AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in branch master + AND from the directory definitions/systems the user attempts to morph build the system ../systems/base-system.morph THEN morph succeeded FINALLY the git server is shut down SCENARIO build system with absolute path - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master - AND from the directory workspace/master/test/morphs/systems the user attempts to morph build the system using the absolute path to base-system.morph + GIVEN a git server + WHEN the user clones definitions + AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in branch master + AND from the directory definitions/systems the user attempts to morph build the system using the absolute path to base-system.morph THEN morph succeeded FINALLY the git server is shut down @@ -47,20 +43,19 @@ so every library path in `/etc/ld.so.conf` can be found, and it can look up libraries more quickly. SCENARIO using system integrations - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user attempts to build the system systems/test-system.morph in branch master THEN morph succeeded In our example, we have a system integration that creates /etc/passwd, so when we deploy the system, we can check whether it exists. - GIVEN a cluster called test-cluster.morph in system branch master - AND a system in cluster test-cluster.morph in branch master called test-system - AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph - AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar - WHEN the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" + GIVEN a cluster called clusters/test-cluster.morph in system branch master + AND a system in cluster clusters/test-cluster.morph in branch master called test-system + AND system test-system in cluster clusters/test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster clusters/test-cluster.morph in branch master has deployment type: tar + WHEN the user attempts to deploy the cluster clusters/test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" THEN morph succeeded AND tarball test.tar contains etc/passwd FINALLY the git server is shut down @@ -70,16 +65,15 @@ Distbuilding SCENARIO distbuilding ASSUMING the morph-cache-server can be run - GIVEN a workspace - AND a git server + GIVEN a git server AND a communal cache server AND a distbuild worker AND a distbuild controller Distbuilding works much the same way as regular building. - WHEN the user checks out the system branch called master - AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in system branch master + WHEN the user clones definitions + AND the user creates an uncommitted system morphology called systems/base-system.morph for our architecture in branch master THEN morph distbuild the system systems/base-system.morph of the branch master Distbuilt systems can be deployed locally, since the building process @@ -106,9 +100,8 @@ Empty strata don't build ------------------------ SCENARIO empty-strata - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called empty-stratum + GIVEN a git server + WHEN the user clones definitions AND the user attempts to build the system systems/empty-stratum-system.morph in branch empty-stratum THEN morph failed FINALLY the git server is shut down @@ -117,9 +110,8 @@ Partial building ---------------- SCENARIO partial building - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user attempts to build build-essential from the system systems/test-system.morph in branch master THEN morph succeeded FINALLY the git server is shut down diff --git a/yarns/deployment.yarn b/yarns/deployment.yarn index 85bb2c9d..a3a8a679 100644 --- a/yarns/deployment.yarn +++ b/yarns/deployment.yarn @@ -2,47 +2,42 @@ Morph Deployment Tests ====================== SCENARIO deploying a non-cluster morphology - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user attempts to deploy the system systems/test-system.morph in branch master + GIVEN a git server + WHEN the user clones definitions + AND the user attempts to deploy the system systems/test-system.morph in branch master THEN morph failed AND the deploy error message includes the string "morph deployment commands are only supported for cluster morphologies" FINALLY the git server is shut down SCENARIO deploying a cluster morphology as a tarfile - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user builds the system systems/test-system.morph in branch master + GIVEN a git server + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master AND the user attempts to deploy the cluster clusters/test-cluster.morph in branch master THEN morph succeeded FINALLY the git server is shut down SCENARIO deploying a cluster using a relative path - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user builds the system systems/test-system.morph in branch master - AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster test-cluster.morph in branch master + AND from directory definitions/clusters the user attempts to deploy the cluster test-cluster.morph in branch master THEN morph succeeded FINALLY the git server is shut down SCENARIO deploying a cluster using a relative path (second variant) - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user builds the system systems/test-system.morph in branch master - AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster ../clusters/test-cluster.morph in branch master + AND from directory definitions/clusters the user attempts to deploy the cluster ../clusters/test-cluster.morph in branch master THEN morph succeeded FINALLY the git server is shut down SCENARIO deploying a cluster using an absolute path - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user builds the system systems/test-system.morph in branch master - AND from directory workspace/master/test/morphs/clusters the user attempts to deploy the cluster using the absolute path to test-cluster.morph in branch master + AND from directory definitions/clusters the user attempts to deploy the cluster using the absolute path to test-cluster.morph in branch master THEN morph succeeded FINALLY the git server is shut down @@ -50,9 +45,8 @@ Some deployment types support upgrades, but some do not and Morph needs to make this clear. SCENARIO attempting to upgrade a tarfile deployment - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user builds the system systems/test-system.morph in branch master AND the user attempts to upgrade the cluster clusters/test-cluster.morph in branch master THEN morph failed @@ -65,18 +59,17 @@ base OS as an upgrade. While pointless, this is permitted and does exercise the same code paths as a real upgrade. SCENARIO deploying a cluster morphology as rawdisk and then upgrading it - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - GIVEN a cluster called test-cluster.morph in system branch master - AND a system in cluster test-cluster.morph in branch master called test-system - AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph - AND system test-system in cluster test-cluster.morph in branch master has deployment type: rawdisk - AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar - WHEN the user builds the system systems/test-system.morph in branch master - AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.DISK_SIZE=20M test-system.VERSION_LABEL=test1 + GIVEN a git server + GIVEN a cluster called clusters/test-cluster.morph in system branch master + AND a system in cluster clusters/test-cluster.morph in branch master called test-system + AND system test-system in cluster clusters/test-cluster.morph in branch master builds systems/test-system.morph + AND system test-system in cluster clusters/test-cluster.morph in branch master has deployment type: rawdisk + AND system test-system in cluster clusters/test-cluster.morph in branch master has deployment location: test.tar + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster clusters/test-cluster.morph in branch master with options test-system.DISK_SIZE=20M test-system.VERSION_LABEL=test1 THEN morph succeeded - WHEN the user attempts to upgrade the cluster test-cluster.morph in branch master with options test-system.VERSION_LABEL=test2 + WHEN the user attempts to upgrade the cluster clusters/test-cluster.morph in branch master with options test-system.VERSION_LABEL=test2 THEN morph succeeded FINALLY the git server is shut down @@ -94,9 +87,7 @@ For the use-cases of: It is convenient to be able to deploy one system inside another. SCENARIO deploying a cluster morphology with nested systems - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server GIVEN a cluster called test-cluster.morph in system branch master AND a system in cluster test-cluster.morph in branch master called test-system AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph @@ -112,7 +103,8 @@ We specify the location as a file path, this is relative to the parent system's extracted rootfs, before it is configured. AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment location: var/lib/sysroots/test-system - WHEN the user builds the system systems/test-system.morph in branch master + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" THEN morph succeeded @@ -141,9 +133,7 @@ There's a few ways of creating an initramfs. We could: SCENARIO deploying a system with an initramfs ASSUMING there is space for 5 512M disk images - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server GIVEN a cluster called C.morph in system branch master AND a system in cluster C.morph in branch master called S @@ -190,15 +180,16 @@ Fully testing that the system is bootable requires a lot more time, infrastructure and dependencies, so we're just going to build it and inspect the result of the deployment. - WHEN the user builds the system systems/test-system.morph in branch master + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master AND the user attempts to deploy the cluster C.morph in branch master THEN morph succeeded - AND file workspace/master/test/morphs/test.img exists + AND file definitions/test.img exists If the initramfs write extension works, the rootfs image should contain `boot/initramfs.gz`. - WHEN disk image workspace/master/test/morphs/test.img is mounted at mnt + WHEN disk image definitions/test.img is mounted at mnt THEN file mnt/systems/default/run/boot/initramfs.gz exists If the `rawdisk` write extension worked, then the bootloader config file @@ -219,10 +210,7 @@ Starting from the well-defined position of having a cluster morphology with only one definition. SCENARIO partially deploying a cluster morphology - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user builds the system systems/test-system.morph in branch master + GIVEN a git server GIVEN a cluster called test-cluster.morph in system branch master AND a system in cluster test-cluster.morph in branch master called test-system AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph @@ -242,17 +230,19 @@ of the systems in the cluster are deployed. Here a successful deployment will have morph exit sucessfully and in the case of tarball deployments, the tarballs for both the systems will be created. - WHEN the user attempts to deploy the cluster test-cluster.morph in branch master + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master + AND the user attempts to deploy the cluster test-cluster.morph in branch master THEN morph succeeded - AND file workspace/master/test/morphs/test-system.tar exists - AND file workspace/master/test/morphs/second-system.tar exists + AND file definitions/test-system.tar exists + AND file definitions/second-system.tar exists However, we don't need to deploy every system defined in a cluster at once. This is useful for cases such as having a cluster morphology for deploying a whole distbuild network, and re-deploying only nodes that have failed. - GIVEN the files workspace/master/test/morphs/test-system.tar and workspace/master/test/morphs/second-system.tar are removed + GIVEN the files definitions/test-system.tar and definitions/second-system.tar are removed WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master A successful deployment will have morph exit successfully, and in the @@ -260,8 +250,8 @@ case of tarball deployments, only the tarball for the system we asked for will be created. THEN morph succeeded - AND file workspace/master/test/morphs/test-system.tar exists - AND file workspace/master/test/morphs/second-system.tar does not exist + AND file definitions/test-system.tar exists + AND file definitions/second-system.tar does not exist Cluster morphs can contain "nested systems", i.e. systems which have subsystems to deploy as part of them. @@ -280,11 +270,12 @@ system's extracted rootfs, before it is configured. The system which contains a nested system is deployed the same as before, we don't need to mention the nested deployment. - AND the file workspace/master/test/morphs/test-system.tar is removed - WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master + AND the file definitions/test-system.tar is removed + WHEN the user pulls definitions from origin master + AND the user attempts to deploy test-system from cluster test-cluster.morph in branch master THEN morph succeeded - AND file workspace/master/test/morphs/test-system.tar exists - AND tarball workspace/master/test/morphs/test-system.tar contains var/lib/sysroots/test-system/baserock + AND file definitions/test-system.tar exists + AND tarball definitions/test-system.tar contains var/lib/sysroots/test-system/baserock Morph will abort deployment if the system to deploy that is specified on the command line is not defined in the morphology. @@ -308,9 +299,8 @@ It is possible to only deploy one or more strata or chunks from a system when deploying to a tarball or sysroot. SCENARIO partially deploying a system - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master + GIVEN a git server + WHEN the user clones definitions AND the user builds the system systems/test-system.morph in branch master AND the user attempts to partially deploy tar from cluster clusters/partial-test-cluster.morph in branch master THEN morph succeeded @@ -322,49 +312,3 @@ something other than tarball or sysroot. WHEN the user attempts to partially deploy rawdisk from cluster clusters/partial-test-cluster.morph in branch master THEN morph failed - -Deploying branch-from-image produced systems -============================================ - -We have this nifty subcommand called branch-from-image, which can be -used to build the same thing as an existing image. - -There's no special requirements for making the image reproducible. - - SCENARIO reproducing systems - GIVEN a workspace - AND a git server - WHEN the user checks out the system branch called master - AND the user builds the system systems/test-system.morph in branch master - GIVEN a cluster called test-cluster.morph in system branch master - AND a system in cluster test-cluster.morph in branch master called test-system - AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph - AND system test-system in cluster test-cluster.morph in branch master has deployment type: sysroot - AND system test-system in cluster test-cluster.morph in branch master has deployment location: test-system - WHEN the user attempts to deploy the cluster test-cluster.morph in branch master - THEN morph succeeded - AND file workspace/master/test/morphs/test-system exists - -To reproduce an existing image, do a checkout with the extracted root -filesystem's /baserock directory as the `--metadata-dir` argument. - - WHEN the user attempts to check out the system branch from workspace/master/test/morphs/test-system called mybranch - THEN morph succeeded - AND the system branch mybranch is checked out - -After it is checked-out, the system can be rebuilt. - - WHEN the user attempts to build the system systems/test-system.morph in branch mybranch - THEN morph succeeded - -Once it is rebuilt, it can be deployed. - - GIVEN a cluster called test-cluster.morph in system branch mybranch - AND a system in cluster test-cluster.morph in branch mybranch called test-system - AND system test-system in cluster test-cluster.morph in branch mybranch builds systems/test-system.morph - AND system test-system in cluster test-cluster.morph in branch mybranch has deployment type: tar - AND system test-system in cluster test-cluster.morph in branch mybranch has deployment location: test-system.tar - WHEN the user attempts to deploy the cluster test-cluster.morph in branch mybranch - THEN morph succeeded - AND file workspace/mybranch/test/morphs/test-system.tar exists - FINALLY the git server is shut down diff --git a/yarns/implementations.yarn b/yarns/implementations.yarn index 1aa4c088..ea0ae2c8 100644 --- a/yarns/implementations.yarn +++ b/yarns/implementations.yarn @@ -4,38 +4,20 @@ IMPLEMENTS implementations Implementation sections for definitions repo --------------------------------------------- - IMPLEMENTS GIVEN a definitions repo + IMPLEMENTS WHEN the user clones definitions git clone "$(cat $DATADIR/definitions-repo-url)" "$DATADIR/definitions" + IMPLEMENTS WHEN the user pulls definitions from (\S+) (\S+) + run_in "$DATADIR/definitions" git pull "$MATCH_1" "$MATCH_2" + IMPLEMENTS WHEN the user creates a new definitions branch (\S+), based on (\S+) run_in "$DATADIR/definitions" git branch "$MATCH_1" "$MATCH_2" IMPLEMENTS WHEN the user checks out definitions branch (\S+) run_in "$DATADIR/definitions" git checkout "$MATCH_1" -Implementation sections for workspaces --------------------------------------- - -We'll use `$DATADIR/workspace` as the workspace directory that is used. - - IMPLEMENTS GIVEN no workspace - true - - IMPLEMENTS GIVEN an empty workspace directory - mkdir "$DATADIR/workspace" - - IMPLEMENTS GIVEN a non-empty workspace directory - mkdir "$DATADIR/workspace" - touch "$DATADIR/workspace/random-file" - -We run `morph init` in two different ways: either the simple way, -letting yarn catch errors, or in a way that catches the error so -we can test it later in a THEN step. - - IMPLEMENTS WHEN the user (attempts to initialise|initialises) a workspace - set init "$DATADIR/workspace" - if [ $MATCH_1 == "initialises" ]; then run_morph "$@" - else attempt_morph "$@"; fi +Implementation sections for checking morph exit status +------------------------------------------------------ IMPLEMENTS THEN morph failed case $(cat "$DATADIR/morph-exit") in @@ -54,21 +36,6 @@ we can test it later in a THEN step. IMPLEMENTS THEN morph output (.*) grep -q "$MATCH_1" "$DATADIR/out-latest" -We need to check that a workspace creation worked. This requires the -directory to exist, and its `.morph` subdirectory to exist, and nothing -else. - - IMPLEMENTS THEN an empty workspace exists - is_dir "$DATADIR/workspace" - is_dir "$DATADIR/workspace/.morph" - assert_equal $(ls -A "$DATADIR/workspace" | wc -l) 1 - -Tests for things other than `morph init` just want to have a workspace -created. - - IMPLEMENTS GIVEN a workspace - run_morph init "$DATADIR/workspace" - Implementation sections related to a simulated Trove ---------------------------------------------------- @@ -515,43 +482,6 @@ You need an architecture to build a system, we don't default to the host archite run_in "$DATADIR/gits/morphs" git add "$MATCH_1" run_in "$DATADIR/gits/morphs" git commit -m "Added $MATCH_1." -Implementation sections for system branch operations ----------------------------------------------------- - -Checkout out an existing system branch. We parameterise this so the -same phrase can be used to check out any system branch. - - IMPLEMENTS WHEN the user (attempts to check|checks) out the system (branch|tag) called (\S+) - cd "$DATADIR/workspace" - set checkout test:morphs "$MATCH_3" - if [ $MATCH_1 == "checks" ]; then run_morph "$@" - else attempt_morph "$@"; fi - -Attempt to check out a system branch from a root that has no systems. - - IMPLEMENTS WHEN the user attempts to check out from a repository with no systems - cd "$DATADIR/workspace" - attempt_morph checkout test:test-chunk master - - IMPLEMENTS WHEN the user attempts to check out the system branch from (\S+) called (\S+) - cd "$DATADIR/workspace" - attempt_morph branch-from-image --metadata-dir "$DATADIR/$MATCH_1/baserock" "$MATCH_2" - -We also need to verify that a system branch has been checked out. - - IMPLEMENTS THEN the system branch (\S+) is checked out - is_dir "$DATADIR/workspace/$MATCH_1/test/morphs" - is_file "$DATADIR/workspace/$MATCH_1/test/morphs/systems/test-system.morph" - is_file "$DATADIR/workspace/$MATCH_1/test/morphs/strata/core.morph" - -We can create a new branch, off master. - - IMPLEMENTS WHEN the user (attempts to create|creates) a system branch called (\S+) - cd "$DATADIR/workspace" - set branch test:morphs "$MATCH_2" - if [ $MATCH_1 == "creates" ]; then run_morph "$@" - else attempt_morph "$@"; fi - Cloning chunk repos with `morph get-repo`. IMPLEMENTS WHEN the user gets the repo for chunk (\S+) in branch (\S+) @@ -593,10 +523,12 @@ To produce buildable morphologies, we need them to be of the same architecture as the machine doing the testing. This uses `morph print-architecture` to get a value appropriate for morph. - IMPLEMENTS WHEN the user creates an uncommitted system morphology called (\S+) for our architecture in system branch (\S+) + IMPLEMENTS WHEN the user creates an uncommitted system morphology called (\S+) for our architecture in branch (\S+) arch=$(run_morph print-architecture) name="$(basename "${MATCH_1%.*}")" - install -m644 -D /dev/stdin << EOF "$DATADIR/workspace/$MATCH_2/test/morphs/$MATCH_1" + cd "$DATADIR/definitions" + git checkout "$MATCH_2" + install -m644 -D /dev/stdin << EOF "$MATCH_1" arch: $arch configuration-extensions: [] description: A system called $name for architectures $arch @@ -608,6 +540,7 @@ print-architecture` to get a value appropriate for morph. - name: core morph: strata/core.morph EOF + git checkout - Reporting status of checked out repositories: @@ -699,9 +632,11 @@ Generating a manifest. fi die "Expected CSV manifest not found" - IMPLEMENTS WHEN the user commits all changes in system branch (\S+) - cd "$DATADIR/workspace/$MATCH_1/test/morphs" + IMPLEMENTS WHEN the user commits all changes in branch (\S+) + cd "$DATADIR/definitions" + git checkout "$MATCH_1" git commit -a --allow-empty -m 'Commit all changes' + git checkout - Implementations for `morph certify` ----------------------------------- @@ -785,13 +720,15 @@ Implementation sections for building ==================================== IMPLEMENTS WHEN the user (attempts to )?((dist)?build)s? the system (\S+) in branch (\S+) - cd "$DATADIR/workspace/$MATCH_5/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_5" set "$MATCH_2" "$MATCH_4" if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@" else attempt_morph "$@"; fi IMPLEMENTS WHEN the user (attempts to )?((dist)?build)s? (\S+) from the system (\S+) in branch (\S+) - cd "$DATADIR/workspace/$MATCH_6/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_5" set "$MATCH_2" "$MATCH_5" "$MATCH_4" if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@" else attempt_morph "$@"; fi @@ -800,7 +737,8 @@ Implementation sections for cross-bootstrapping =============================================== IMPLEMENTS THEN the user cross-bootstraps the system (\S+) in branch (\S+) of repo (\S+) to the arch (\S+) - cd "$DATADIR/workspace/$MATCH_2/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_2" set -- cross-bootstrap "$MATCH_4" "$MATCH_3" "$MATCH_2" "$MATCH_1" run_morph "$@" @@ -816,8 +754,9 @@ them, so they can be added to the end of the implements section. then cd "$DATADIR/$MATCH_2" else - cd "$DATADIR/workspace/$MATCH_7/test/morphs" + cd "$DATADIR/definitions" fi + git checkout "$MATCH_7" if [ "$MATCH_5" != "" ] then @@ -832,26 +771,32 @@ them, so they can be added to the end of the implements section. fi if [ "$MATCH_3" = "deploys" ]; then run_morph "$@" else attempt_morph "$@"; fi + git checkout - IMPLEMENTS WHEN the user (attempts to deploy|deploys) (.*) from cluster (\S+) in branch (\S+) - cd "$DATADIR/workspace/$MATCH_4/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_4" set -- deploy "$MATCH_3" systems=$(echo "$MATCH_2" | sed -e 's/, /\n/g' -e 's/ and /\n/g') set -- "$@" $systems if [ "$MATCH_1" = "deploys" ]; then run_morph "$@" else attempt_morph "$@"; fi + git checkout - IMPLEMENTS WHEN the user (attempts to partially deploy|partially deploys) (.*) from cluster (\S+) in branch (\S+) - cd "$DATADIR/workspace/$MATCH_4/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_4" set -- deploy "$MATCH_3" systems=$(echo "$MATCH_2" | sed -e 's/, /\n/g' -e 's/ and /\n/g') echo "partial=True" >> "$DATADIR/morph.conf" set -- "$@" $systems if [ "$MATCH_1" = "partially deploys" ]; then run_morph "$@" else attempt_morph "$@"; fi + git checkout - IMPLEMENTS WHEN the user (attempts to upgrade|upgrades) the (system|cluster) (\S+) in branch (\S+)( with options (.*))? - cd "$DATADIR/workspace/$MATCH_4/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_4" set -- upgrade "$MATCH_3" if [ "$MATCH_5" != '' ]; then # eval used so word splitting in the text is preserved @@ -859,6 +804,7 @@ them, so they can be added to the end of the implements section. fi if [ "$MATCH_1" = "upgrades" ]; then run_morph "$@" else attempt_morph "$@"; fi + git checkout - Implementations sections for reading error messages =================================================== @@ -1001,11 +947,13 @@ Implementations for building systems ------------------------------------ IMPLEMENTS THEN morph ((dist)?build) the system (\S+) of the (branch|tag) (\S+) - cd "$DATADIR/workspace/$MATCH_5/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_5" run_morph "$MATCH_1" "$MATCH_3" IMPLEMENTS WHEN the user builds (\S+) of the (\S+) (branch|tag) - cd "$DATADIR/workspace/$MATCH_2/test/morphs" + cd "$DATADIR/definitions" + git checkout "$MATCH_2" run_morph build "$MATCH_1" IMPLEMENTS WHEN from the directory (\S+) the user attempts to morph build the system (\S+) @@ -1044,68 +992,67 @@ Altering morphologies in their source repositories git add "$MATCH_1" git commit -m "Make $MATCH_1 match $MATCH_2" - IMPLEMENTS WHEN chunk repository (\S+) is re-tagged as (\S+) - cd "$DATADIR/gits/$MATCH_1" - git commit --allow-empty -m "Prepare for $MATCH_2" - git tag -af -m "Release $MATCH_2" "$MATCH_2" - -Altering morphologies in the workspace --------------------------------------- - -### Altering strata ### - - IMPLEMENTS GIVEN stratum (\S+) in system branch (\S+) has match rules: (.*) - cd "$DATADIR/workspace/$MATCH_2/test/morphs" - "$SRCDIR/scripts/edit-morph" set-stratum-match-rules \ - "$MATCH_1" "$MATCH_3" - - IMPLEMENTS WHEN chunk (\S+) in stratum (\S+) in system branch (\S+) is updated to use (\S+) from chunk repository (\S+) - cd "$DATADIR/workspace/$MATCH_3/test/morphs" - sha1=$(cd "$DATADIR/gits/$MATCH_5" && git rev-parse "$MATCH_4") - "$SRCDIR/scripts/edit-morph" update-stratum-chunk-ref \ - "$MATCH_2" "$MATCH_1" "$sha1" - ### Altering clusters ### IMPLEMENTS GIVEN a cluster called (\S+) in system branch (\S+) - name="$MATCH_1" + cluster="$MATCH_1" branch="$MATCH_2" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-init \ - "$DATADIR/workspace/$branch/test/morphs/$name" + "$DATADIR/gits/morphs/$cluster" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Add cluster $cluster" + run_in "$DATADIR/gits/morphs" git checkout - IMPLEMENTS GIVEN a (sub)?system in cluster (\S+) in branch (\S+) called (\S+) cluster="$MATCH_2" branch="$MATCH_3" name="$MATCH_4" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-system-init \ - "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" + "$DATADIR/gits/morphs/$cluster" "$name" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Add system/subsystem $name to cluster $cluster" + run_in "$DATADIR/gits/morphs" git checkout - IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) builds (\S+) name="$MATCH_2" cluster="$MATCH_3" branch="$MATCH_4" morphology="$MATCH_5" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-system-set-morphology \ - "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \ + "$DATADIR/gits/morphs/$cluster" "$name" \ "$morphology" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Make system $name in cluster $cluster build $morphology" + run_in "$DATADIR/gits/morphs" git checkout - IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment type: (\S+) name="$MATCH_2" cluster="$MATCH_3" branch="$MATCH_4" type="$MATCH_5" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-type \ - "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \ + "$DATADIR/gits/morphs/$cluster" "$name" \ "$type" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Add system $name to cluster $cluster" + run_in "$DATADIR/gits/morphs" git checkout - IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment location: (\S+) name="$MATCH_2" cluster="$MATCH_3" branch="$MATCH_4" location="$MATCH_5" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-location \ - "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \ + "$DATADIR/gits/morphs/$cluster" "$name" \ "$location" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Set deploy location of system $name in cluster $cluster" + run_in "$DATADIR/gits/morphs" git checkout - IMPLEMENTS GIVEN (sub)?system (\S+) in cluster (\S+) in branch (\S+) has deployment variable: ([^=]+)=(.*) name="$MATCH_2" @@ -1113,9 +1060,38 @@ Altering morphologies in the workspace branch="$MATCH_4" key="$MATCH_5" val="$MATCH_6" + run_in "$DATADIR/gits/morphs" git checkout "$branch" "$SRCDIR/scripts/edit-morph" cluster-system-set-deploy-variable \ - "$DATADIR/workspace/$branch/test/morphs/$cluster" "$name" \ + "$DATADIR/gits/morphs/$cluster" "$name" \ "$key" "$val" + run_in "$DATADIR/gits/morphs" git add "$cluster" + run_in "$DATADIR/gits/morphs" git commit -m "Set deploy variable for system $name in cluster $cluster" + run_in "$DATADIR/gits/morphs" git checkout - + + IMPLEMENTS GIVEN stratum (\S+) in branch (\S+) has match rules: (.*) + run_in "$DATADIR/gits/morphs" git checkout "$MATCH_2" + "$SRCDIR/scripts/edit-morph" set-stratum-match-rules \ + "$DATADIR/gits/morphs/$MATCH_1" "$MATCH_3" + run_in "$DATADIR/gits/morphs" git add "$MATCH_1" + run_in "$DATADIR/gits/morphs" git commit -m "Make $MATCH_1 match $MATCH_2" + run_in "$DATADIR/gits/morphs" git checkout - + + +Altering morphologies in the definitions repo +--------------------------------------------- + + IMPLEMENTS WHEN chunk (\S+) in stratum (\S+) in branch (\S+) is updated to use (\S+) from chunk repository (\S+) + cd "$DATADIR/definitions" + git checkout "$MATCH_3" + sha1=$(cd "$DATADIR/gits/$MATCH_5" && git rev-parse "$MATCH_4") + "$SRCDIR/scripts/edit-morph" update-stratum-chunk-ref \ + "$MATCH_2" "$MATCH_1" "$sha1" + git checkout - + + IMPLEMENTS WHEN chunk repository (\S+) is re-tagged as (\S+) + cd "$DATADIR/gits/$MATCH_1" + git commit --allow-empty -m "Prepare for $MATCH_2" + git tag -af -m "Release $MATCH_2" "$MATCH_2" Distbuild diff --git a/yarns/regression.yarn b/yarns/regression.yarn index aa98eec5..6448aa62 100644 --- a/yarns/regression.yarn +++ b/yarns/regression.yarn @@ -7,44 +7,19 @@ Tests for check we don't introduce some bugs again. Testing if we can build after checking out from a tag. SCENARIO morph build works after checkout from a tag - GIVEN a workspace - AND a git server - WHEN the user checks out the system tag called test-tag + GIVEN a git server + WHEN the user clones definitions THEN morph build the system systems/test-system.morph of the tag test-tag FINALLY the git server is shut down - -Running `morph branch` when the branch directory exists doesn't -remove the existing directory. - - SCENARIO re-running 'morph branch' fails, original branch untouched - GIVEN a workspace - AND a git server - WHEN the user creates a system branch called foo - THEN the system branch foo is checked out - -The branch is checked out correctly, now it should fail if the user executes -`morph branch` with the same branch name. - - WHEN the user attempts to create a system branch called foo - THEN morph failed - AND the branch error message includes the string "File exists" - -The branch still checked out. - - AND the system branch foo is checked out - FINALLY the git server is shut down - - It doesn't make much sense to be able to build a system with only bootstrap chunks, since they will have been constructed without a staging area, hence their results cannot be trusted. SCENARIO building a system with only bootstrap chunks fails - GIVEN a workspace - AND a git server + GIVEN a git server AND a system containing only bootstrap chunks called bootstrap-system.morph - WHEN the user checks out the system branch called master + WHEN the user clones definitions AND the user attempts to build the system bootstrap-system.morph in branch master THEN the build error message includes the string "No non-bootstrap chunks found" FINALLY the git server is shut down @@ -56,18 +31,18 @@ from a source it depended on, but not the number of artifacts from that source it depended on. SCENARIO changing the artifacts a system uses - GIVEN a workspace - AND a git server + GIVEN a git server AND system systems/test-system.morph uses core-runtime from core AND stratum strata/core.morph has match rules: [{artifact: core-runtime, include: [.*-devel]}, {artifact: core-devel, include: [.*-runtime]}] - WHEN the user checks out the system branch called master GIVEN a cluster called test-cluster.morph in system branch master AND a system in cluster test-cluster.morph in branch master called test-system AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar - WHEN the user builds the system systems/test-system.morph in branch master - GIVEN stratum strata/core.morph in system branch master has match rules: [{artifact: core-runtime, include: [.*-runtime]}, {artifact: core-devel, include: [.*-devel]}] - WHEN the user builds the system systems/test-system.morph in branch master + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master + GIVEN stratum strata/core.morph in branch master has match rules: [{artifact: core-runtime, include: [.*-runtime]}, {artifact: core-devel, include: [.*-devel]}] + WHEN the user pulls definitions from origin master + AND the user builds the system systems/test-system.morph in branch master AND the user deploys the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" THEN tarball test.tar contains baserock/test-chunk-runtime.meta FINALLY the git server is shut down diff --git a/yarns/splitting.yarn b/yarns/splitting.yarn index 75831e53..4342260b 100644 --- a/yarns/splitting.yarn +++ b/yarns/splitting.yarn @@ -8,8 +8,7 @@ To verify that the products fields are parsed correctly, we have a scenario that uses all of them, not relying on the default rules. SCENARIO building a system with morphologies that have splitting rules - GIVEN a workspace - AND a git server + GIVEN a git server To test that all the fields are recognised, we set the new fields to their default values. @@ -22,7 +21,7 @@ The default rules produce a system that is identical to not providing them, and since this test is about validation, we don't care about the result, so much as it succeeding to build something. - WHEN the user checks out the system branch called master + WHEN the user clones definitions THEN morph build the system systems/test-system.morph of the branch master FINALLY the git server is shut down @@ -35,14 +34,12 @@ information, such as the documentation, C library headers and C static libraries. SCENARIO building a system only using runtime strata - GIVEN a workspace - AND a git server + GIVEN a git server The only change we need to make is to add a field to the system morphology to select which artifact to use in the system. AND system systems/test-system.morph uses core-runtime from core - WHEN the user checks out the system branch called master The best way to test that only using some stratum artifacts works is to check which files the output has, so we deploy a tarball and inspect @@ -52,7 +49,8 @@ its contents. AND a system in cluster test-cluster.morph in branch master called test-system AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar - WHEN the user builds the system systems/test-system.morph in branch master + WHEN the user clones definitions + AND the user builds the system systems/test-system.morph in branch master AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar" The -runtime artifacts include executables and shared libraries. @@ -73,8 +71,7 @@ to go into our system, such that there are chunk artifacts that are not needed, then they don't get built. SCENARIO building a system that has unused chunks - GIVEN a workspace - AND a git server + GIVEN a git server This GIVEN has a chunk in the stratum that never successfully builds, so we know that if the system successfully builds, then we only built @@ -82,7 +79,7 @@ chunks that were needed. AND stratum strata/core.morph has chunks that aren't used in core-minimal AND system systems/test-system.morph uses core-minimal from core - WHEN the user checks out the system branch called master + WHEN the user clones definitions THEN morph build the system systems/test-system.morph of the branch master FINALLY the git server is shut down -- cgit v1.2.1