diff options
Diffstat (limited to 'yarns/noncore-plugins.yarn')
| -rw-r--r-- | yarns/noncore-plugins.yarn | 254 |
1 files changed, 254 insertions, 0 deletions
diff --git a/yarns/noncore-plugins.yarn b/yarns/noncore-plugins.yarn new file mode 100644 index 00000000..1cf4fd90 --- /dev/null +++ b/yarns/noncore-plugins.yarn @@ -0,0 +1,254 @@ +Non-Core Morph Plugins Tests +============================ + +Anchoring commits for reproducibility +------------------------------------- + +For full reproducibility, we need to ensure we anchor the commits we use. +`morph anchor` handles making this easier by pushing refs to the repositories +we use. + + SCENARIO morph anchors commits used by a release + GIVEN a git server + +Note that we only require a definitions repo here to allow us to obtain +a list of shas so we can verify each sha has been anchored. + + WHEN the user clones definitions + +The `morph anchor` command must be given a semantic name, to make the anchor +refs less opaque to humans, and allow anchor ref cleanup by semantic name, such +as when a given release is no longer supported. + + AND the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master + THEN morph succeeded + +After the `morph anchor` command completes, there are branches on the git +server in every repository used by the listed systems referring to the commits +used. + + AND every commit used in master has anchor branches on the git server + +`morph anchor` needs to handle being told to create anchors when they already +exist for that system, as we may have forgotten to anchor a system as part of a +release, and systems likely share contents. + + WHEN the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master + THEN morph succeeded + AND every commit used in master has anchor branches on the git server + +`morph anchor` will roll back any pushes when it is unable to push, and report +failure. To test this we need a new system to anchor. + + GIVEN a chunk called extra-chunk + AND push access is denied to extra-chunk + WHEN the user creates a new definitions branch two, based on master + AND the user checks out definitions branch two + AND the user adds a new system to build called expanded-system that uses chunk extra-chunk + AND the user attempts to anchor systems/expanded-system.morph with semantic name foo-release in branch two + THEN morph failed + +When this happens `morph anchor` must not remove anchors that already existed +before attempting to anchor, as they may be from a different system that shares +some components. + + THEN every commit used in master has anchor branches on the git server + FINALLY the git server is shut down + +### Anchor implementations + + IMPLEMENTS WHEN the user (attempts to )?(anchor) (.*) with semantic name (.*) in branch (\S+) + + systems=$(echo "$MATCH_3" | sed -e 's/, /\n/g' -e 's/ and /\n/g') + set "$MATCH_2" "$MATCH_4" "file://$DATADIR/definitions" "$MATCH_5" $systems + if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@" + else attempt_morph "$@"; fi + + IMPLEMENTS THEN every commit used in (.*) has anchor branches on the git server + set +e + run_in "$DATADIR/definitions" git checkout "$MATCH_1" + # extract sha1s from ref: fields + find "$DATADIR/definitions" -name '*.morph' \ + -exec sed -rn '/[^-]ref:\s*[0-9a-f]{40}/s/^.*([0-9a-f]{40})/\1/p' {} + | sort -u >"$DATADIR/sha1s" + for sha1 in $(cat "$DATADIR/sha1s"); do + sha1found=false + # for each git repository + for gitdir in $(find "$DATADIR/gits" -name '.git' -prune -exec dirname {} ';'); do + # for each ref, check if the commit it points to is the sha1 we seek + eval "$(cd "$gitdir" && git for-each-ref 'refs/heads/*/anchors/**' --shell \ + --format='"$sha1found" || [ %(objectname) = "$sha1" ] && sha1found=true')" + done + if ! "$sha1found"; then + exit 1 + fi + done + + IMPLEMENTS GIVEN push access is denied to (.*) + install -m 755 /dev/stdin "$DATADIR/gits/$MATCH_1/.git/hooks/pre-receive" <<'EOF' + #!/bin/sh + touch "$GIT_DIR/hook-ever-run" + echo No push for you! + exit 1 + EOF + + IMPLEMENTS GIVEN a chunk called (.*) + mkdir "$DATADIR/gits/$MATCH_1" + cd "$DATADIR/gits/$MATCH_1" + git init . + install -D -m644 /dev/stdin <<'EOF' "usr/share/doc/$MATCH_1/README" + No other content here + EOF + git add . + git commit -m "$MATCH_1 exists" + + IMPLEMENTS WHEN the user adds a new system to build called (.*) that uses chunk (.*) + cd "$DATADIR/definitions" + + install -m644 -D /dev/stdin << EOF "strata/tools/$MATCH_2.morph" + name: extra-chunk + kind: chunk + build-system: manual + EOF + git add strata/tools/extra-chunk.morph + + install -m644 -D /dev/stdin << EOF "strata/tools.morph" + name: tools + kind: stratum + build-depends: + - morph: strata/build-essential.morph + chunks: + - name: "$MATCH_2" + morph: strata/tools/extra-chunk.morph + repo: test:extra-chunk + unpetrify-ref: master + ref: $(run_in "$DATADIR/gits/$MATCH_2" git rev-parse master) + build-depends: [] + EOF + git add strata/tools.morph + + arch=$(run_morph print-architecture) + install -m644 -D /dev/stdin <<EOF "systems/$MATCH_1.morph" + name: $MATCH_1 + kind: system + arch: $arch + strata: + - name: build-essential + morph: strata/build-essential.morph + - name: core + morph: strata/core.morph + - name: tools + morph: strata/tools.morph + EOF + git add "systems/$MATCH_1.morph" + git commit -m 'Add extended system' + +Manifests +--------- + +Generating a manifest works + + SCENARIO morph generates a manifest + GIVEN a git server + AND a system artifact + WHEN the user clones definitions + AND morph generates a manifest + THEN the manifest is generated + +A CSV manifest can also be generated. + + SCENARIO morph generates a csv manifest + GIVEN a git server + + WHEN morph generates a manifest for system systems/test-system.morph at ref HEAD in repository test:definitions + THEN morph succeeded + AND the csv manifest is generated + +Definition diffing +------------------ + + SCENARIO diff reporting changes + GIVEN a git server + + WHEN the user clones definitions + AND chunk repository test-chunk is re-tagged as test-tag + AND chunk test-chunk in stratum strata/core.morph in branch master is updated to use HEAD from chunk repository test-chunk + AND the user commits all changes in branch master + AND the user requests a definition diff on all systems on branches HEAD and HEAD^ in branch master + + THEN morph succeeded + AND morph output test-tag + +### diff implementations + + IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+) in branch (\S+) + repo="$DATADIR/definitions" + attempt_morph diff "$repo" "$MATCH_1" - "$repo" "$MATCH_2" - + +Certify +------- + +`morph certify` can be used to get an indication whether or not building +a system is likely to have a reproducible result. + + SCENARIO using morph certify + GIVEN a git server + + WHEN the user certifies the system systems/test-system.morph at ref HEAD in repository test:definitions + THEN morph succeeded + AND morph output Reproducibility certification PASSED + +Listing artifacts in a system +----------------------------- + +The `morph list-artifacts` command can be used to list the names of each +artifact in a system. This name includes the cache key of the artifact. + + SCENARIO using morph list-artifacts + GIVEN a git server + + WHEN the user lists the artifacts which make up the system systems/test-system.morph at ref HEAD in repository test:definitions + THEN morph succeeded + +Printing the architecture +------------------------- + +This is short and simple. Morph can print the name for the current +architecture, and we verify not that it is correct, but that exactly +one line is printed to the standard output. The reason we're not +checking it's correct is because that would require the test code +to duplicate the architecture name list that is in the code already, +and that wouldn't help with tests. However, verifying there's exactly +one line in stdout (and nothing in stderr) means the plugin does at +least something sensible. + +Oh, and the one line should contain no spaces, either. + + SCENARIO morph print-architecture prints out a single word + WHEN morph print-architecture is run + THEN stdout contains a single line + AND stdout contains no spaces + AND stderr is empty + + IMPLEMENTS WHEN morph print-architecture is run + set +x + run_morph print-architecture > "$DATADIR/stdout" 2> "$DATADIR/stderr" + + IMPLEMENTS THEN stdout contains a single line + n=$(wc -l < "$DATADIR/stdout") + if [ "$n" != 1 ] + then + die "stdout contains $n lines, not 1" + fi + + IMPLEMENTS THEN stdout contains no spaces + n=$(tr < "$DATADIR/stdout" -cd ' ' | wc -c) + if [ "$n" != 0 ] + then + die "stdout contains spaces" + fi + + IMPLEMENTS THEN stderr is empty + if [ -s "$DATADIR/stderr" ] + then + die "stderr is not empty" + fi |