Morph black box tests for system branches and workspaces
========================================================

Anchoring commits for reproducibility
-------------------------------------

For full reproducibility, we need to ensure we anchor the commits we use.
`morph anchor` handles making this easier by pushing refs to the repositories
we use.

    SCENARIO morph anchors commits used by a release
    GIVEN    a git server

Note that we only require a definitions repo here to allow us to obtain
a list of shas so we can verify each sha has been anchored.

    WHEN     the user clones definitions

The `morph anchor` command must be given a semantic name, to make the anchor
refs less opaque to humans, and allow anchor ref cleanup by semantic name, such
as when a given release is no longer supported.

    AND      the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master
    THEN     morph succeeded

After the `morph anchor` command completes, there are branches on the git
server in every repository used by the listed systems referring to the commits
used.

    AND      every commit used in master has anchor branches on the git server

`morph anchor` needs to handle being told to create anchors when they already
exist for that system, as we may have forgotten to anchor a system as part of a
release, and systems likely share contents.

    WHEN     the user attempts to anchor systems/test-system.morph with semantic name foo-release in branch master
    THEN     morph succeeded
    AND      every commit used in master has anchor branches on the git server

`morph anchor` will roll back any pushes when it is unable to push, and report
failure. To test this we need a new system to anchor.

    GIVEN    a chunk called extra-chunk
    AND      push access is denied to extra-chunk
    WHEN     the user creates a new definitions branch two, based on master
    AND      the user checks out definitions branch two
    AND      the user adds a new system to build called expanded-system that uses chunk extra-chunk
    AND      the user attempts to anchor systems/expanded-system.morph with semantic name foo-release in branch two
    THEN     morph failed

When this happens `morph anchor` must not remove anchors that already existed
before attempting to anchor, as they may be from a different system that shares
some components.

    THEN     every commit used in master has anchor branches on the git server
    FINALLY  the git server is shut down

### Anchor implementations

    IMPLEMENTS WHEN the user (attempts to )?(anchor) (.*) with semantic name (.*) in branch (\S+)

    systems=$(echo "$MATCH_3" | sed -e 's/, /\n/g' -e 's/ and /\n/g')
    set "$MATCH_2" "$MATCH_4" "file://$DATADIR/definitions" "$MATCH_5" $systems
    if [ "$MATCH_1" != "attempts to " ]; then run_morph "$@"
    else attempt_morph "$@"; fi

    IMPLEMENTS THEN every commit used in (.*) has anchor branches on the git server
    set +e
    run_in "$DATADIR/definitions" git checkout "$MATCH_1"
    # extract sha1s from ref: fields
    find "$DATADIR/definitions" -name '*.morph' \
          -exec sed -rn '/[^-]ref:\s*[0-9a-f]{40}/s/^.*([0-9a-f]{40})/\1/p' {} + | sort -u >"$DATADIR/sha1s"
    for sha1 in $(cat "$DATADIR/sha1s"); do
        sha1found=false
        # for each git repository
        for gitdir in $(find "$DATADIR/gits" -name '.git' -prune -exec dirname {} ';'); do
            # for each ref, check if the commit it points to is the sha1 we seek
            eval "$(cd "$gitdir" && git for-each-ref 'refs/heads/*/anchors/**' --shell \
                        --format='"$sha1found" || [ %(objectname) = "$sha1" ] && sha1found=true')"
        done
        if ! "$sha1found"; then
            exit 1
        fi
    done

    IMPLEMENTS GIVEN push access is denied to (.*)
    install -m 755 /dev/stdin "$DATADIR/gits/$MATCH_1/.git/hooks/pre-receive" <<'EOF'
    #!/bin/sh
    touch "$GIT_DIR/hook-ever-run"
    echo No push for you!
    exit 1
    EOF

    IMPLEMENTS GIVEN a chunk called (.*)
    mkdir "$DATADIR/gits/$MATCH_1"
    cd "$DATADIR/gits/$MATCH_1"
    git init .
    install -D -m644 /dev/stdin <<'EOF' "usr/share/doc/$MATCH_1/README"
    No other content here
    EOF
    git add .
    git commit -m "$MATCH_1 exists"

    IMPLEMENTS WHEN the user adds a new system to build called (.*) that uses chunk (.*)
    cd "$DATADIR/definitions"

    install -m644 -D /dev/stdin << EOF "strata/tools/$MATCH_2.morph"
    name: extra-chunk
    kind: chunk
    build-system: manual
    EOF
    git add strata/tools/extra-chunk.morph

    install -m644 -D /dev/stdin << EOF "strata/tools.morph"
    name: tools
    kind: stratum
    build-depends:
    - morph: strata/build-essential.morph
    chunks:
    -   name: "$MATCH_2"
        morph: strata/tools/extra-chunk.morph
        repo: test:extra-chunk
        unpetrify-ref: master
        ref: $(run_in "$DATADIR/gits/$MATCH_2" git rev-parse master)
        build-depends: []
    EOF
    git add strata/tools.morph

    arch=$(run_morph print-architecture)
    install -m644 -D /dev/stdin <<EOF "systems/$MATCH_1.morph"
    name: $MATCH_1
    kind: system
    arch: $arch
    strata:
    - name: build-essential
      morph: strata/build-essential.morph
    - name: core
      morph: strata/core.morph
    - name: tools
      morph: strata/tools.morph
    EOF
    git add "systems/$MATCH_1.morph"
    git commit -m 'Add extended system'

Manifests
---------

Generating a manifest works

    SCENARIO    morph generates a manifest
    GIVEN       a git server
    AND         a system artifact
    WHEN        the user clones definitions
    AND         morph generates a manifest
    THEN        the manifest is generated

A CSV manifest can also be generated.

    SCENARIO morph generates a csv manifest
    GIVEN    a git server

    WHEN     morph generates a manifest for system systems/test-system.morph at ref HEAD in repository test:definitions
    THEN     morph succeeded
    AND      the csv manifest is generated

Definition diffing
------------------

    SCENARIO diff reporting changes
    GIVEN    a git server

    WHEN     the user clones definitions
    AND      chunk repository test-chunk is re-tagged as test-tag
    AND      chunk test-chunk in stratum strata/core.morph in branch master is updated to use HEAD from chunk repository test-chunk
    AND      the user commits all changes in branch master
    AND      the user requests a definition diff on all systems on branches HEAD and HEAD^ in branch master

    THEN     morph succeeded
    AND      morph output test-tag

### diff implementations

    IMPLEMENTS WHEN the user requests a definition diff on all systems on branches (\S+) and (\S+) in branch (\S+)
    repo="$DATADIR/definitions"
    attempt_morph diff "$repo" "$MATCH_1" - "$repo" "$MATCH_2" -

Certify
-------

`morph certify` can be used to get an indication whether or not building
a system is likely to have a reproducible result.

    SCENARIO using morph certify
    GIVEN    a git server

    WHEN     the user certifies the system systems/test-system.morph at ref HEAD in repository test:definitions
    THEN     morph succeeded
    AND      morph output Reproducibility certification PASSED

Listing artifacts in a system
-----------------------------

The `morph list-artifacts` command can be used to list the names of each
artifact in a system. This name includes the cache key of the artifact.

    SCENARIO using morph list-artifacts
    GIVEN    a git server

    WHEN     the user lists the artifacts which make up the system systems/test-system.morph at ref HEAD in repository test:definitions
    THEN     morph succeeded