| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| |
| |
| |
| | |
'origin/baserock/richardmaw/S8544/deploy-cleanup-v3'
Reviewed-by: Lars Wirzenius
Reviewed-by: Daniel Silverstone
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
When a configuration extension or write extension fails, then it should
abort then, not continue to run other configuration extensions.
There was a period where this didn't happen, due to a missing feature
of cliapp that was assumed to be there, so failure to run these
extensions was not noticed.
This has since been fixed, but this would cause deploy to fail to
clean up its temporary directories.
Now it will cleanup the contents of the temporary directory after
any failures after it has been created.
A small amount of re-ordering was performed to make this easier.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
It's a waste of time to unpack the rootfs, only to have to clean it up
again when you find out that you messed up the command line arguments.
This also has the benefit of reducing the amount of resources that have
to be considered for cleanup.
|
| | |
| |
| |
| | |
It is now a tested helper function in morphlib.util
|
| |/
|
|
|
|
|
|
|
| |
We can't assume an extension cleans up after itself, as they can be
arbitrary shell scripts, and the best shell has to offer for cleanup is
`trap`, which is difficult to use.
So now, anything created with `mktemp` will get automatically cleaned
up by morph.
|
| |\ |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Checkout needs the branch repository and ref. It was previously only
getting the ref.
This was not noticed, since the implementation was only used in
cases where it was expected to fail, and the nature of the error
was not being checked.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
It's rather easy to accidentally try to create a system branch of
the `morph` repository rather than the `morphs` repository.
This commit verifies that a checked out branch contains system
morphologies, otherwise it aborts and cleans up.
|
| |\ \
| |/
|/|
| | |
'origin/baserock/richardmaw/S8532/validate-stratum-build-depends'
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A stratum morphology must either have stratum build-depends, or
have some bootstrap chunks, otherwise there's no way to have the
required set of commands to be able to build chunks.
A concession has been made to also allow strata that contain chunks
built in test mode, but this opens a reproducibility hole.
Unit tests for these failures have been added, and the stratum
used by other unit tests has been fixed.
|
| | |
| |
| |
| |
| |
| | |
This is already needed elsewhere for tests that perform builds,
however in future, validation will be more strict, so now these
dummy strata need to be marked as tests as well.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
It is better to test whether a straum is empty here, since it will be
noticed earlier in the build, as soon as the morphologies are loaded,
rather than after they have all been parsed.
It is also conceptually nicer to put it here, since the morphologyfactory
was written to perform this kind of validation.
On a more practical note, the validation is moved here so that the test
for this error isn't masked by the test for no build dependencies.
To ensure tests still pass, we alter the stratum morphology used by
other unit tests to no longer be empty, and add an empty one to test.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This creates a small exception hierarchy for failures to validate
stratum morphologies. This is currently a line rather than a tree,
but it will be expanded later in the patch series.
This also adds test coverage for chunk build dependencies being
omitted.
|
| |/
|
|
| |
Reviewed-by: Lars Wirzenius and Jonathan Maw on IRC
|
| |\
| |
| |
| | |
Reviewed-by: various
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
This is a helper to avoid having the same code in other places. Have
it in the one place it belongs.
|
| | | |
|
| | |
| |
| |
| |
| | |
This will be useful later, when we start creating system branch
directories.
|
| | |
| |
| |
| |
| |
| |
| | |
These will be used to find workspace and system branch root directories.
Also accidentally removed some whitespace from ends of lines. Too lazy
to split that into a separate commit.
|
| |\ \
| |/
|/|
| | |
'origin/baserock/richardmaw/S8530/atomic-hl-cache-creation'
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This uses the same logic as when a build fails, so it's been
consolidated into `StagingArea.abort()`.
You could argue that if a build fails before any commands are run,
then there's nothing interesting to see, but it will be useful if the
hardlink/tarball extract algorithm fails in some corner case.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Paul had file system problems which led to a partially constructed
chunk hardlink cache, which caused later builds to fail, since they
got a partially extracted chunk.
This patch fixes the case where the failed extract caused unreproducible
builds, but it's possible to corrupt the hardlink cache in other ways.
Read-only btrfs subvolumes would fix this, but either tie us further to
btrfs, or complicates the codebase with fallback logic.
|
| |/
|
|
|
|
|
|
| |
It is expected that the Application creates directories that will
be used, to keep the code for using them cleaner.
The FakeApplication wasn't updated to do this, as it was not yet needed,
but changes for atomicity require it to exist.
|
| |\ |
|
| | | |
|
| |\ \
| |/
|/|
| | |
Reviewed-by: Daniel Silverstone
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Put new implementation into new branch and merge plugin, and remove
old implementation from the old plugin.
Also change the error message for the NotInWorkspace exception, so that
it matches what the test suite expects. It's a crappy error message, I
think, but I don't want to change external behaviour during refactoring.
|
| | |
| |
| |
| |
| |
| |
| | |
Move "morph init" into a plugin, and remove it from the previously
existing plugin. This keeps all the old, tricky code in the old
plugin, and moves new, clean code into a new plugin. Eventually the
old plugin can be removed, since it'll be empty.
|
| |/ |
|
| |\
| |
| |
| |
| | |
Briefly reviewed by Jonathan Maw and Tiago Gomes, but all responsibility
is mine.
|
| | |
| |
| |
| | |
Reported-by: Tiago Gomes
|
| | |
| |
| |
| | |
Reported-by: Jonathan Maw
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
These scenarios test the basics of most of the subcommands the
branch and merge plugin provides. They don't purport to be complete,
but give some indication that things work, and form a basis upon
which further things can be built. Yarn also isn't included in a
Baserock release yet, so we need to keep the cmdtests until Baserock
10 has been released.
The existing cmdtest tests are not modified by this: they are left
intact, until they can analysed in detail for things to be added to
the scenarios. After that, the cmdtest tests will start to go away.
Merging is not covered by these tests: it is not clear how merge should
work, and the current code is known to do the wrong thing in many cases.
Scenarios for merge will be added later.
Building is also not covered. Testing builds well needs some additional,
careful thinking, and that isn't ready for this patch series. It will
be added later.
|
| | | |
|
| |\ \ |
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Cross-bootstrap is a way to build baserock on an architecture that
does not currently have Baserock. It can be used by `morph
cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact
that can be used as a root filesystem with a basic build environment
with a script named `native-bootstrap` which will build and install
every chunk in the system.
If done with a devel system, this will give you a suitable environment
for building a proper Baserock system.
This does not currently provide a kernel for the target architecture.
Apart from adding the cross-bootstrap plugin, it also makes the
following changes:
* Moves the lit of valid_archs into morphlib (instead of locally-scoped
in MorphologyFactory)
* BuildCommand takes an extra argument, build_env
* split BuildCommand's get_artifact_object into create_source_pool and
resolve_artifacts (plus changes things that use get_artifact_object to
use the new way)
* setup_mounts finds out whether to do so by whether build_mode is
'staging', instead of by whether the setting 'staging-chroot' is true.
* Makes ChunkBuilder's get_sources use the
morphlib.builder2.extract_sources() method, and moved
set_mtime_recursively into morphlib.builder2, since it's not currently
used anywhere else.
* moved ChunkBuilder's get_commands into the Morphology class (plus
changes to anything that used get_commands)
|
| |\ \
| |/
|/|
| | |
Reviewed-by: Richard Maw <richard.maw@codethink.co.uk>
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| | |
Pre-reviewed by Daniel Silverstone.
|
| |/ |
|
