| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
It will now load the morphology from the definitions repository if the
"morph" field is present in the chunk spec.
Rather than adapting the loop to fit yet-more changing circumstances, it
has been partially rewritten, so there is one loop for loading
systems and strata from definitions.git, another for chunks from
definitions.git, and a third for chunks in the source repository.
This is tidier than attempting to fit the logic in the main loop, as it
makes it easier to remove afterwards when we no longer need to load
chunk morphologies from the source repository.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rather than repeatedly stripping and appending an optional .morph extension
morphology names, instead always use the file path of the morphology
relative to the definitions repository.
This is an inversion of the previous logic, which would strip the .morph
extension and use the "name" internally.
The exception to this rule of always using the filename, is that `morph
edit CHUNK` uses the name of the morphology as-defined in the stratum.
This is based off Adam Coldrick's inital patch, but this version will
allow the old style of providing the "name" by converting it into a path
if it does not have either a / or a . in it.
An unfortunate consequence of this change is that the show-dependencies
command's output changed, so the test needed updating.
|
| |
|
|
|
|
| |
After a failed attempt to connect to the controller node
the initiator will wait 30 seconds before attempting a reconnect,
if this reconnect fails the initiator gives up.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Artifacts can have multiple parts; while this may not be an ideal
design, changing the format of artifacts has implications for backwards
compatibility.
We should transfer all parts at once and delete them all if we encounter
any errors, to reduce the change of getting the local artifact cache
into an inconsistent state.
|
| |
|
|
|
|
|
|
|
|
|
| |
Previously Morph would check if an artifact is present in the remote
artifact cache, then fetch the necessary files. If an error occured
during fetching, it would raise an error and abort.
Instead, we should just try and fetch the files and if anything fails,
move on to building locally. This avoids the situation where an error in
the remote cache makes local building impossible, which we experienced
recently.
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
morphlib/plugins/deploy_plugin.py
without-test-modules
Reviewed by:
Richard Maw
Lars Wirzenius
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
| |
Make the message "Installing chunk..." be chatty, i.e., only displayed
when the user turns verbosity higher. Most of the time the chunk is
already unpacked in the cache, so installing it takes a fraction of
a second.
Add a new message, at default verbosity, when a chunk needs to be
unpacked. This can take a while, so a message is appropriate, so the
user knows what is happening.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
When you attempt to build a stratum or chunk, the ArtifactResolver can
return multiple root artifacts, since the root source produces multiple
artifacts.
Rather than having the BuildCommand complain that there's multiple root
artifacts, it now validates all the produced artifacts too, since that
will validate the kinds of artifacts produced, and give a more useful
error message, that you're trying to build a stratum or chunk directly.
If all the produced artifacts validate, then an exception is raised to
signal that it got multiple artifacts, when it only expected one.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bootstrap chunks don't make it into the final system, so there needs to
be an extra check for empty systems after the sources have been collected.
This was complicated slightly by the fact that if you try to build a chunk
directly you will have no strata in your sources, hence no non-bootstrap
chunks, but validation for having been told to build a chunk is best
handled later.
This amends the old yarns that depended on building a bootstrap chunk
and adds a new one that explicitly builds a system with bootstrap
chunks.
|
| | |
|
| | |
|
| | |
|
| |\
| |
| |
| |
| | |
Changed the message according to suggestion by Rob Kendrick,
supported by Daniel Silverstone.
|
| |/ |
|
| | |
|
| |
|
|
|
| |
This also removes the long-obsolete code to install staging fillers
in the staging area. We've not allowed users to do that for ages now.
|
| |
|
|
|
|
|
|
|
| |
This uses the same logic as when a build fails, so it's been
consolidated into `StagingArea.abort()`.
You could argue that if a build fails before any commands are run,
then there's nothing interesting to see, but it will be useful if the
hardlink/tarball extract algorithm fails in some corner case.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cross-bootstrap is a way to build baserock on an architecture that
does not currently have Baserock. It can be used by `morph
cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact
that can be used as a root filesystem with a basic build environment
with a script named `native-bootstrap` which will build and install
every chunk in the system.
If done with a devel system, this will give you a suitable environment
for building a proper Baserock system.
This does not currently provide a kernel for the target architecture.
Apart from adding the cross-bootstrap plugin, it also makes the
following changes:
* Moves the lit of valid_archs into morphlib (instead of locally-scoped
in MorphologyFactory)
* BuildCommand takes an extra argument, build_env
* split BuildCommand's get_artifact_object into create_source_pool and
resolve_artifacts (plus changes things that use get_artifact_object to
use the new way)
* setup_mounts finds out whether to do so by whether build_mode is
'staging', instead of by whether the setting 'staging-chroot' is true.
* Makes ChunkBuilder's get_sources use the
morphlib.builder2.extract_sources() method, and moved
set_mtime_recursively into morphlib.builder2, since it's not currently
used anywhere else.
* moved ChunkBuilder's get_commands into the Morphology class (plus
changes to anything that used get_commands)
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
This puts the information in the status prefix, so it appears
every time status() is used.
This also required the places where the name was manually spliced in
to be removed.
|
| |
|
|
|
| |
I find it easier to read without them, since there are less variables
to remember.
|
| |
|
|
|
|
| |
Now, inside the temporary directory we will have the following
subdirectories: chunks, staging, failed and deployments. The failed
directory will contain the staging areas of failed builds.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Define a specific set of 4 architectures that Morph supports, and only
expose that value to morphologies.
Since GNU triplets are very common we also expose a GNU triplet. Other
morphologies should work out their configuration based on MORPH_ARCH.
This commit also removes the morphlib.util.arch() function, which
detected the machine Morph is running on via 'uname -m'. Morph's
architecture names do not necessarily map to the output of 'uname -m'
so we should not rely on it anywhere.
|
| |
|
|
|
| |
We can no longer build strata individually because we don't know
what architecture they are for.
|
| |
|
|
|
|
|
| |
This gives us some forward compatibility leeway to introduce new
build modes, as long as they are compatible with staging.
As suggested by Richard Maw.
|
| |
|
|
|
|
|
|
|
|
|
| |
When building a stratum artifact it's easy to only include chunks that
were built in 'staging' mode. Constructing the staging area doesn't use
that code path, though, so we need an extra hack to filter out those
artifacts while building.
It may be neater to express stratum build-depends as actual
dependencies on the stratum artifact, rather than each of its
constituent chunks as we do now.
|
| |
|
|
|
|
|
|
|
| |
Morph no longer supports setting the prefix using the --prefix
argument / setting. This was only used in tests and during bootstrap.
If a chunk build-depends on a chunk within a stratum which has
a custom prefix, that prefix is appended to the PATH in the build
environment.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Allowed values:
staging: build with a staging chroot (default)
test: build with the host's tools
bootstrap: build with the host's tools, and do not include this
chunk in the final stratum artifact
In the past, 'normal mode' has been used to describe building a chunk
with the host's tools. We don't want that mode to ever be used,
because it is a huge hole in reproducability, but we need to keep it
around to avoid making Morph's cmdtest suite depend on Baserock.
Hopefully naming it 'test' should discourage potential abusers.
It is unfortunate that the build tests now take a separate code path
compared to real-world usage of Morph. However, this is necessary to
avoid a circular dependency between Morph's test suite and the
build-essential stratum in Baserock.
We do whole-build testing of Baserock, too, so the 'staging' code path
is still tested outside of Morph. However, testing a staging area
requires populating it with at minimum a working shell, and this is a
bit too complex to go in Morph's test suite.
|
| |
|
|
|
|
| |
In the future we will allow this to be modified to provide a
cross-bootstrap mode, but for now we use the same target as
the host compiler.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reorganise the build_artifact() and build_artifacts() functions to
allow more complex work when setting up chunk builds in
build_artifact().
The staging area now holds the BuildEnvironment object (the
environment variables that should be set during build). This makes sense
because all build commands should be run inside the staging area and
therefore through the StagingArea object.
The BuildEnvironment object is now considered immutable after it is
created. The environment is used in cache key computation when
computing what artifacts are required; if it changes after that point
we risk either computing different artifact keys for the same artifact
or missing data in the cache key that should be included in the hash.
Better to force changes into a separate 'extra_env' variable.
|
| |
|
|
|
| |
This involved having a staging area with split personalities
and was generally a bit ugly.
|
| |
|
|
|
|
| |
That means that bootstrapping Baserock is currently not possible with
this branch of Morph, but there's no reason it cannot be bootstrapped
using an older version of Morph instead.
|
| |
|
|
|
|
| |
Added chunk name in status when starting build
Added the name of parent artifact (chunk that installing chunk was installed for) to start of build message
|
| |
|
|
| |
This is mainly to fix bootstrap.
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
This adds a [Build 1/12765] to the output of the building of each
artifact. This makes it easier to see how much work there might
still be remaining.
|
| |/
|
|
|
|
|
|
| |
Building invididual chunks is bad because we have no idea what
build mode to use. Building individual strata would work, but all of
the stratum's build dependencies would need to be built first so there
would be little time saved in any case. There is also no way to test
them beyond checking that the build was successful.
|
| |
|
|
|
| |
Making the changes mentioned in the comment would be quite invasive,
and we don't have to do so quite yet.
|
| | |
|
