| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since git's parsing of .gitattributes can't deal with spaces, or
even with quotes, replace any whitespace with [[:space:]]. Also,
make sure to escape any glob metacharacters to avoid accidental
pattern matches.
Note:
This approach could cause a file "foo<tab>bar" to be treated
as a binary if a file "foo bar" has already been added with
morph add-binary. However this is as good of a workaround as
I can find.
Signed-off by: Adam Coldrick
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
morphlib/plugins/deploy_plugin.py
without-test-modules
Reviewed by:
Richard Maw
Lars Wirzenius
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
tarfile's open needs the file-like object
to have a tell() method, objects returned
from sockets don't have this method.
So instead we fetch the artifact from the
remote and cache it locally
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
The constructor for BuildCommand sets up the caches,
we want the caches to be set up for distbuild too
so that we can deploy a system (system artifact will be
fetched from the artifact cache)
|
| | |
| |
| |
| |
| | |
We want to be able to transfer all
source artifacts in a single transaction
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Whenever the controller finds a source artifact
it wants to build, it changes its state to BUILDING.
We build all a chunk's source artifacts in one go.
So for any chunk artifact, we change the state of
all chunk artifacts that are built from the
same source to BUILDING
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Serialisation was simple when we only had 1 artifact per source.
However, to allow smaller systems, we need artifact splitting to produce
multiple artifacts per chunk source.
So now the new serialisation format has a separate list of artifacts
and sources, rather than the Source being generated from the artifact's
serialisation.
Python's id() function is used to encode the references between the
various Sources and Artifacts, these are replaced with a reference to
the new object after deserialisation.
Previously the cache-key was used, but this is no longer sufficient to
uniquely identify an Artifact.
The resultant build graph after deserialisation is a little different
to what went in: Strata end up with a different Source per Artifact,
so it _is_ a 1 to 1 mapping, as opposed to Chunks, where it's many to 1.
We serialise strata and chunks differently because stratum artifacts
from the same source can have different dependencies, for example
core-devel can have different dependencies to core-runtime.
Without intervention we would serialise core-devel and core-devel's
dependencies without including core-runtime's dependencies.
To solve this we've decided to encode stratum artifacts completely
indepedently: each stratum artifact has its own source. This is safe
because stratum artifacts can be constructed independently,
as opposed to Chunks where all the Artifacts for a Source
are produced together.
This is a little hacky in its current form, but it simplifies matters
later in distbuild with regards to how it handles expressing that
every Artifact that shares a Source is built together.
Arguably, this should be the output of producing the build graph
anyway, since it more helpfully represents which Artifacts are built
together than checking the morphology kind all the time, but more
assumptions need checking in morph before it's safe to make this
change across the whole of the morph codebase.
|
| | |
| |
| |
| | |
Worker name is not sent in message
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A semantic error in the BuildBranch class meant that it was not possible
to push temporary branches.
This escaped testing since BuildBranch interacts too tightly with
other components to be easily unit-tested, so testing was deferred to
a yarn test.
However, coverage isn't measured in yarn tests, so this code path was
forgotten.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
This function causes a UnicodeDecodeError for some repositories
when building. Use os.path.isfile() when looking for .gitfat instead.
|
| |\ \
| | |
| | |
| | |
| | | |
Reviewed-By: Daniel Silverstone (on IRC)
Reviewed-By: Richard Maw (on IRC)
|
| | | | |
|
| |/ /
| |
| |
| |
| | |
We were building it from a variable in some places and hardcoding it in
others; now we build it from a variable everywhere.
|
| |\ \
| | |
| | |
| | |
| | |
| | | |
Reviewed by:
Richard Maw
Lars Wirzenius
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Move some code to the '.check' extension to verify that the deployment
can happen *before* spending 5 minutes unpacking and configuring the
rootfs.
This is not a perfect solution yet because when multiple systems are
being deployed in a cluster, we do not check all systems and then
deploy them all. Instead, we check one, then deploy it, then check the
second, etc.
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
There is no longer a default /etc/fstab in the Baserock fhs-dirs chunk,
and the nfsboot.write extension does not use the default Btrfs system
layout so no entry is added for / to /etc/fstab at deploy-time.
We cannot have / in /etc/fstab for nfsboot deployments because it
causes systemd to remount / during bootup, which breaks everything.
|
| |\ \ \
| | | |
| | | |
| | | | |
Reviewed-by: Richard Maw
|
| | | | | |
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The scripts will be created into the folder:
/baserock/sytem-integration/
The system integration commands have the following
syntax:
name: linux
kind: chunk
...
install-commands:
...
system-integration:
linux-libs:
00-depmod:
- depmod -a
70-more-integration:
- touch /baserock/FILE
- |
for FOLDER in $(ls /)
do
echo "$FOLDER"
done
linux-misc:
70-more-integration:
- echo "Hello world"
In this concrete example, the following files will be created:
$DESTDIR/baserock/system-integration/
00-depmod-linux-libs-0000:
#!/bin/sh
set -xev
depmod -a
70-more-integration-linux-libs-0000
#!/bin/sh
set -xev
touch /baserock/FILE
70-more-integration-linux-libs-0001
#!/bin/sh
set -xev
for FOLDER in $(ls /)
do
echo "$FOLDER"
done
70-more-integration-linux-misc-0000
#!/bin/sh
set -xev
echo "Hello world"
|
| | | | | |
|
| | |/ / |
|
| |\ \ \
| |/ /
|/| |
| | |
| | |
| | |
| | |
| | |
| | | |
Author: Adam Coldrick <adam.coldrick@codethink.co.uk>
Reviewed by:
* Richard Maw <richard.maw@codethink.co.uk>
* Lars Wirzenius <lars.wirzenius@codethink.co.uk>
* Richard Ipsum <richard.ipsum@codethink.co.uk>
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Add a plugin to implement both `morph push` and `morph pull`.
These commands are wrappers around the corresponding git commands
push and pull, which also implement the functionality of pushing
and pulling large files provided by git-fat.
For example, running `morph pull` will pull any commits from the
remote branch not on your local branch, and then pull any large
files from the separate git-fat/rsync store on the Trove.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Add a plugin which implements the morph add-binary command. This
command is used to add large files to a git repository. It sets
up the files needed to use git-fat, and then runs `git add` with
git-fat initiated.
|
| | | |
| | |
| | |
| | |
| | |
| | | |
When cloning a repository, the files stored using git-fat need to
be pulled. This situation occurs in `morph branch`, `morph edit`,
and `morph checkout`.
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
These commands are:
* git fat <init|push|pull>
* git pull
They are required for the morph add-binary and push/pull plugins. Also
make sure that GitDirectory is working in the root directory of the
specified git repository, and add some helper functions for handling
paths of files in the working tree.
|
| |\ \ |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This hard-coded the expected artifact key, which is dependent on
architecture.
Since we don't care about the output, so much as it failing, let's just
discard it.
|
| | | |
| | |
| | |
| | |
| | | |
This is tested in yarns already, and this test breaks the test suite on
any architecture but x86_64.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
They hard-code the expected sha1, so break the test suite for anything
but x86_64.
If we decide tag has a place in the new world order, yarns can be made,
but until then, we're better off without these tests.
|
| | | |
| | |
| | |
| | | |
Older versions of btrfs fail with just 10M.
|
| | | |
| | |
| | |
| | |
| | | |
uname tends to only give us a valid morph architecture on x86_64,
this makes it work on other architectures.
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| | |
./check with no arguments is as-before, similarly ./check --full,
but now you may also specify individual tests to run.
So just the style check is `./check --style`. Everything but style
is `./check --full --no-style`.
I found this convenient when working on the test suite.
|
| |\ \
| | |
| | |
| | | |
Reviewed-by: Richard Ipsum and Lars Wirzenius
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This reverts commit f366960273b026322f7e7cc3c1eb0cd632ebc73e.
These changes break building on x86_64, which is our main development
platform.
Better patches will be forthcoming later.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
uname will return the same string morph expects as an architecture on
x86_64, but the canonical x86_32 architecture names match i?86 instead.
To make it work on 32-bit, we replace calls to uname with morph
print-architecture, which does the translation for us.
|
| | | | |
|
| | | |
| | |
| | |
| | | |
This also makes it obey status prefix
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This moves the deployed system to somewhere on the host.
Any existing contents of the directory is deleted, so don't try to be
clever and deploy a new system on top of / in place of a proper upgrade.
It can be used to deploy a chroot, sysroot or container, but its
current use is to allow for nested deployments to include another
system in itself, since the parent deployment's "$1" is prepended to
the sub-deployment's "$2".
|
| | | |
| | |
| | |
| | |
| | |
| | | |
These define a vocabulary for altering a cluster morphology's fields,
since defining an implementation that is only used by one scenario for
setup is cumbersome.
|
