| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
This is required for reproducing the morphologies that created the
artifacts.
It is possible, but expensive, to `unexpand` a url, provided your
repository aliases are the same as the morphology's, but this may
not always be the case.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For Morph to work it's necessary for the staging area to contain a
full FHS heirarchy. However, if the staging area is empty we want to
give the user an obvious error. Previously we would say something like:
OSError: [Errno 2] No such file or directory: '/tmp/ccache'
This makes it seem like the problem is with ccache. With this patch,
we will instead get as far as:
/bin/sh: No such file or directory
This should make the real problem clearer.
Also, aborting while trying to mount the ccache directory causes the
/proc and /dev/shm mounts to persist which is fixed by this patch.
|
| |
|
|
|
|
| |
This can be used to rebuild an artifact with the exact same version
of morph. Eventually there may be a tool to do this, but it allows
for expert diagnosis until that point.
|
| |
|
|
|
|
|
|
|
| |
This patch makes morph store its ccaches by the url's basename instead
of using the whole url. This fixes the problem of morph choking when it
tries to mount a directory with a colon in its path, and also makes
all clones of one repository use the same cache (probably). This will
cause projects with the same name to use the same cache, but that is
not likely to cause serious problems.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is set up so that each individual project repository has its own
ccache, all under one defined directory.
The top-level ccache directory is added as the setting
'compiler-cache-dir', and defaults to $cachedir/ccache.
When a build is performed, this will bind-mount a project's ccache into
the /tmp/ccache of the staging-area and set up the environment variables
so that ccache will be used (if appropriate executables are installed
to /usr/lib/ccache in the staging-area).
In addition, this removes code for ccache-remotedir, as it is unrelated
to this implementation of ccache.
Reviewed-by: Jannis Pohlmann <jannis.pohlmann@codethink.co.uk>
Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit c859d70d86423a52bc7053abf64e9ca21f62a487, reversing
changes made to 3085a672d1e8b5177be33f0463385de72a0ef5bf.
Unfortunately, hardlinking and linux-user-chroot both break builds
in various ways. Hardlinking breaks the bootstrap process by creating
symlinks like /usr/libexec that can then not be overwritten with real
files by install scripts from morphologies like gcc.
linux-user-chroot caused problems by breaking privileged operations such
as chgrp and CAP_SETFCAP. As a consequence, chunks like util-linux and
libcap can no longer be built.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The patch gives two things.
Improves morph build time by reusing decompressed files of
chunks/stage-fillers with hardlinks from the chroot. Rather than
decompressing each time into each chroot.
Original: real 5h 17m 47s
Hardlink: real 2h 52m 27s
It uses linux-user-chroot to create the chroot and make all but
the basics readonly.
|
| |
|
|
|
|
|
| |
gzip files can store the name and date of the uncompressed original, and
this is displayed in some GUI archive managers, or when gunzip -N is used.
In our case the original filename is something like 'tmpXsgaY15', which
is useless information and serves only to confuse.
|
| |
|
|
| |
Includes new test.
|
| |
|
|
|
|
|
| |
This prevents us from leaving file handles open when code throws an
exception. When the file handle is on a loopback mount, this is a real
problem because Morph then cannot unmount the image in its exception
handler. In most cases 'with' is the best option.
|
| | |
|
| |
|
|
|
|
| |
This is intended to for human consumption, with the full set of
metadata in /baserock/system-artifact-name.meta. It currently lists
system name, the symbolic ref of the system morphology and build date.
|
| |
|
|
|
|
|
|
| |
This makes it easier to identify what version of a system we are running.
Tests are updated to check inside the contents of the rootfs we created,
and the code to mount the rootfs as a loopback device was extracted out
into tests.as-root/lib
|
| |
|
|
|
| |
This fixes a bunch of tests so they pass on squeeze, which has a version
of Python whose GzipFile doesn't support the "with" protocol.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Since tarball rootfs images are compressed with gzip -1, we should do
the same for disk images. This means they'll need decompressing before
they can be used as disk images, but it means we'll be shunting around
something closer to their true data size rather than the disk image size.
For example, a 2G development system image compresses down to around
380M which is a lot faster to transfer and doesn't risk de-sparseification.
|
| |
|
|
|
|
|
|
| |
This reworks the code for managing and using the git caches in morph to treat
the caches as bare repositories which are mirrors of where we clone from. In
addition we correctly prune the branches during updates, so that we don't end
up accumulating pointless branches over and over. This is even more important
with branch-and-merge generating temporary build refs for things.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Unfortunately we were running into issues with unicode filenames
in repositories causing set_mtime_recursively() to explode.
This fixes the problem.
Reviewed-By: Lars Wirzenius (In real life)
|
| |
|
|
|
|
|
|
|
| |
This kind of reverts commit: c4facff97b27968f00a75ffb3d49cea2080ad9a4
Currently our bootstrap depends on building an empty stratum; this
should be changed so that it builds a tarball system image, but this
is currently broken too. This commit should be reverted when bootstrap
is fixed to not depend on building an empty stratum.
|
| |
|
|
|
| |
This is mostly useful for diagnosing Morph bugs. Currently empty strata
just cause confusing errors when building systems.
|
| |
|
|
|
|
|
|
| |
The squeeze version of coverage.py seems to have problems with methods
that consist of a single pass statement. This patch replaces those
statements with placeholder docstrings so that ./check passes on squeeze.
Reviewed-By: Daniel Silverstone (on irc)
|
| |
|
|
|
|
|
| |
This adds a disk system image kind which does everything the syslinux-disk
one does, but without syslinux. It deliberately carries stubbed bootloader
operations so that we can later make syslinux-disk inherit from this one
and override it.
|
| |
|
|
|
|
|
|
|
| |
This was done with the aid of the pep8 script, available by running
`easy_install pep8`.
It may be worth making this part of ./check, but that will require
putting pep8 into the development tools stratum.
This should be easy, given pep8 has no external dependencies.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
Also, rename the syslinux-disk builder plugin file to make it clear
it's a system builder plugin.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Normally, the build log would go into the artifact cache, but it's tedious
to fetch from there, so we log it into the normal morph log as well if
things fail.
|
| |
|
|
|
|
| |
This should never happen any more, so the check is now redundant.
This may have a negligible performance benefit, but it prevents the
code implying that None is a valid architecture.
|
| |
|
|
|
|
|
|
|
|
| |
This is required for systemd's journald to start.
This is probably a bad dependency in systemd, trying to start the
journal before it has mounted everything properly.
This required a compat change, it is a string to make it more
noticeable that it's a temporary version.
|
| |
|
|
|
| |
Data buffers can get confused if two file handles are writing to the
same file, python's subprocess module has subprocess.STDOUT for this.
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
fstab had errors=remount-ro, this is an ext option, btrfs doesn't
support it.
However it should be mounted noatime, to prevent the metadata
trees being duplicated because they were read.
extlinux.conf also has its mounting options changed so that the
rootfs is mounted read-only initially. This should prevent the
metadata trees being updated by reading them until it is remounted
with noatime.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Also, some bug fixes.
Also, when handling a BaseException, log the exception (with traceback)
that we're handling, in case there is an error while handling it, because
the second error will otherwise mask the first one.
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
The new method takes a list of keyword arguments. This is more useful
than the old way of giving just a string, since now the presentation
layer may transmogrify the status update. For example, it can usefully
translate the message to another language.
Add --verbose option to allow more control over what the user sees.
|
| |/
|
|
|
|
|
| |
The kernel should be considered a full artifact, not just metadata.
The rootfs is now suffixed by -rootfs and the kernel is named
$name-kernel. This is similar to how chunk splitting works.
|
| |
|
|
|
|
| |
This required refactoring mount_proc and unmount_proc into more
generic mounting functions.
do_mounts returns the paths that were mounted instead of just one
|
| |
|
|
|
| |
This change is purely cosmetic, but reflects the fact that we need
to mount more than just proc sometimes.
|
| | |
|
