| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
| |
This only adds tests for the bits which were moved from
morphologyfactory into sourceresolver, namely detection
of build systems and the '_get_morphology()' function.
These are just the morphologyfactory tests reworked
slightly to work properly with the modified API.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Most morphologies involved in a build are in the definitions repo these
days. Currently we read each of them using `git cat-file`, which is
slow. It's quicker to check out all the files in one go to a temporary
directory and then read them from there.
With the current workflow users often have definitions.git checked out
on disk. It seems strange to not just read the files from there. There
are two reasons why I don't want to do that yet:
- there are commands which don't run inside a system branch, which
would be broken if we expected to always be in a system branch
- there may be local changes in the checked-out repo, and it takes
around 5 seconds on each build to check if there aren't any local
changes. It actually seems faster to just check out a known clean
version from the cache.
|
|
|
|
|
|
| |
This is nice because it's fast. We don't have to copy all the Git
history along with it like we do with a clone. And it doesn't touch
any files in the cached repo.
|
| |
|
|
|
|
|
|
|
| |
This will speed up builds of chunks which don't have a chunk
morph. It won't have much (if any) effect on the speed of the
first build, but subsequent builds will be much faster as we
won't have to query the artifact cache.
|
|
|
|
|
|
|
|
|
|
|
|
| |
This uses the PyLRU module, from:
<https://pypi.python.org/pypi/pylru/1.0.6>.
Python 3.2 and newer provide a built-in LRU cache, but this is
specifically for in-memory use. See <http://bugs.python.org/issue17528>.
Git commits are immutable, so caching information about their contents
is fairly easy and trouble-free. There's no danger of the cache becoming
stale.
|
| |
|
|
|
|
|
| |
There's no need for this stuff to be in a separate class. This allows
integrating it with the caching in the SourceResolver class.
|
| |
|
|
|
|
|
|
|
| |
We will use MorphologyLoader.load_from_file() to load morphologies
from a temporary file, so the output will be unpredictable. That
will break this test, so we may as well convert it into a yarn
to fix it.
|
|
|
|
|
|
| |
This commit fixes an issue where running morph's yarn tests would
leave behind a number of scripts/git-daemon-wrap processes due to
some tests not stopping their git server properly.
|
|\
| |
| |
| |
| |
| |
| |
| | |
Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk>
Reviewed-By: Rob Kendrick <rob.kendrick@codethink.co.uk>
Reviewed-By: Paul Sherwood <paul.sherwood@codethink.co.uk>
Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
Reviewed-By: Zara Zaimeche <zara.zaimeche@codethink.co.uk>
|
|/ |
|
| |
|
|\
| |
| |
| |
| | |
Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Not everyone is a fan of the `morph build` magic that collects up your
changes and puts them in a temporary branch. Now you can disable it by
setting 'local-changes=ignore' in your morph.conf file.
This speeds up `morph build` and `morph deploy` by 5-10 seconds on my
machine.
I looked an option to make `morph build` warn if there are uncommitted
changes. I found that with a cold cache, it takes about 5 seconds on my
machine to verify that there are no uncommitted changes to a checkout of
definitions.git. That defeats the main purpose of this patch for me, so
I didn't include the option.
|
| | |
|
| | |
|
| | |
|
|\ \
| |/
|/|
| |
| | |
Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
|
| |
| |
| |
| |
| | |
There is still code here that duplicates stuff the GitDirectory class
should be doing, I think.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This consolidates a bunch of code paths that were previously duplicated.
This also changes the API for local cached repos to match the
function names GitDirectory uses. Note that the remote repo cache still
uses the old names, and should be fixed when time permits.
Some unit tests that use the CachedRepo module required a bit of
inelegant monkey-patching in order that they continue to work. A better
way to do this would be with the 'mock' library (which would need to be
added to Baserock 'build' and 'devel' systems before we could use it).
|
| | |
|
|\ \
| | |
| | |
| | |
| | | |
Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk>
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
|
| | | |
|
|\ \ \
| | | |
| | | |
| | | |
| | | | |
Reviewed-By: Paul Sherwood <paul.sherwood@codethink.co.uk>
Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Ensure that
a) the deployment directory must not exist
b) the extension can not be used to upgrade a system
|
|\ \ \ \
| |/ / /
|/| | |
| | | |
| | | | |
Reviewed-By: Paul Sherwood <paul.sherwood@codethink.co.uk>
Reviewed-By: Mike Smith <mike.smith@codethink.co.uk>
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This avoids writing each command to the log file twice, as we did
previously. Also, the user-visible message is now only constructed if
we are definitely going to write it to the screen (a tiny optimisation).
Hopefully the logic is clearer now too.
|
|/ / /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
We were attempting to display commandlines with correct shell-escaping,
but the logic was super broken so users would end up seeing this sort of
thing instead:
2015-01-02 10:25:42 # g | i | t | | r | e | v | - | p | a | r | s | e
| | - | - | v | e | r | i | f | y | | ' | 9 | 8 | f | e | a | 8 | 7
| b | 7 | 2 | 7 | 2 | 5 | 3 | e | 7 | f | f | 8 | 1 | 0 | 5 | 4 | 3 | 4
| c | 9 | e | a | 9 | 0 | 2 | b | b | 6 | a | 6 | f | 7 | e | ^ | { | c
| o | m | m | i | t | } | '
Commandlines should now display as intended, more like this:
2015-01-02 10:57:17 # git rev-parse --verify
'9df9643842e4b4d8ece710fe6105f32fa38a0d22^{commit}'
This broken logic was introduced as a post-review fixup in merge commit
c57952ef44a0f1f161441970fcf2f27a39b0de7c.
|
|\ \ \
| | | |
| | | |
| | | |
| | | |
| | | | |
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
|
|/ / /
| | |
| | |
| | |
| | |
| | | |
Also add support to allow building compatible architectures on armv8
machines, as per the rationale in the comment in
_validate_architecture().
|
|\ \ \
| |/ /
|/| |
| | | |
Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
|
| | |
| | |
| | |
| | |
| | |
| | | |
There's no chroot used in bootstrap mode, so the host's /dev/shm can be
used, and nothing in the world will be looking to
/src/tmp/staging/tmpXXX/dev/shm for anything.
|
| | | |
|
|\ \ \
| | | |
| | | |
| | | |
| | | | |
Reviewed-By: James Thomas <james.thomas@codethink.co.uk>
Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | | |
| | | |
| | | | |
parameters
|
| |/ / |
|
|\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
Reviewed-By: Mike Smith <mike.smith@codethink.co.uk>
|
| | | | |
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Morph tells the user that an error occurred in the staging area, then
moves the staging area somewhere else. Giving the old path rather than
the new path is pretty annoying.
|
| | | | |
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This affects errors encountered at build time and at system-integration time.
New errors look like this:
ERROR: Command failed: baserock/system-integration/02-install-gerrit-gerrit-installation-binaries-misc-0000:
Containerisation settings: {'mounts': (('dev/shm', 'tmpfs', 'none'), ('tmp', 'tmpfs', 'none')), 'mount_proc': True, 'root': '/var/tmp/staging/tmp1YQ2yN/minimal-system-x86_64-generic.inst'}
Error output:
+ install -D /usr/share/gerrit/gerrit-2.9.war /home/gerrit2/gerrit/gerrit-2.9.war -o gerrit2 -g gerrit2 -m 644
install: can't change ownership of /home/gerrit2/gerrit/gerrit-2.9.war: Operation not permitted
Previously the error message would have been this:
Command failed: unshare --mount -- sh -ec.
mount --make-rprivate /
root="$1"
shift
while true; do
case "$1" in
--)
shift
break
;;
*)
mount_point="$1"
mount_type="$2"
mount_source="$3"
shift 3
path="$root/$mount_point"
mount -t "$mount_type" "$mount_source" "$path"
;;
esac
done
exec "$@"
- /var/tmp/staging/tmppeA1Iw/gerrit-x86_64.inst/ dev/shm tmpfs none tmp tmpfs none -- linux-user-chroot --chdir . --mount-proc proc
/var/tmp/staging/tmppeA1Iw/gerrit-x86_64.inst/ baserock/system-integration/02-install-gerrit-gerrit-installation-binaries-misc-0000
+ install -D /usr/share/gerrit/gerrit-2.9.war /home/gerrit2/gerrit/gerrit-2.9.war -o gerrit2 -g gerrit2 -m 644
install: can't change ownership of /home/gerrit2/gerrit/gerrit-2.9.war: Operation not permitted
|
| |/ /
|/| |
| | |
| | | |
I meant to do this as part of the previous merge.
|
|\ \ \
| | | |
| | | |
| | | |
| | | | |
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
|
| |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
In order to check the user's credentials at the start of deployment, we
try to run `glance image-list`. I found a situation where this command
failed despite my credentials being correct. Morph outputted a
misleading error message that said 'Wrong OpenStack credentials'
The code now checks that the error returned by 'glance' does indeed
look like a credentials error. If it doesn't, the full error output is
displayed. The error I encountered now gets a message like this:
ERROR: openstack.check failed with code 1: ERROR: Failed to connect
to OpenStack instance at https://example.com:5000/v2.0:
[('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify
failed')]
(If you are curious, I fixed this by running `update-ca-certificates`.)
|