| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
This works towards allowing multiple concurrent builds in the same
system, which has the same problem as deployments.
This is the easy bit, since linux-user-chroot has support for bind
mounts and /proc mounts.
We also need to get rid of the /dev/shm mount to be able to build in
parallel though.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a build happens, it creates a new network namespace, and if this
happens while you have a disk image mounted, then you can't remove the
mount-point, because the other namespace is using it.
We can avoid the other namespace keeping this mount-point open by
creating the disk image in a private mount namespace, so it never sees
it.
The nicest way to do this is to have every extension run in a private
mount namespace, since you'd have to have extensions re-exec themselves,
since the appropriate system calls aren't exposed very well.
|
|\
| |
| |
| |
| |
| | |
Reviewed-by: Sam Thursfield
Reviewed-by: Pedro Alvarez
Reviewed-by: Daniel Silverstone
|
| |
| |
| |
| |
| | |
These were previously completely untested, so their unsafe mounting
wasn't noticed, but now both are fixed.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
All temporary mounts should be done inside a namespace, so that they
don't interfere with other namespaces.
system-integrations may need capabilities that regular builds don't
have, so they're chrooted, rather than linux-user-chrooted, which means
it's more complicated to do namespaces.
In the absence of a better command for it, we can do this with an
in-lined shell script.
This also stops us using the run-parts inside the system, and executes
the integrations directly.
|
| | |
|
| |
| |
| |
| |
| |
| | |
It now works when passed a file, rather than a command to run, and
ignores comments and set commands, so the generated preamble is
accepted.
|
|/ |
|
|\
| |
| |
| |
| | |
Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
|
|/
|
|
|
| |
Previously it was incorrect in the `morph help` output and in Morph's
man page.
|
|\
| |
| |
| |
| | |
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
|
| | |
|
|/
|
|
|
|
|
| |
There's no need because all strata live in the same Git repository now.
This message was added back when there could be multiple versions of
strata coming from different repos or different refs in the same repo
and life was very confusing.
|
|\
| |
| |
| |
| |
| | |
Reviewed-by: Richard Maw
Reviewed-by: Richard Ipsum
Reviewed-by: Sam Thursfield
|
| | |
|
| |
| |
| |
| |
| |
| | |
Since we're no longer adding anything to the queue we can replace it with a loop.
This patch also splits the code up a little bit.
|
| |
| |
| |
| |
| |
| |
| | |
Artifact resolution is no longer recursive, so there's no need to requeue
sources, removing this queue provides a significant speed up. Fixing an
issue where artifact resoltuion was taking too long on some systems
(several minutes in some cases).
|
| | |
|
|\ \
| | |
| | |
| | |
| | |
| | | |
Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
|
| |/
| |
| |
| | |
It was broken by the per-source building changes.
|
|\ \
| |/
|/|
| |
| |
| | |
Reviewed-by: Sam Thursfield
Reviewed-by: Jim MacArthur
Reviewed-by: Richard Ipsum
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A quirk in the resource cleanup code meant that if you gave the same
version label when deploying a new version, then it would fail, then
remove the old version, as it had assumed that it was the one to create
those directories.
This patch fixes this issue by making short context managers for all the
resource allocation, so cleanup is done by walking up the context
managers, so only the mount and the temporary directory need to be
cleaned up if the `mkdir "$VERSION_ROOT"` fails.
I've tested this with a deploy of a version that doesn't already exist,
and the version I'm currently running, so I can conclusively say it's
fixed that problem.
|
|\
| |
| |
| |
| | |
Reviewed-by: Pedro Alvarez
Reviewed-by: Richard Maw
|
| | |
|
| |
| |
| |
| |
| |
| | |
The original resolve_artifacts method is essentially unchanged,
it's just private now. This means that aside from this name change
the tests remain the same.
|
| |
| |
| |
| |
| |
| |
| |
| | |
This patch started off as an attempt to address the comment in
find_root_artifacts, though this patch doesn't impose an ordering on the list
of artifacts as the comment suggested. The artifact resolver now returns
a list of root artifacts which may make it easier to add multi-system builds
to morph.
|
|\ \
| | |
| | |
| | |
| | | |
Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk>
Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
It now gives an error message.
Previously it would fail with a backtrace like this:
2014-10-08 09:51:37 [systems/genivi-baseline-system-armv7lhf-jetson.morph][self]Removing temporary mounts
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run
self.process_args(args)
File "/src/morph/morphlib/exts/ssh-rsync.write", line 54, in process_args
self.upgrade_remote_system(location, temp_root)
File "/src/morph/morphlib/exts/ssh-rsync.write", line 107, in upgrade_remote_system
location, ['btrfs', 'subvolume', 'delete', orig_dir])
UnboundLocalError: local variable 'orig_dir' referenced before assignment
|
|\ \ \
| |/ /
|/| |
| | |
| | |
| | | |
Reviewed-by: Sam Thursfield
Reviewed-by: Richard Ipsum
Reviewed-by: Pedro Alvarez
|
| | | |
|
| | | |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
You need both bottle and flup for this to work, and it would wait
forever for a cache-server that is never going to start unless you have
bottle and flup.
So we now check whether the cache server is functional before attempting
distbuild tests.
|
| | |
| | |
| | |
| | |
| | |
| | | |
JSON can only handle unicode strings, but commands can write anything to
stdout/stderr, so we do the same trick as for the serialise, and json
encode yaml.
|
| | |
| | |
| | |
| | |
| | |
| | | |
The horrible json.dumped, yaml dump is because we need it to be both
binary safe (which yaml gives us) and one line per message (which json
gives us).
|
| | | |
|
| | | |
|
| | | |
|
| | |
| | |
| | |
| | | |
Deploying after distbuild currently broken
|
| | | |
|
| | | |
|
| |\ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This is a runtime dependency of distbuild, so you can't exactly test
distbuild without it.
The cache server itself depends on bottle and flup, so rather than
having a distbuild stratum and a trove stratum both containing the cache
server, we should split morph out of the tools stratum and include it in
both devel and trove systems.
|
| | |\ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Reviewed by:
Lars Wirzenius
Daniel Silverstone (on IRC)
Sam Thursfield (v4 of patch)
Richard Maw (on IRC)
|
| | |/ /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
With this we can request the state of a set of artifacts
in a single request. Artifacts are sent as a json array.
We check whether each artifact is in the cache or not
and send our findings back to the client as a json object.
|
| | | | |
|
| | |\ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Reviewed by:
Pedro Alvarez
Daniel Silverstone
|
| | |/ /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
lighttpd does not support WSGI, so we need to be able to run
the server in FCGI mode. The flup package provides one.
Running with the default bottle server is still useful
for testing purposes.
|
| | |\ \
| | | | |
| | | | |
| | | | | |
Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk
|
| | |/ / |
|
| | |\ \
| | | | |
| | | | |
| | | | | |
'origin/jannispohlmann/add-batch-sha1-and-file-queries'
|
| | |/ /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This commit adds support for resolving multiple refs into SHA1s and
requesting multiple files at once, each using a single POST request.
The (repo, ref) and (repo, ref, filename) parameters are passed to
the POST requests as JSON dictionaries in a list. The response to
both types of requests is a JSON list with the same dictionaries
again, to which a "sha1" and "data" field are added, respectively.
The file contents returned by "POST /1.0/files" requests are
base64-encoded and need to be decoded at the receiver's end. This
is because the contents may be binary or contain quotes and
therefore cause JSON syntax errors.
|