summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * stagingarea: Remove vestigial pre-command mounting logicRichard Maw2014-10-301-22/+2
| |
| * Make build commands use containerised_cmdlineRichard Maw2014-10-301-31/+14
| |
| * Make deployment extensions use unshared_cmdlineRichard Maw2014-10-301-4/+1
| |
| * Make system-integration commands use containerised_cmdlineRichard Maw2014-10-301-55/+5
| |
| * Move unsharing and containerising logic to utilRichard Maw2014-10-301-0/+114
| | | | | | | | | | This way the build commands, system integration commands and deployment extension commands can all share the logic.
| * Allow invert_paths to skip the top level directoryRichard Maw2014-10-291-0/+5
|/ | | | | | We're going to be passing it "$CHROOT/" as a writable directory by default, so it needs to be able to tell that it means to leave everything writable.
* Fix lines longer than 79 charactersSam Thursfield2014-10-292-3/+4
|
* Merge remote-tracking branch 'origin/sam/build-fix-3'Sam Thursfield2014-10-295-17/+38
|\ | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * Fix distbuild to allow passing a commit instead of a named ref to be builtSam Thursfield2014-10-275-17/+38
| | | | | | | | | | | | | | | | | | | | | | The recent changes to the BuildCommand.build() function caused distbuild to break, because I didn't make the same change to the InitiatorBuildCommand.build() function but did change how it was called. This commit adds the ability to have optional fields in distbuild messages. This is used to add an optional 'original_ref' field, which will get passed to `morph serialise-artifact` by new distbuild controllers, and will be ignored by older ones.
* | Make pushed_build_branch always return (url, commit, ref)Richard Ipsum2014-10-281-2/+7
|/
* Merge branch 'baserock/ps/message-fix'Sam Thursfield2014-10-271-1/+1
|\ | | | | | | | | Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk> Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
| * Fix confusing messagebaserock/ps/message-fixPaul Sherwood2014-10-271-1/+1
| |
| * Merge branch 'master' of git://git.baserock.org/baserock/baserock/morph into ↵Paul Sherwood2014-10-2713-100/+229
| |\ | |/ |/| | | HEAD
* | Merge branch 'sam/build-fix-2'Sam Thursfield2014-10-275-26/+49
|\ \ | | | | | | | | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk> Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * | build: Resolve the ref being built in the user's local definitions.gitSam Thursfield2014-10-243-7/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most usefully, this patch means that Morph no longer updates its cached copy of definitions.git every time you run `morph build`. Also, it prevents confusion in the following situation. Imagine I have run: morph checkout baserock:baserock/definitions master I then wait a while, during which time someone pushes to 'master' in the definitions.git repo that I cloned from. Now I run: cd master morph build systems/whatever.morph Which commit does it build, the local head of 'master' or the remote head of 'master'? The answer, both before and after this patch, is that it builds the local version of master. But previously, this only happened because of the magic that we have to detect local changes. With this patch, the local change detection could be disabled and `morph build` would still build what the user had checked out as 'master' locally, not whatever 'master' pointed to in the remote repo.
| * | build: Allow passing in the name of the original ref to create_source_pool()Sam Thursfield2014-10-242-7/+15
| | | | | | | | | | | | | | | This means that we can force the building of a specific commit without losing the original branch name in the metadata of the resulting system.
| * | build: Restructure how BuildCommand.build() is calledSam Thursfield2014-10-242-15/+19
|/ / | | | | | | | | | | | | | | | | Rather than take a list of triplets to build, the BuildCommand.build() function now takes a single repo/ref/morph triplet. Iterating through multiple sets of triplets is now done in the build plugin. There are a couple of cosmetic changes to the status output at the start and end of a build as a result.
* | Merge branch 'baserock/richardmaw/parallelism-improvements'Richard Maw2014-10-243-11/+23
|\ \ | | | | | | | | | | | | | | | | | | | | | This stripped out the commit from the patch series on the mailing list to remove /dev/shm. Reviewed-by: Sam Thursfield Reviewed-by: Daniel Silverstone
| * | stagingarea: Mount proc and ccache inside the namespaceRichard Maw2014-10-242-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This works towards allowing multiple concurrent builds in the same system, which has the same problem as deployments. This is the easy bit, since linux-user-chroot has support for bind mounts and /proc mounts. We also need to get rid of the /dev/shm mount to be able to build in parallel though.
| * | deploy extensions: Don't crash if someone builds at the same timeRichard Maw2014-10-241-1/+3
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | If a build happens, it creates a new network namespace, and if this happens while you have a disk image mounted, then you can't remove the mount-point, because the other namespace is using it. We can avoid the other namespace keeping this mount-point open by creating the disk image in a private mount namespace, so it never sees it. The nicest way to do this is to have every extension run in a private mount namespace, since you'd have to have extensions re-exec themselves, since the appropriate system calls aren't exposed very well.
* | Merge branch 'baserock/richardmaw/test-system-integrations'Richard Maw2014-10-245-54/+152
|\ \ | | | | | | | | | | | | | | | Reviewed-by: Sam Thursfield Reviewed-by: Pedro Alvarez Reviewed-by: Daniel Silverstone
| * | yarns: Add yarns for system-integrationsRichard Maw2014-10-242-0/+38
| | | | | | | | | | | | | | | These were previously completely untested, so their unsafe mounting wasn't noticed, but now both are fixed.
| * | Run system-integrations in a namespaceRichard Maw2014-10-241-23/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All temporary mounts should be done inside a namespace, so that they don't interfere with other namespaces. system-integrations may need capabilities that regular builds don't have, so they're chrooted, rather than linux-user-chrooted, which means it's more complicated to do namespaces. In the absence of a better command for it, we can do this with an in-lined shell script. This also stops us using the run-parts inside the system, and executes the integrations directly.
| * | yarns: Fix missing FINALLYRichard Maw2014-10-241-0/+1
| | |
| * | test-shell: Allow to be used in system-integrationsRichard Maw2014-10-241-39/+49
| | | | | | | | | | | | | | | | | | It now works when passed a file, rather than a command to run, and ignores comments and set commands, so the generated preamble is accepted.
| * | test-shell: add create file commandRichard Maw2014-10-241-0/+20
|/ /
* | Merge branch 'sam/fix-edit-synopsis'Sam Thursfield2014-10-221-1/+1
|\ \ | | | | | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * | Fix synopsis of `morph edit` commandSam Thursfield2014-10-221-1/+1
|/ / | | | | | | | | Previously it was incorrect in the `morph help` output and in Morph's man page.
* | Merge remote-tracking branch 'origin/sam/validation-fixes'Sam Thursfield2014-10-171-8/+4
|\ \ | | | | | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
| * | Give clearer error when the same 'name' is used by two strataSam Thursfield2014-10-161-5/+4
| | |
| * | Don't print the ref of each stratum in the buildSam Thursfield2014-10-161-3/+0
|/ / | | | | | | | | | | | | There's no need because all strata live in the same Git repository now. This message was added back when there could be multiple versions of strata coming from different repos or different refs in the same repo and life was very confusing.
| * Merge branch 'master' of git://git.baserock.org/baserock/baserock/morph into ↵Paul Sherwood2014-10-140-0/+0
| |\ | |/ |/| | | HEAD
* | Merge branch 'baserock/richardipsum/arfix'Richard Maw2014-10-131-74/+51
|\ \ | | | | | | | | | | | | | | | Reviewed-by: Richard Maw Reviewed-by: Richard Ipsum Reviewed-by: Sam Thursfield
| | * Merge remote-tracking branch 'origin/baserock/richardipsum/arfix' into HEADPaul Sherwood2014-10-111-74/+51
| | |\ | |/ / |/| / | |/
| * Simplify _resolve_system_artifactsRichard Ipsum2014-10-101-12/+4
| |
| * Replace resolver queue with loopRichard Ipsum2014-10-101-62/+57
| | | | | | | | | | | | Since we're no longer adding anything to the queue we can replace it with a loop. This patch also splits the code up a little bit.
| * Remove queueingRichard Maw2014-10-101-16/+6
| | | | | | | | | | | | | | Artifact resolution is no longer recursive, so there's no need to requeue sources, removing this queue provides a significant speed up. Fixing an issue where artifact resoltuion was taking too long on some systems (several minutes in some cases).
* | Remove out of date commentRichard Ipsum2014-10-101-3/+0
| |
* | Merge branch 'sam/fix-list-artifacts'Sam Thursfield2014-10-101-6/+7
|\ \ | | | | | | | | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * | Fix the `morph list-artifacts` commandSam Thursfield2014-10-101-6/+7
| |/ | | | | | | It was broken by the per-source building changes.
* | Merge branch 'baserock/richardmaw/fix-ssh-rsync-destroying-versions'Richard Maw2014-10-101-77/+105
|\ \ | |/ |/| | | | | | | Reviewed-by: Sam Thursfield Reviewed-by: Jim MacArthur Reviewed-by: Richard Ipsum
| * ssh-rsync: Don't delete version if it existsRichard Maw2014-10-101-77/+105
|/ | | | | | | | | | | | | | | | A quirk in the resource cleanup code meant that if you gave the same version label when deploying a new version, then it would fail, then remove the old version, as it had assumed that it was the one to create those directories. This patch fixes this issue by making short context managers for all the resource allocation, so cleanup is done by walking up the context managers, so only the mount and the temporary directory need to be cleaned up if the `mkdir "$VERSION_ROOT"` fails. I've tested this with a deploy of a version that doesn't already exist, and the version I'm currently running, so I can conclusively say it's fixed that problem.
* Merge remote-tracking branch 'origin/baserock/ps/log-chunk-build-times'Richard Maw2014-10-091-0/+8
|\ | | | | | | | | Reviewed-by: Pedro Alvarez Reviewed-by: Richard Maw
| * Log elapsed time for each chunk buildbaserock/ps/log-chunk-build-timesPaul Sherwood2014-09-271-0/+7
| |
* | Update artifact resolver and ckc testsRichard Ipsum2014-10-092-9/+9
| | | | | | | | | | | | The original resolve_artifacts method is essentially unchanged, it's just private now. This means that aside from this name change the tests remain the same.
* | Tidy up artifact resolvingRichard Ipsum2014-10-092-34/+24
| | | | | | | | | | | | | | | | This patch started off as an attempt to address the comment in find_root_artifacts, though this patch doesn't impose an ordering on the list of artifacts as the comment suggested. The artifact resolver now returns a list of root artifacts which may make it easier to add multi-system builds to morph.
* | Merge branch 'sam/handle-missing-version-label'Sam Thursfield2014-10-082-2/+7
|\ \ | | | | | | | | | | | | Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk> Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
| * | deploy: Make ssh-rsync upgrade extension handle unset VERSION_LABELSam Thursfield2014-10-082-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It now gives an error message. Previously it would fail with a backtrace like this: 2014-10-08 09:51:37 [systems/genivi-baseline-system-armv7lhf-jetson.morph][self]Removing temporary mounts Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run self.process_args(args) File "/src/morph/morphlib/exts/ssh-rsync.write", line 54, in process_args self.upgrade_remote_system(location, temp_root) File "/src/morph/morphlib/exts/ssh-rsync.write", line 107, in upgrade_remote_system location, ['btrfs', 'subvolume', 'delete', orig_dir]) UnboundLocalError: local variable 'orig_dir' referenced before assignment
* | | Merge branch 'baserock/richardmaw/fix-distbuild-v3'Richard Maw2014-10-0874-2431/+1436
|\ \ \ | |/ / |/| | | | | | | | | | | Reviewed-by: Sam Thursfield Reviewed-by: Richard Ipsum Reviewed-by: Pedro Alvarez
| * | distbuild: serialize dependent sources of graphRichard Maw2014-10-082-24/+46
| | |