summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* morph petrify: Avoid petrifying chunks that have already been editedSam Thursfield2012-10-185-23/+130
| | | | | | | | | | | | | This change causes 'morph petrify' to avoid petrifying any chunk whose ref matches the current system branch, because it makes no sense to petrify something that is also being edited. It also improves efficiency slightly and adds warning where different systems point to different refs of the same stratum. A non-obvious effect of this is that if you try to petrify 'master', many of the chunks won't get petrified because they are built from 'master'. However, petrifying master makes no sense so I'm not sure that we need to worry.
* Merge branch 'samthursfield/morph-status'Lars Wirzenius2012-10-127-33/+208
|\
| * morph status, foreach: Find root repo in correct waySam Thursfield2012-10-121-5/+8
| | | | | | | | | | | | | | | | Previously if the user had renamed the directory holding the root repository, the commands would break tragically. Also fix find_repository() to avoid aborting if it encounters a git repo in the branch checkout that wasn't put there by Morph.
| * Factor out common function iterate_branch_repos()Sam Thursfield2012-10-121-10/+14
| |
| * morph status: Add initial version of commandSam Thursfield2012-10-127-0/+163
| | | | | | | | This provides a user-friendly summary of the workspace or branch status.
| * Reorder branch and merge plumbing commands to end of fileSam Thursfield2012-10-121-25/+30
|/ | | | | Users do not need these now due to 'morph status' existing. However, they are still useful for scripts to call.
* Merge remote-tracking branch 'origin/liw/fix-check-on-squeeze'Daniel Silverstone2012-10-124-6/+16
|\
| * Disable two tests if Python 2.6 is being usedLars Wirzenius2012-10-122-0/+8
| |
| * Use . instead of source in /bin/sh scriptLars Wirzenius2012-10-121-1/+1
| |
| * Fix use of GzipFile to not assume "with" protocolLars Wirzenius2012-10-122-5/+7
|/ | | | | This fixes a bunch of tests so they pass on squeeze, which has a version of Python whose GzipFile doesn't support the "with" protocol.
* Merge remote-tracking branch ↵Daniel Silverstone2012-10-115-9/+119
|\ | | | | | | | | | | 'origin/samthursfield/S4873-warn-when-merge-causes-petrification' Renamed petrification test slightly as merge fixup.
| * Add test for petrify warningSam Thursfield2012-10-112-0/+33
| |
| * morph merge: Warn if a merge will cause unexpected petrificationSam Thursfield2012-10-112-9/+84
| | | | | | | | | | | | | | | | | | | | | | | | | | The concept of a component path is new. This is simply a concise way of referring to a component in an error message, and looks like this: base-system-x86_64-generic.bsp-x86_64-generic.linux We currently only touch the 'edited chunks' in merge_stratum(), i.e. those in the FROM branch where 'morph edit' was run. However, the petrification can affect any chunk so there is a new method added to obtain all components in a morphology. This function also returns the differences between the two, which we will make use of at a later date.
| * Fix test that should have been doing 'morph edit'Sam Thursfield2012-10-111-0/+2
| |
* | Merge remote-tracking branch 'origin/liw/morphology-validation'Sam Thursfield2012-10-112-1/+117
|\ \ | |/ |/|
| * Validate that systems, strata reference correct kindsLars Wirzenius2012-10-111-0/+43
| |
| * Warn about unknown morphology fieldsLars Wirzenius2012-10-111-0/+32
| |
| * Add morphology validation to branch+mergeLars Wirzenius2012-10-111-0/+35
| |
| * Handle ValueErrors from JSON libraryLars Wirzenius2012-10-111-1/+7
|/
* Merge branch 'samthursfield/S5211-fix-foreach-with-full-uris'Sam Thursfield2012-10-113-3/+35
|\ | | | | | | Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * morph foreach: Handle repos addressed by full URLs correctlySam Thursfield2012-10-103-3/+35
|/ | | | | | | | | | This should not normally be used, because we make no attempt to detect when a full URL and a keyed URL are equivalent, so they cannot be used interchangably. However, 'foreach' would previously fail completely if the branch root happened to be a full URL because it didn't call convert_uri_to_path() correctly.
* Merge branch 'baserock/richardmaw/S5182/fix-link-paths' of ↵Richard Maw2012-10-095-13/+148
|\ | | | | | | | | | | | | git://git.baserock.org/baserock/morph Signed-off-by: Lars Wirzenius on IRC Signed-off-by: Daniel Silverstone on IRC
| * Add regression test for created tarballsRichard Maw2012-10-093-0/+140
| |
| * Make hardlink paths relativeRichard Maw2012-10-091-0/+3
| | | | | | | | | | | | Symlink paths are not altered, since their paths are relative to where the link is, not the root of the file system, and should be handled by the chunk's creation.
| * Use os.path.relpath for fixing paths in tarball system builderRichard Maw2012-10-091-4/+2
| |
| * Use standard library function for relative pathsRichard Maw2012-10-091-9/+3
|/ | | | | os.path.relpath does approximately the same as mkrel, with the exception of asserting.
* Merge branch 'danielsilverstone/fix-up-make-patch' of ↵Richard Maw2012-10-094-37/+120
|\ | | | | | | git://git.baserock.org/baserock/morph
| * foldmeDaniel Silverstone2012-10-091-2/+7
| |
| * Block second half of make-patch test.Daniel Silverstone2012-10-091-0/+14
| | | | | | | | | | | | The make-patch test assumes system images will be mountable. This patch disables that part of the test and adds a warning that we need to sort this out longer-term.
| * Fix up make-patchDaniel Silverstone2012-10-091-34/+56
| | | | | | | | | | | | This corrects make-patch to decompress system images as they're now compressed. Longer-term this should cope with tarball system images etc.
| * Add a sparse-aware (for writing) copyfileobj.Daniel Silverstone2012-10-091-0/+39
| |
| * Correct a few long lines.Daniel Silverstone2012-10-091-3/+6
|/
* Merge branch 'danielsilverstone/compress-disk-images'Daniel Silverstone2012-10-081-29/+45
|\ | | | | | | | | Reviewed-By: Lars Wirzenius (over the shoulder) Reviewed-By: Richard Maw (over the shoulder)
| * Compress system disk imagesDaniel Silverstone2012-10-081-29/+45
| | | | | | | | | | | | | | | | | | | | Since tarball rootfs images are compressed with gzip -1, we should do the same for disk images. This means they'll need decompressing before they can be used as disk images, but it means we'll be shunting around something closer to their true data size rather than the disk image size. For example, a 2G development system image compresses down to around 380M which is a lot faster to transfer and doesn't risk de-sparseification.
* | Fix typoLars Wirzenius2012-10-081-1/+1
|/ | | | | Reviewed-By: Daniel Silverstone (on irc) Reviewed-By: Richard Maw (on irc)
* Merge branch 'master' of git://git.baserock.org/baserock/morphLars Wirzenius2012-10-082-1/+20
|\
| * Handle branch name kind changes more gracefully.Daniel Silverstone2012-10-081-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Scenario: a repository contains a ref: refs/heads/foo and we have that in our repository cache. Action: upstream deletes that ref and then pushes a new refs at refs/heads/foo/bar. Action: we attempt to update the git and the update fails because git remote update cannot cope with the 'kind' change of refs/heads/foo from ref to directory. Remedy: If we get an exception from the remote update --prune we try the less efficient but more resilient combination of first pruning and then updating. Reviewed-By: Lars Wirzenius (over the shoulder) Reviewed-By: Richard Maw (over the shoulder)
| * Merge branch 'danielsilverstone/tarball-rootfs-fix'Daniel Silverstone2012-10-051-0/+15
| |\
| | * Actually construct a tarball for the rootfs-tarball system kind.Daniel Silverstone2012-10-051-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | This adds tarball construction for rootfs-tarball. We deliberately choose compression level 1 because it gets us sufficient savings (ca. 60% smaller) while not wasting too much time (roughly 60% more time than uncompressed tar generation).
* | | Change the time sanity checkLars Wirzenius2012-10-081-13/+6
|/ / | | | | | | | | | | | | | | | | | | | | | | | | We were having trouble with NTP setting the system time to be correct after .pyc files had already been created. The .pyc files got created at startup, before NTP had had a chance to run. This sometimes resulted in .pyc timestamps being wildly wrong and far in the future or past, and that would prevent morph from ever working again on the machine. We now just check that the system time is not far in the past. This is enough to make sure we don't create timestamps that break tar. Reviewed-By: Daniel Silverstone (on irc)
* | Merge branch 'liw/lac-clear'Lars Wirzenius2012-10-052-0/+26
|\ \ | |/ |/| | | Reviewed-By: Richard Maw (on irc)
| * Add a way for a local artifact cache to clear itselfLars Wirzenius2012-10-052-0/+26
|/
* Pass -d to umount to clean up loops.Daniel Silverstone2012-10-041-1/+1
| | | | | | | | This alters the test behaviour so that umount;losetup becomes umount -d. This means that util-linux and busybox line up in terms of behaviour for this test. Reviewed-By: Richard Maw (Over the shoulder)
* Merge remote-tracking branch 'origin/samthursfield/S4732-edit-handle-submodules'Daniel Silverstone2012-10-044-6/+99
|\
| * Handle submodules in 'morph edit'Sam Thursfield2012-10-034-6/+99
|/ | | | | | | | | | | | | | | | We often have .gitmodules edited to contain a URI such as upstream:gnulib, so that we can transparently mirror these in different locations. It would be nice to set up git url.insteadOf rules to expand these for the submodules, but 'git submodule update' uses 'git clone' to fetch them, which will not take into account the configuration of the parent repository. Instead, we set up the submodules automatically and rewrite the URLs directly in the configuration. The user will need to recreate their system branch checkouts if their URL configuration changes, or update the URLs manually, but that should not happen often.
* Merge remote-tracking branch 'origin/samthursfield/ref-access-fixes'Daniel Silverstone2012-10-0312-265/+257
|\
| * Fix after reviewSam Thursfield2012-10-033-4/+3
| |
| * Revert "Save __file__ so we can recursively run morph later on"Sam Thursfield2012-10-031-4/+1
| | | | | | | | | | | | No longer needed, we don't rerun morph as a merge driver any more. This reverts commit 3b0a4d7346cef0091b00ad0cc6610476fdea61d1.
| * morph merge: Rework how we interact with gitSam Thursfield2012-10-037-189/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | Upsides: - clearer error messages on conflicts (we no longer dump the git output) - merge base is available during morphology merging - we no longer need to go through the complexity of implementing a git merge driver We now manually fetch and then merge, instead of using git pull. This is not strictly necessary, but it makes it clearer in the code how FETCH_HEAD is involved in the process.
| * Use repository cache for git access other than current system branchSam Thursfield2012-10-033-73/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The origin/ refs in the system branch checkout repos may or may not be up to date, and may or may not have been tampered with by the user. Much better to use our central cache for everything other than changes to the system branch ref itself, where we should indeed be honouring the user's local changes. At a later date we could warn if the user modifies refs other than the system branch ref but does not push, as these changes will have no effect. NOTE: this commit breaks 'morph merge'. It is fixed in the next commit.
View Lorry Controller Status