summaryrefslogtreecommitdiff
path: root/morphlib
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'baserock/richardmaw/S8847/branch-from-image-v3'Jonathan Maw2013-09-305-45/+312
|\ | | | | | | | | Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk> Reviewed-by: Jonathan Maw <jonathan.maw@codethink.co.uk>
| * B&M: refactor branch-from-imageRichard Maw2013-09-252-9/+107
| | | | | | | | | | | | | | | | | | | | | | | | This changes the interface of branch-from-image to only take 1 parameter, the name of the new system branch, as the root repository is loaded from the metadata. This was also what the previous version of branch-from-image did, but that silently ignored the parameter. Given there are not many users of branch-from-image, I felt it was a reasonable change.
| * morphlib: Add SystemMetadataDir classRichard Maw2013-09-253-0/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This provides access to the /baserock directory as if it were a dict, abstracting away the details of how to get data out of it. The abstraction is useful since it is easier to use than accessing /baserock yourself, and allows the storage format to be changed more easily. Keys with / in may be supported in the future. since there have been discussions about allowing morphologies to be placed in subdirectories. Adding this support would require creating and removing directory components when values are set and deleted respectively. Iterating would require using os.walk instead of glob.iglob, since python doesn't support ** in globs.
| * b&m: checkout and branch use context managerRichard Maw2013-09-251-36/+7
| |
| * b&m: Add system branch initializing context managerRichard Maw2013-09-251-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | This creates an object that the with statement can use to handle the context and clean up the workspace if the body raises an exception. This is roughly equivalent to having a function that takes a callback of what to do while the branch is being initialized, but with less boilerplate at the call site. contextlib is used to create a context manager from a generator function. This is less verbose than defining a class with __enter__ and __exit__ methods.
* | Make branch commands work with null refsRichard Maw2013-09-262-4/+11
| |
* | Allow building with null repo/refRichard Maw2013-09-264-9/+14
| |
* | Merge branch 'baserock/larswirzenius/S8698-morph-help-fmt'Lars Wirzenius2013-09-251-0/+25
|\ \ | |/ |/| | | | | Reviewed-by: Richard Maw Reviewed-by: Pedro Alvarez
| * Avoid formatting "morph help foo" outputLars Wirzenius2013-09-241-0/+25
| | | | | | | | | | | | | | | | The cliapp formatting doesn't understand Markdown. Adding Markdown support in Morph is more work than there is time for right now, but we'll do it later. This quick hack just means we output the raw Markdown rather than something that is incomprehensible due to ruined formatting.
* | Make cross-bootstrap successfully build everythingJonathan Maw2013-09-241-7/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This does three things: 1. It makes sure that the native-bootstrap script terminates if any steps fail. 2. It installs files to a temporary directory, then copies them out, so that builds don't break in cases where we delete files from DESTDIR. 3. It makes appropriate minor fixes so that a cross-bootstrap build can happen from beginning to end. Note: native-bootstrap does not succeed in every case. It has been observed to fail in an x86_64 virtual machine.
* | b&m: Use new MorphSet api in unpetrifyRichard Maw2013-09-241-23/+1
| |
* | MorphSet: add unpetrify_all() methodRichard Maw2013-09-242-0/+27
| | | | | | | | | | Add an unpetrify method to MorphSet, since it's a good place to put it and may be needed for merge.
* | b&m: Refactor petrify to use new morphset methodsRichard Maw2013-09-241-17/+12
| |
* | MorphSet: Add method for changing refs en-masseRichard Maw2013-09-242-0/+34
| | | | | | | | | | This is used to change all the refs of the morphologies in the root repository to point to the system branch instead.
* | MorphSet: Add petrify_chunks() methodRichard Maw2013-09-242-1/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | petrify_chunks handles regular petrification and branch_from_image's petrifying to another point in time. It is given the values to petrify to, instead of doing the ref resolution itself, since now it will consistently resolve refs to the same thing, and a different resolution can be passed in to petrify to another point in time. It only petrifies chunks, since petrifying strata and systems is a more complex operation that is not currently handled anyway.
* | MorphSet: add a list_refs() methodRichard Maw2013-09-242-0/+26
| | | | | | | | | | | | | | This will be used to factor the morphology traversal code out of petrify, since it is cleaner to find out which refs are present, resolve them, then update them, instead of the previous approach which updated as part of the traversal.
* | MorphSet: Factor traversal logic out of change_refRichard Maw2013-09-242-21/+69
|/ | | | | | | | | | | | | | | | | | Tweaking all the morphology specs in a MorphologySet is a common operation. - Edit does it to rewrite refs - petrify does it to fix refs to their state at the current point in time - unpetrify does this to reverse the petrify operation - branch-from-image works similarly to petrify After the refs inside morphologies are tweaked, any morphologies in the MorphSet that are referred to by the changed ref are also updated. This was previously an operation of change_ref, but it will be useful for the other use cases. Changes other than refs are not handled, since there are currently no use cases for it.
* Fix the graphing_plugin so it actually worksLars Wirzenius2013-09-231-3/+2
|
* Remove --staging-chroot, --staging-filler optionsLars Wirzenius2013-09-172-38/+0
| | | | | This also removes the long-obsolete code to install staging fillers in the staging area. We've not allowed users to do that for ages now.
* b&m: Use new-status as the defaultRichard Maw2013-09-172-2/+2
| | | | The previous status is currently still available as old-status.
* b&m: morph new-status provides per-branch statusRichard Maw2013-09-171-1/+23
|
* gitdir: Add method for getting uncommitted changesRichard Maw2013-09-172-0/+37
| | | | | | | | | | get_uncommitted_changes() is needed for morph status to tell if git repositories have changes in them. _get_status() is private, since it does not currently have a user, the small amount of code in get_uncommitted_changes() wrapping _get_status() is in the GitDirectory class instead of the branch and merge plugin, since `morph build` will also need to know about uncommitted changes.
* gitdir: add HEAD property to find the current branchRichard Maw2013-09-172-0/+15
| | | | | This is needed for status to tell if a repo is checked out in an unexpected branch.
* b&m: Add new-status subcommandRichard Maw2013-09-171-0/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This currently performs the workspace-wide status, but not the system-branch status functionality of the older status subcommand. A quick estimate showed the new code to be 5x faster, comparing # time (echo 2 >/proc/sys/vm/drop_caches; morph status) System branches in current workspace: baserock/richardmaw/S8537/ssh-copy-id baserock/richardmaw/S8564/ro-staging-area baserock/richardmaw/S8591/lzo-shared baserock/richardmaw/cliapp-pipefail baserock/richardmaw/malformed-strata-test master tutorial-1/tutorial/master tutorial-1/tutorial/update-ssh real 0m2.517s user 0m0.998s sys 0m1.482s # time (echo 2 >/proc/sys/vm/drop_caches; morph new-status) System branches in current workspace: baserock/richardmaw/S8537/ssh-copy-id baserock/richardmaw/S8564/ro-staging-area baserock/richardmaw/S8591/lzo-shared baserock/richardmaw/cliapp-pipefail baserock/richardmaw/malformed-strata-test master tutorial-1/tutorial/master tutorial-1/tutorial/update-ssh real 0m0.506s user 0m0.207s sys 0m0.233s
* workspace: add method for listing system branchesRichard Maw2013-09-172-0/+15
|
* util: add find_leaves() supplimenting find_leaf()Richard Maw2013-09-173-21/+26
| | | | | | | | | | | | | | | The workspace needs to be able to list all its system branches, and the system branches need to be able to list all their git repositories. This is broadly the same thing, just with a different directory to look out for, so provide that utility in morphlib.util. find_leaf() is rewritten to use find_leaves(), this is less efficient since it waits until every leaf is found. I felt it was better to reduce the code than maintain a slightly more optimal algorithm. The find_leaf() algorithm could become more optimal if it could lazily check for at least one result in a generator.
* Merge branch 'baserock/richardmaw/S8717/refactor-petrify-v4'Richard Maw2013-09-1611-12/+478
|\ | | | | | | | | | | | | | | Reviewed-by: Lars Wirzenius Reviewed-by: Daniel Silverstone Daniel gave his +1 with the caveat that he would like tests for petrifying a system branch other than master.
| * b&m: re-implement morph unpetrifyRichard Maw2013-09-052-2/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This also undoes the test suite change to keep it using the old petrify. Petrify and Unpetrify have the same structure: 1. Set up 2. Load all the morphologies 3. Make changes to the morphologies 4. Write back any changes This suggests some re-use in the form of passing a function callback to a morphology visitor.
| * b&m: Re-implement morph petrifyRichard Maw2013-09-052-1/+92
| | | | | | | | | | This includes amending the test suite to use the old petrify, since the old unpetrify depends on some implementation details of it.
| * morphlib: Add MorphologyFinder classRichard Maw2013-09-053-0/+183
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MorphologyFinder is a small wrapper on top of GitDirectory that allows the inspection of morphologies in the repository. Its purpose is to isolate the logic for reading morphologies into one place. It is used by passing a GitDirectory and optionally a ref to the MorpholgyFinder constructor, then list_morphologies and read_morphology may be used. The ref is passed directly to the GitDirectory, so its semantics for a ref of None or omitted are used. i.e. It uses the working tree. Ref resolving is deferred until a morphology is listed or read, so it will not raise an exception for an invalid ref until then.
| * gitdir: Add methods to inspect contentsRichard Maw2013-09-052-0/+142
| | | | | | | | | | | | This adds methods to list and read files. The difference between doing this to a commit and the currrent working tree is abstracted over by whether the passed ref is None or omitted.
| * MorphSet: keep old ref as unpetrify-refRichard Maw2013-09-052-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This makes petrify much simpler, since adding unpetrify-ref is handled by MorphSet. This has the side-effect of adding unpetrify-ref when editing, but this information is also useful for this use-case. The edit tests now have different diffs and commits, so those have been updated. Also, the old merge code currently warns that now there appears to be petrified changes, since it detects if a branch is petrified by the presence of the unpetrify-ref field, even though the ref is not a SHA1. The old code does not handle undoing an edit the new way, so I did not feel it was worth attempting to comprehend it to fix this bug, which will be later fixed by the introduction of the new merge code.
| * morphlib.git: Refactor is_valid_sha1Richard Maw2013-09-051-2/+2
| | | | | | | | | | | | | | It now uses a generator expression instead of a list, which makes the operation lazy. It also uses string.hexdigits instead of redefining it.
| * sysbranchdir: simplify prefix / stripRichard Maw2013-09-051-2/+1
| | | | | | | | There's a built-in function to do this.
| * Fix MorphologySet bug: right Morphology object changedLars Wirzenius2013-09-051-4/+4
| | | | | | | | | | | | The old code worked, but it was depending on really obscure details of how the Python scoping works. This commit changes it to not do that anymore.
* | foreach: sort repositories asciibeticallyRichard Maw2013-09-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The test suite checks the output of the foreach command, this requires repositories to be returned in a stable order. The order returned from os.listdir differs by the filesystem used. btrfs returns roughly in insertion order, tmpfs alphabetically. This allowed the failure to sort pass through unchecked, since I was testing on btrfs, but people discovered issues when using /tmp, which is a tmpfs. To fix this, foreach sorts the result of list_git_directories() by directory name. The previous code would always return the root repository first, this was a unnecessary complication, so now everything is returned asciibetically and the tests amended to reflect this.
* | exts: Remove tab from vdaboot.configureRichard Maw2013-09-161-1/+1
| | | | | | | | | | This snuck in since the test suite could not be run when TMPDIR was on a tmpfs.
* | exts: Add openstack configure/write extsPedro Alvarez2013-09-112-0/+174
|/ | | | | | | | | openstackssh.write: Write extension which deploy a raw image of baserock directly to an OpenStack machine using python-glanceclient. The raw image deployed has modified its bootloader to use virtio disks. vdaboot.configure: Configuration extension to change the mount point of "/" to use virtio disks (/dev/vda).
* b&m: Re-implement morph foreachRichard Maw2013-09-042-1/+59
| | | | | | | | | | | | | | The previous version attempted to separate the output of the commands run with newlines by printing a newline at the beginning if it wasn't the first repository. The check it used was for the first repository was flawed by assuming that the root repository was the first repository processed. Since the complication was not worth keeping the output identical, it has been changed to separate by printing a newline after every command and the test output has been amended to include the extra trailing newline.
* Merge branch 'baserock/richardmaw/S8694/edit-no-write-defaults-v3'Richard Maw2013-08-304-166/+104
|\ | | | | | | | | Reviewed-by: Lars Wirzenius Reviewed-by: Jonathan Maw
| * edit: Unset defaults before writingRichard Maw2013-08-301-0/+1
| | | | | | | | | | | | | | | | This also amends the test suite. The git apply command to alter the stratum proved inflexible when anything changed, so it has been replaced by a small in-line python script.
| * morphloader: Add method to unset default valuesRichard Maw2013-08-302-2/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | The original contents of the morphology is not generally trackable, comments are lost for example, and it's simpler to output a canonical format than attempt to preserve everything of the original. However, we don't want to clutter the output with fields that have been filled out to be the defaults, so provide a method to remove fields that are the same as their default. This also removes a check in set_defaults that it is a valid kind, since it explicitly declares it assumes the morphology is valid.
| * morph edit: Check for sufficient argumentsRichard Maw2013-08-301-0/+5
| | | | | | | | | | It's nicer to get a clean error message explaining what's wrong than a stack trace.
| * Revert "Merge branch 'liw/restore-old-cmds'"Richard Maw2013-08-292-164/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 2dc382a2a9ae977b1158002cd2631ec5593959c1, reversing changes made to 89a019af088ff62459699a6efdadf8ac8fe35dd9. We decided to restore the old commands for the release, as we weren't confident the new implementations were correct yet. To gain that confidence we need to use them, so the old versions are no longer available, and the new edit code is used when the edit subcommand is invoked. Doing so also requires the test suite to use the edit command instead of new-edit.
* | rootfs-protection: Fix building with tempdir=/tmpRichard Maw2013-08-282-15/+44
|/ | | | | | | | | | | | | | | | | | The issue is that the tempdir is added at the end of the list of directories to keep writable, and entries earlier in the list are subdirectories of tempdir. The check encountered the subdirs first, so decided it must recurse and make everything else read-only. It never got as far as noticing that /tmp was requested writable. Now, every path is checked for being equal, then checked for being a subdirectory. This changed style to use any and generator expressions, as it was more concise than having an explicit loop for checking equality, then an explicit loop for checking subdirectory.
* Merge branch 'baserock/richardmaw/S8646/edit-odd-submodules' of ↵Richard Maw2013-08-271-6/+6
|\ | | | | | | | | | | | | | | | | | | git://git.baserock.org/baserock/baserock/morph Reviewed-by: Lars Wirzenius Reviewed-by: Jonathan Maw Note: This branch was based off the `baserock/richardmaw/S8646/fix-tests` branch, so that has also been merged with this commit.
| * Use Submodules class to read submodule configRichard Maw2013-08-231-6/+6
| | | | | | | | | | | | | | | | | | | | Previously it would use `git config --get-regex`, which while more terse, is a little arcane and the regular expression did not account for slashes or dashes in the option names, and the names usually correspond to a file path. Now we use the Submodules class, which validates that the listed commit is available.
* | Remove redundant dict.getRichard Maw2013-08-231-1/+0
| |
* | Merge branch 'baserock/richardmaw/S8563/bootstrap-rootfs-protection'Richard Maw2013-08-233-39/+184
|\ \ | |/ |/| | | Reviewed-by: Lars Wirzenius
| * Make the tempdir writable in bootstrap mode.Richard Maw2013-08-161-3/+7
| | | | | | | | | | Somehow this was working on x86 even though it had no ability to write to tempdirs, but on ARM it wasn't working.
View Lorry Controller Status