summaryrefslogtreecommitdiff
path: root/morphlib
Commit message (Collapse)AuthorAgeFilesLines
...
* morphlib: Add BuildBranch abstractionRichard Maw2013-11-292-0/+261
| | | | | | | This is an abstraction on top of SystemBranchDirectories, providing the ability to add uncommitted changes to the temporary build branch, push temporary build branches and retrieve the correct repository URI and ref to build the system.
* branchmanager: Allow deferred and optional cleanup on success.Richard Maw2013-11-292-14/+179
| | | | | | | | | | | | Now it will optionally clean up on success based on a constructor parameter. It can be later cleaned up explicitly by calling close(). It is called close, rather than something more obvious, like cleanup(), since it means the manager can be re-used with contextlib.closing(). This now means that using the Managers without a context manager is less ugly, since you can explicitly call .close() in a finally block.
* sysbranchdir: Move load_all_morphologies helper hereRichard Maw2013-11-293-10/+16
| | | | | | | | | | | | | | | This was previously a private method of the branch and merge plugin, but it's useful to other plugins, so has been moved to the SystemBranchDirectory class, where everything else can get to it. It has an unpleasant amount of coupling to other classes, but in a *good* object oriented design it would either be a tiny module on its own, or not exist and leave all its users to re-implement the same logic multiple ways, so we've opted for a less clean, but more useful design. It is left un-covered by the unit tests, since it requires a great deal of instrumentation to test, at which point it may be best to leave it to integration tests.
* morphloader: use getattr for validate, set defaultsRichard Maw2013-11-291-16/+5
| | | | It saves some boilerplate.
* validation: Require there be non-bootstrap chunks in systemsRichard Maw2013-11-291-6/+40
| | | | | | | | | | | | | | Bootstrap chunks don't make it into the final system, so there needs to be an extra check for empty systems after the sources have been collected. This was complicated slightly by the fact that if you try to build a chunk directly you will have no strata in your sources, hence no non-bootstrap chunks, but validation for having been told to build a chunk is best handled later. This amends the old yarns that depended on building a bootstrap chunk and adds a new one that explicitly builds a system with bootstrap chunks.
* morphloader: Require systems have at least one stratumRichard Maw2013-11-292-34/+140
| | | | | | | | | | It doesn't currently make sense to build a system which contains no strata. We may later add other fields, such as initramfs to contribute to the system's artifact, but until then it's another bug to trip over. This uses collections.Sequence for checking the type of the systems entry in the morphology as a style choice, though it allows more flexibility if the types in the parsed morphology change.
* morphloader: Set default values for cluster morphsRichard Maw2013-11-292-3/+54
| | | | | | | | This was omittted from the MorphologyLoader due to cluster morphologies being added at about the same time. This bug escaped detection since the MorphologyLoader was not required to deploy. It soon will be.
* morphloader: Don't use ValueError exceptionRichard Maw2013-11-292-4/+26
| | | | | Dedicated exceptions allow more fine-grained exception catching and can have extra data.
* Merge branch 'master' of git://git.baserock.org/baserock/baserock/morphLars Wirzenius2013-11-282-1/+2
|\
| * Make morph able to use ppc64 architecture (POWER PC 64 bits)Pedro Alvarez2013-11-272-1/+2
| |
* | Merge remote-tracking branch ↵Lars Wirzenius2013-11-281-0/+3
|\ \ | |/ |/| | | 'remotes/origin/baserock/ps/show-stratum-versions-in-build'
| * show which ref is expected for each stratum when preparing buildPaul Sherwood2013-11-211-0/+3
| |
* | morphlib: Add branch context managersRichard Maw2013-11-223-0/+490
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a LocalRefManager, which handles ref updates to local repositories (i.e. your workspace). It provides proxy methods for ref updates to a set of repositories. If an exception occurs in the body of the context manager, the updates will be rolled back to before the context manager was entered. The purpose for using a LocalRefManager instead of making the changes to the repositories directly, is to provide atomic updates to a set of refs in a set of repositories, where all refs are updated, or none are. This also adds a RemoteRefManager, which handles pushing branches to remote repositories. It provides a proxy push method, which will delete pushed branches, and re-push deleted branches after the context manager exits. Its purpose, instead of providing atomic updates to remote repositories, is to provide temporary branches. This is because it is used to provide temporary build branches. The difference between atomic update and temporary push, is that the remote branches are deleted when the context is left, rather than kept, as LocalRefManager does. The RemoteRefManager currently cannot provide the same atomicity guarantees as the LocalRefManager, so if there is a push between the branch being created and the RemoteRefManager cleaning it up, that change is lost without RemoteRefManager even knowing it existed. Git 1.8.5 will add functionality to make this possible.
* | GitDir: Add remote.push(RefSpec...)Richard Maw2013-11-222-0/+300
| | | | | | | | | | | | | | | | Remotes have a push method, which takes multiple RefSpecs, runs git push using arguments derived from the set of refspecs, then returns the push's result. If it fails the push, it will return the result in the exception.
* | GitDir: Add support for push urls in RemoteRichard Maw2013-11-222-19/+56
| |
* | GitDir: Split out Remote objectRichard Maw2013-11-223-25/+74
| | | | | | | | Operations on remotes are now accessed through this proxy object.
* | GitDir: Add methods for ref managementRichard Maw2013-11-222-0/+201
| |
* | GitDir: Add GitDirectory.commit_tree() methodRichard Maw2013-11-222-0/+53
| | | | | | | | | | This is used to create commit objects. This is used by build without commit to provide the behind-the-scenes history.
* | gitindex: Add GitIndex.write_tree()Richard Maw2013-11-222-0/+9
| | | | | | | | | | This generates a tree object from the index. This can then be used to create a commit.
* | gitindex: Add GitIndex.add_files_from_working_tree()Richard Maw2013-11-222-0/+43
| | | | | | | | | | | | | | | | | | | | This is like GitIndex.add_files_from_index_info, but uses the working tree and a list of paths. This is more convenient when the changes exist in the working tree. The comment describes its relationship with GitIndex.add_files_from_index_info, but it is faster, since it calls out to `git add` rather than calling out to `git hash-object` for every file.
* | gitindex: Add GitIndex.add_files_from_index_info()Richard Maw2013-11-222-0/+31
| | | | | | | | | | | | This is used to add files directly to the index, without touching the working tree. This is used in the build without commit code for creating new morphologies pointing to different branches.
* | gitindex: Add GitIndex.set_to_tree(treeish) methodRichard Maw2013-11-222-0/+13
| | | | | | | | | | | | | | This is used to set the state of the index to that of a previously known commit. This is needed for the build without commit logic, so that commits generated are that of the last commit in the repository and anything uncommitted.
* | GitDir: Add GitIndex classRichard Maw2013-11-226-38/+165
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This represents the state of the index of a GitDirectory. Methods that use the index are now used via the GitIndex class, rather than using the default index, as previously used when the methods were in GitDirectory. GitIndex may be constructed with an alternative path, which can be used to manipulate a git checkout without altering a developer's view of the repository i.e. The working tree and default index. This is needed for `morph build` and `morph deploy` to handle the build without commit logic.
* | GitDir: add store_blob methodRichard Maw2013-11-222-0/+28
| | | | | | | | This is needed for making commits without touching the workspace.
* | util: Add helper for splitting up huge iterablesRichard Maw2013-11-222-0/+21
| | | | | | | | | | | | | | | | This is used to split a large iterable into more manageable chunks. This is important when a consumer can only handle so much input at once e.g. A program can take a large, but finite number of arguments, xargs exists to handle this for shell programs.
* | MorphologySet: Export traverse_specs methodRichard Maw2013-11-221-6/+6
| | | | | | | | | | | | | | | | The set of helpers isn't helping for re-usability, since every new use for the traverse_specs method is different enough that it is only used in one place. So just export traverse_specs itself and be done with it.
* | GitDir: Add resolve_ref_to_{commit,tree} methodsRichard Maw2013-11-222-5/+22
| | | | | | | | | | | | | | | | | | | | | | | | Some APIs should take SHA1 object IDs, there needs to be a way to get one from a ref. To handle this, we add APIs for getting either the commit or the tree pointed to by the ref. Commits are provided because we need to create commits from existing commits and update refs from existing values. Trees are provided because we need trees to create commits, and we can read trees into the index to eventually create another tree.
* | GitDir: Provide more specific object access than cat-fileRichard Maw2013-11-222-5/+29
| | | | | | | | | | | | | | We need to use cat-file for files by SHA1, commits by SHA1 and files by ref and path, so provide access in separate methods, since while it's all the same thing "under the hood", it avoids the user needing to know the command-line syntax.
* | GitDir: Allow config values ending in whitespaceRichard Maw2013-11-221-2/+2
| |
* | GitDir: remove duplicate definition of update_remotesRichard Maw2013-11-221-4/+0
| |
* | Remove unused invent_new_branch methodRichard Maw2013-11-221-12/+0
|/
* Fix typoRichard Ipsum2013-11-061-1/+1
|
* Merge remote-tracking branch 'refs/remotes/origin/danielfirth/S9122'Lars Wirzenius2013-11-061-162/+0
|\
| * Removed ssh.configure extensionDan Firth2013-10-081-162/+0
| |
* | Merge remote-tracking branch 'origin/danielfirth/language-changes'Lars Wirzenius2013-11-052-2/+2
|\ \ | | | | | | | | | | | | | | | Needed to update copyright years when merging. Don't understand why the problem didn't show up earlier, since it doesn't seem to have been caused by Dan's changes.
* | | Fix RemoteArtifactCache to quote filenames in URLsLars Wirzenius2013-11-042-1/+8
|/ / | | | | | | This will allow fetching of gtk+ artifacts.
* | Added scenario test to prevent morph edit from being used without a system ↵Daniel Firth2013-11-011-0/+2
| | | | | | | | argument
* | Added scenario test for preventing building uncommitted systemsDaniel Firth2013-11-011-1/+8
| |
* | Don't remove an existing branch in 'morph branch'Pedro Alvarez2013-10-311-0/+3
| |
* | Deploying a non-cluster morphology now displays an error messageDaniel Firth2013-10-301-0/+4
| |
* | Added scenario test for preventing cross-buildingDaniel Firth2013-10-302-1/+13
| |
* | Add --artifact-cache-server and --git-resolve-cache-serverLars Wirzenius2013-10-293-4/+37
| |
* | Building with references to tags will now instead use the underlying commitDaniel Firth2013-10-281-2/+3
| |
* | Tidy up debug output of morph buildDaniel Firth2013-10-284-13/+11
| |
* | Merge remote-tracking branch 'origin/liw/fstab.configure'Lars Wirzenius2013-10-241-0/+40
|\ \ | | | | | | | | | | | | | | | | | | Reviewed-by: Richard Maw At his suggestion, fixed the call to sorted() to be a call to asciibetical().
| * | Add fstab.configureLars Wirzenius2013-10-241-0/+40
| | | | | | | | | | | | | | | This will allow the user to append text to /etc/fstab during a deployment, without having to write custom configuration extensions.
* | | Merge remote-tracking branch 'origin/liw/cachedrepo-unit-test-fix-v2-rebase'Lars Wirzenius2013-10-243-7/+17
|\ \ \ | |/ / |/| | | | | Reviewed-by: Richard Maw
| * | Fix app.status call in log_dict_diffLars Wirzenius2013-10-241-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This bug was triggered by the fix to CachedRepo.resolve_ref and without this fix, the resolve_ref fix will break the test suite. The bug is that log_dict_diff calls the status method with an msg keyword argument that may contain percentage characters. status interprets the value of msg as a format string, and the percentage characters trigger formatting to happen. The fix for that is to not interpolate the value of key and dictA[key] and dictB[key] into msg before calling status, but letting status do that. Thus the msg values are changed to reference %(key)s instead and passing in a value for key as a separate argument. Ditto for dictA[key] and dictB[key].
| * | Make CachedRepo.resolve_ref handle non-existent SHA1Lars Wirzenius2013-10-241-4/+9
| | | | | | | | | | | | | | | This changes how CachedRepo runs git to get the SHA1 information it needs, based on a suggestion by Richard Maw.
| * | Fix CachedRepo.resolve_ref unit test for missing SHA1Lars Wirzenius2013-10-231-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CachedRepo.resolve_ref does, effectively, this: absref = git rev-parse --verify $ref git log -1 --format=format:%T $absref Roughly, "git rev-parse" takes any ref and returns the corresponding SHA1. If the ref looks like a SHA1 (40 hex digits), it is returned as-is, and despite --verify is not checked for existence. "git log" then takes the SHA1 and returns the **tree** SHA1, as opposed to the commit one, and if the commit doesn't exist, barfs. The unit test for resolve_ref with an invalid SHA1 currently succeeds for the wrong reason. The mocked _rev_parse fails for an unknown SHA1 (raising cliapp.AppException), which causes resolve_ref to raise InvalidReferenceError, which the unit test expects. However, the real implementation of _rev_parse wouldn't fail in that way, and so the unit test doesn't test the thing it's meant to test: that resolve_ref actually works the expected way for an unknown SHA1. What actually happens is that resolve_ref calls _show_tree_hash, which raises cliapp.AppException for an unknown SHA1, resulting in horror and despair, instead of resolve_ref raising InvalidReferenceError. This commit fixes the unit test so that it causes the right code path in resolve_ref to be executed. This makes the unit test suite to fail.
View Lorry Controller Status