summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Add a new 'morph build' for building from a system branchJannis Pohlmann2012-09-076-31/+409
| | | | | | | | | | | | | | | | This new command does a lot. First of all, its command line interface has now changed to 'morph build SYSTEM' and it needs to be run from a system branch. When called, the new 'build' command will identify the repositories and morphologies involved in building the system from the system branch, create a build ref behind the scenes based on the system branch and add a commit with all uncommitted changes to this build branch for every repo involved. It will then push those build branches to the repository server and kick off a build of BRANCH_ROOT BUILD_BRANCH SYSTEM.morph. After building has finished, the remote build branches will be deleted again.
* Add 'build-prefix-ref' setting for creating temporary build refsJannis Pohlmann2012-09-071-0/+6
| | | | | | | This is needed to create temporary build refs and push them to the repository server. We want this to be configurable to make things more flexible with regards to different repository server configurations.
* Rename 'morph build' command to 'morph build-morphology'Jannis Pohlmann2012-09-0724-34/+45
| | | | | | | | | | This is needed because we are replacing the current build command by a new one that builds from a local system branch. Since that will be the more common scenario, we want the new implementaiton to go by the name 'morph build'. This commit therefore renames 'morph build' and updates all the tests to use 'morph build-morphology' instead.
* Merge remote-tracking branch 'origin/jannis/use-tree-sha1-for-cache-key'Daniel Silverstone2012-09-0514-83/+110
|\
| * Compute cache keys using tree SHA1s instead of commit SHA1sJannis Pohlmann2012-09-051-1/+1
| | | | | | | | | | This avoids rebuilding things when commits are made that do not change the actual source code or the morphologies.
| * Resolve tree SHA1 along with commit SHA1 when resolving refsJannis Pohlmann2012-09-0513-82/+109
|/ | | | Adjust all other parts and the tests to work with this.
* Merge branch 'jannis/rewrite-edit-command'Sam Thursfield2012-09-0412-191/+303
|\
| * Remember system branch name and allow branch to be renamedJannis Pohlmann2012-09-041-21/+32
| | | | | | | | | | | | | | This requires a few changes to most of the branch and merge commands. The main internal change is that deduce_system_branch() now returns a tuple (branch name, branch directory) whereas before it only returned the branch directory, assuming that this was equal to the branch name.
| * Update synposis for branch, checkout and edit commandsJannis Pohlmann2012-09-041-3/+3
| |
| * Add convenience method for traversing a directory treeJannis Pohlmann2012-09-041-36/+27
| | | | | | | | | | | | | | | | | | This method (walk_special_directories) will yield all subdirectories of a root directory that have a special subdirectory and at most N non-hidden immediate subdirectories. This is used to e.g. iterate over all git repositories (directories with a .git subdirectory) and system branch directories.
| * Make "morph edit" print the changes for the user, rework static methodsJannis Pohlmann2012-09-041-48/+84
| | | | | | | | | | | | | | | | | | | | This commit introduces a new way of logging changes made to repositories or morphologies as part of executing a morph branch plugin command. These changes can then logged to the user grouped by repositories. Some of the static methods in the branch and merge plugin are converted to non-static methods.
| * Rewrite edit command to expect SYSTEM STRATUM [CHUNK] parametersJannis Pohlmann2012-09-0411-119/+190
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Editing no longer requires a repository to be specified, neither does it require a branch. It now starts off from a system morphology that is required to exist in the branch root repository. Relative to this system, "morph edit" realises the repository of a stratum and, optionally, a chunk, creates edit branches named after the system branch, if necessary, and update the references in the system and stratum morphology accordingly. The changes made to any of the repositories in the system branch are not committed. All existing changes are updated to work with this new input syntax for "morph edit".
| * Allow 'disk-size' in system morphologies to be an integerJannis Pohlmann2012-09-041-8/+11
| | | | | | | | | | | | This is needed for being able to write morphologies to disk programmatically after making automatic changes to them using morph.
* | README: describe systems.strata correctlySam Thursfield2012-09-041-2/+5
|/
* Merge branch 'jannis/implement-chunk-strata-lookup-differently'Sam Thursfield2012-09-042-30/+61
|\
| * Rework morphology child lookup to be more useful for branch/mergeJannis Pohlmann2012-09-042-30/+61
|/ | | | | | | | | | | | | In order to modify morphologies in place and then write them back to disk in system branches (e.g. when running "morph edit"), we need to have access to the dicts that store references to strata in system morphologies and chunks in stratum morphologies, respectively. Therefor, the previous triplet-returning child lookup is replaced with a new internal method to ensure uniqueness of names in morphologies and a new method to lookup children in this commit. The unit tests are adjusted to cover everything in appropriate ways.
* Morph: index component morphologies by nameSam Thursfield2012-09-042-12/+123
| | | | | | This requires that we enforce uniqueness. New method: Morphology.lookup_morphology_by_name()
* ArtifactResolver: chunks cannot be duplicated in a stratumSam Thursfield2012-09-042-57/+54
| | | | | We now enforce name uniqueness and don't yet implement a way of providing an alias for chunks, so self-dependency is impossible.
* Warning instead of error for empty strata, temporarilySam Thursfield2012-09-031-1/+1
| | | | | | | | | This kind of reverts commit: c4facff97b27968f00a75ffb3d49cea2080ad9a4 Currently our bootstrap depends on building an empty stratum; this should be changed so that it builds a tarball system image, but this is currently broken too. This commit should be reverted when bootstrap is fixed to not depend on building an empty stratum.
* Merge branch 'jannis/find-repos-using-git-config-option'Sam Thursfield2012-09-035-8/+142
|\ | | | | | | | | Conflicts: morphlib/plugins/branch_and_merge_plugin.py
| * Find branch root using the morph.repository option in .git/configJannis Pohlmann2012-09-033-4/+78
| | | | | | | | | | | | | | | | This is more flexible than relying on the branch root repository directory to have the original name. The user might rename the branch root directory and we still want to be able to find it. Add a test that this new functionality works.
| * Fix branch dir search by not aborting when seeing a directory twiceJannis Pohlmann2012-08-311-4/+3
| | | | | | | | | | | | Instead, simply clear the list of subdirs to recurse into and continue. Also make the filtering of hidden subdirectories a one-liner.
| * Remember the repo that is cloned from during branch/checkout/editJannis Pohlmann2012-08-313-0/+61
| | | | | | | | | | | | | | | | | | | | | | We do this by storing a morph.repository option in the local clone's .git/config file. This way the user can rename local repositories or move them around in a system branch in whatever way he or she likes and we can still find the repository later by the same name. Previously, we could only identify repositories in a system branch by their directory name. Now things are more flexible and tolerant of "unexpected" user behavior.
* | Merge branch 'jannis/git-config-for-branch-options'Sam Thursfield2012-09-037-28/+32
|\ \
| * | Use git config to store the branch root in .morph-system-branch/configJannis Pohlmann2012-08-317-28/+32
| |/ | | | | | | | | | | | | This way we can store more branch config options in the future more conveniently without having store them all in separate files or writing our own code to parse the options into a branch config object or something like that.
* | README: Hopefully we won't silently ignore invalid keys foreverSam Thursfield2012-09-031-1/+1
| |
* | README: Fix example system to use full triples for strataSam Thursfield2012-09-031-2/+10
| |
* | README: Fix example stratum to use "repo" for chunksSam Thursfield2012-09-031-1/+4
| |
* | Merge remote-tracking branch 'origin/liw/joe-ioerror-filenames'Daniel Silverstone2012-09-031-7/+6
|\ \
| * | Fix error reporting from tar extract errorsLars Wirzenius2012-08-311-7/+6
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mutliple issues here: * tarfile can throw IOErrors in addition to OSErrors, and both may need the filename attribute set * tarfile needs to be told to raise exceptions for errors, rather than ignoring them (errorlevel=1 to the constructor) * tarfile needs to be told to raise exceptions for tar format corruptions and not just I/O errors (errorlevel=2) * the exception should be re-raised with a plain "raise" instead of "raise e" to preserve the stack trace correctly; it's OK to modify the exception before re-raising it Also moved the creating of tf to the rest of the code that sets it up, for clarity. I've tested these changes manually, but not written a test case, since a test case is tricky to write for this. Originally found by Joe Burmeister.
* | Merge branch 'joeldillon/documentation'Lars Wirzenius2012-08-311-5/+35
|\ \ | |/ |/| | | | | Made some tweaks to fix a markup problem and to clarify the wording of what chunks are and to add a link to the baserock.com site.
| * Modified man page to go into more detail about what morph doesJoel Dillon2012-08-311-5/+29
|/
* Rename 'minedir' to 'workspace' in testsSam Thursfield2012-08-305-0/+0
|
* Merge branch 'jannis/show-branch-root-and-cwd-fixes'Sam Thursfield2012-08-3012-11/+354
|\
| * Fix branch/checkout to always create branches in the workspace directoryJannis Pohlmann2012-08-306-11/+250
| | | | | | | | | | | | | | | | | | Doing this rather than creating the system branches in the current working directory allows "morph branch" and "morph checkout" to be run anywhere in the workspace (e.g. in a different branch). This commit also adds two tests to verify that new branches are always created in the toplevel workspace directory.
| * Add "morph show-branch-root" commandJannis Pohlmann2012-08-307-0/+104
|/ | | | | | | | This commit introduces a new "morph show-branch-root" command to print the repository that the user branched off from with "morph branch" or that he or she checked out with "morph checkout". Also add tests for this new command.
* Avoid editing a list while iterating itSam Thursfield2012-08-301-1/+2
| | | | | Fixes previous commit where multiple hidden directories need to be removed.
* Correctly ignore hidden dirs when searching for system branchSam Thursfield2012-08-301-1/+4
| | | | Fixes test failures after merging previous branch
* Fix spelling errorSam Thursfield2012-08-301-1/+1
|
* Merge branch 'jannis/initial-branch-and-merge-work'Sam Thursfield2012-08-3042-257/+572
|\ | | | | | | | | Conflicts: morphlib/plugins/branch_and_merge_plugin.py
| * Rename internal function added in a previous commitJannis Pohlmann2012-08-291-6/+6
| |
| * Add mandatory repository parameter to "morph branch"Jannis Pohlmann2012-08-2915-54/+51
| | | | | | | | | | | | | | | | | | This is complementary to adding a repository parameter to the "morph checkout" command. It allows to branch off arbitrary repositories rather than always branching off baserock:morphs. All affected tests are updated to provide and work with this new parameter.
| * Remember repository that was branched off from for merging and editingJannis Pohlmann2012-08-2910-36/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this commit, "morph branch" and "morph checkout" remember the repository that was branched off from (the "branch root") in a special file called $workspace/$branch/.morph-system-branch/branch-root This information is later used when checking out individual repositories using "morph edit" instead of using the previously hard-coded "$workspace/$branch/morphs" repository as the branch root. This commit also updates the "morph merge" code to handle repositories specified with aliases or as full URLs in the same way "morph checkout" does. All affected tests are updated.
| * Make "morph checkout" require a repository parameterJannis Pohlmann2012-08-299-21/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of hard-coding "baserock:morphs" as the repository we check out from, we want to allow people to check out from arbitrary repositories with system and stratum morphologies. This commit adds a mandatory repository parameter to "morph checkout". This parameter can either be an aliased repo, e.g. baserock:morphs, or a full repo URL such as ssh://gitano@git.baserock.org/baserock/morphs. When cloning the actual repository into a local directory, the following happens: For alias repos baserock:morphs and baserock:foo/bar, the repositories would be cloned into the directories $workspace/$branch/baserock:morphs and $workspace/$branch/baserock:foo/bar. For repos specified using full URLs, the scheme and .git suffix (if present) are stripped off. The above ssh example would be cloned into the following directory: $workspace/$branch/gitano@git.baserock.org/baserock/morphs This commit also adjusts all affected tests and adds a new test to verify that checking out from full repo URLs works as expected.
| * Make "morph show-system-branch" anywhere in a workspace, if possibleJannis Pohlmann2012-08-2930-68/+225
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are mainly three situations to deal with: 1. We are outside a workspace and cannot deduce the system branch. 2. We are inside a workspace and inside a branch. We can detect this by walking up from the working directory towards the workspace. If we find a .morph-system-branch in one of the parent directories, we know the branch name. If we don't find one, something is wrong. 3. We are inside a workspace but outside a branch (or partially into a branch, e.g. in foo/ where the branch is foo/bar). We can deduce the branch if we recurse into subdirectories to find a .morph-system-branch directory. Care needs to be taken to not recurse infinitely. We may also not recurse if there are multiple subdirectories as these could belong to two different branches. This commit makes "morph show-system-branch" work in all of the above scenarios. It also adds tests for all of them.
| * Use .morph-system-branch directories to identify branch directoriesJannis Pohlmann2012-08-294-25/+22
| | | | | | | | | | | | | | | | | | | | The commands "morph branch" and "morph checkout" now create a .morph-system-branch directory for each system branch. This is the per-branch analogue to the .morph directory for a workspace. The .morph-system-branch directories make it possible to store state and other information for system branches. They also make deducing the current system branch more robust and clear.
| * Rename the "morph minedir" command to "morph workspace"Jannis Pohlmann2012-08-2924-84/+88
| | | | | | | | | | | | | | | | | | | | | | | | Mine may be a nice metaphor but it is not very intuitive. Hence, this commit renames "mine" to "workspace". This affects not just the branch and merge plugin (the only code that changes) but also documentation and tests, which are updated along with the code in this commit. I decided to omit the "dir" suffix in the command because I think people know well that a workspace will be a directory in this case and that allows us to refer to this concept with one word ("workspace") instead of two words ("workspace directory").
| * Create mine directory if it does not exists when running "morph init"Jannis Pohlmann2012-08-285-7/+12
| | | | | | | | | | Until now, morph failed when the mine directory did not exist. However, there is no good reason why it can't attempt to create it.
* | Remove unused files in test/Sam Thursfield2012-08-302-10/+0
| |
* | ArtifactResolverTests: use keyword args for FakeStratumMorphologySam Thursfield2012-08-301-34/+43
| | | | | | | | Helps avoid parameter-list-blindness when reading the testcases