summaryrefslogtreecommitdiff
path: root/morphlib/git.py
Commit message (Collapse)AuthorAgeFilesLines
* Check for commit object in submodule pathbaserock/franred/check-for-commit-object-in-submodule-pathFrancisco Redondo Marchena2014-11-041-1/+1
| | | | | Fix checking for commit object in submodules which their path and name are not the same.
* Echo stderr of subcommands that do network IO when --verbose is usedSam Thursfield2014-10-291-0/+11
| | | | | | | | | | | | Morph can appear to hang in situations where it is actually waiting on a slow network operation. This change gives users a way to see the output of the subcommands that are doing the network IO (either 'wget', 'git clone' or 'git remote update'). The status information goes onto stderr, because that is where the subcommands write it. Morph tends to put its status output on stdout, but (a) some commands are machine-parsed, such as `serialise-artifact` and (b) it's tricky to get Git to put status output on stdout.
* Fix `morph edit` for non-file URIsRichard Maw2014-08-261-1/+1
| | | | | | | | | The clone_into function is non-functional when you pass it a sha1 ref. If you have a file:// URI then this doesn't get used, which is how it slipped past the tests. Reviewed-by: Lars Wirzenius +2
* Prevent git-replace refs affecting git operationsRichard Maw2014-08-211-30/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We assumed that the sha1 of the tree of the commit of the ref we care about was sufficient to cache, but `git replace` means that you need to know the state of other branches too, since anything in refs/replace can completely change what the tree you check-out is. This behaviour can be disabled globally by setting GIT_NO_REPLACE_OBJECTS, so we're going to do that. If we need to integrate a project that uses git-replace to change the contents of their git trees then we could support that by: if any(refs/replace/*): potentially_replacable_objects = [ `git rev-parse HEAD`, `git rev-parse HEAD^{commit}`, `git rev-parse HEAD^{tree}`] potentially_replacable_objects.extend( `git ls-tree -r HEAD | awk '{print $3}'`) # NOTE: doesn't handle submodules, and you'd need to expand this # set whenever you process a replacement for object in refs/replace/*: if basename(object) not in potentially_replacable_objects: continue cache_key['replacements'][basename(object)] = `git rev-parse $object` If we were to support this would need to traverse the tree anyway, doing replacements, so we may as well use libgit to do the checkout anyway, and list which replacements were used. However, since the expected use-case of `git replace` is as a better way to do history grafting, we're unlikely to need it, as it would only have any effect if it replaced the commit we were using with a different one. Rubber-stamped-by: Daniel Silverstone
* Make existing morph commands use git-fatAdam Coldrick2014-03-201-1/+9
| | | | | | When cloning a repository, the files stored using git-fat need to be pulled. This situation occurs in `morph branch`, `morph edit`, and `morph checkout`.
* morphlib.git: Refactor is_valid_sha1Richard Maw2013-09-051-2/+2
| | | | | | | It now uses a generator expression instead of a list, which makes the operation lazy. It also uses string.hexdigits instead of redefining it.
* Use Submodules class to read submodule configRichard Maw2013-08-231-6/+6
| | | | | | | | | | Previously it would use `git config --get-regex`, which while more terse, is a little arcane and the regular expression did not account for slashes or dashes in the option names, and the names usually correspond to a file path. Now we use the Submodules class, which validates that the listed commit is available.
* Add morph cross-bootstrapSam Thursfield2013-07-301-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cross-bootstrap is a way to build baserock on an architecture that does not currently have Baserock. It can be used by `morph cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact that can be used as a root filesystem with a basic build environment with a script named `native-bootstrap` which will build and install every chunk in the system. If done with a devel system, this will give you a suitable environment for building a proper Baserock system. This does not currently provide a kernel for the target architecture. Apart from adding the cross-bootstrap plugin, it also makes the following changes: * Moves the lit of valid_archs into morphlib (instead of locally-scoped in MorphologyFactory) * BuildCommand takes an extra argument, build_env * split BuildCommand's get_artifact_object into create_source_pool and resolve_artifacts (plus changes things that use get_artifact_object to use the new way) * setup_mounts finds out whether to do so by whether build_mode is 'staging', instead of by whether the setting 'staging-chroot' is true. * Makes ChunkBuilder's get_sources use the morphlib.builder2.extract_sources() method, and moved set_mtime_recursively into morphlib.builder2, since it's not currently used anywhere else. * moved ChunkBuilder's get_commands into the Morphology class (plus changes to anything that used get_commands)
* Fix lines overflowing the 80 character limitRichard Dale2013-06-251-1/+2
|
* Don't show git config messages for 'morph branch' in verbose modeRichard Dale2013-06-251-1/+1
|
* Changed the error message in log when morph fails to find gitmodulesRic Holland2013-03-201-1/+1
| | | | morph.log will now show an INFO message rther than a ERROR
* B&M: Share error message formatting codeRichard Maw2013-03-041-21/+44
| | | | | | | This makes the warning messages include how to fix the problem. morphlib.git.check_config_set will return the values of the keys which were set, so it can be used to get git config.
* B&M: Check git config before some subcommandsRichard Maw2013-03-011-0/+17
| | | | | | | | | Some subcommands use git to create commits, in which case user config needs to be set. Others imply commits may be created, e.g. by cloning a repository, at which point it is useful to have a reminder that the configuration needs to be set.
* git: clone then checkout instead of clone -bRichard Maw2013-02-111-1/+5
| | | | | | | | | Newer versions of git have changed the behaviour when cloning with --branch=$sha1. It used to be nearly synonamous with clone then checkout. The clone then checkout logic is different to the clone with branch logic, since they have different behaviour with regard to the branches that are checked out in the workspace.
* Merge remote-tracking branch 'remotes/origin/samthursfield/ambiguous-refs' ↵Jannis Pohlmann2012-12-131-1/+1
|\ | | | | | | into baserock/merge-queue
| * Always use `git rev-parse --verify` to resolve refsSam Thursfield2012-12-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Previously some code used `git show-ref`, which is wrong -- given two refs named 'alpha/master' and 'master', `git show-ref master` will return both, sorted alphabetically. This can lead to build failures, etc. due to refs resolving to the wrong SHAs. We should also use `git rev-parse --verify` to verify SHA1s, which we previously did with `git rev-list`. Finally, `git rev-parse --verify` is more than twice as fast as `git show-ref`.
* | Add an initial implementation of "morph tag"Jannis Pohlmann2012-12-131-0/+13
|/ | | | | | | | | | | | | | | | | | | | | | | | | | In order to make releases and freeze system branches entirely, we need to be able to 100% petrify a system branch (that is, resolve ALL refs into SHA1s) and tag this state to be able to check it out again later. This is essentially what "morph tag" does. It takes a tag name and an arbitrary amount of arguments to "git tag", petrifies all morphologies of the current system branch behind the scenes, creates a dangling commit and attaches an annotated tag to it. Petrifying in this case means that all refs used for chunks are resolved into commit SHA1s. For stratum and system morphologies, the refs are replaced by the name of the tag that's being created. The "tag" command also supports tagging when stratum morphologies are spread across multiple repositories. In this case, it will include all statum morphologies from other repos in the tag commi in the branch root repo. The references to these morphologies are updated so that they point to the branch root repo and the tag being created. This commit also adds a few tests for "morph tag" to verify that all this works.
* Avoid caching repos that are on the local machine anywaySam Thursfield2012-11-121-1/+6
| | | | | Currently the message is still displayed "Updating xxx" but no update is actually done.
* Handle submodules in 'morph edit'Sam Thursfield2012-10-031-0/+22
| | | | | | | | | | | | | | | | We often have .gitmodules edited to contain a URI such as upstream:gnulib, so that we can transparently mirror these in different locations. It would be nice to set up git url.insteadOf rules to expand these for the submodules, but 'git submodule update' uses 'git clone' to fetch them, which will not take into account the configuration of the parent repository. Instead, we set up the submodules automatically and rewrite the URLs directly in the configuration. The user will need to recreate their system branch checkouts if their URL configuration changes, or update the URLs manually, but that should not happen often.
* Use origin/ref for all refs other than the system branch ref itselfSam Thursfield2012-10-031-5/+4
| | | | | | | | | | The rationale here is that inside a checkout of a system branch, the user should only be committing to the refs for that system branch, because those are the only ones that 'morph merge' will look at. This removes a problem where we could be confused by a ref with a name that would be sorted before 'master' in the result of 'git show-ref'.
* Extract is_valid_sha1() to morphlib.gitSam Thursfield2012-10-031-0/+6
|
* morph merge: Don't make a merge commit if only the refs changedSam Thursfield2012-09-141-0/+11
| | | | 'git commit' fails if there are no changes in any case.
* Rework git caches to be bare mirrors of the repos.Daniel Silverstone2012-09-141-2/+49
| | | | | | | | This reworks the code for managing and using the git caches in morph to treat the caches as bare repositories which are mirrors of where we clone from. In addition we correctly prune the branches during updates, so that we don't end up accumulating pointless branches over and over. This is even more important with branch-and-merge generating temporary build refs for things.
* build: Fail if user hasn't set user.name (or GIT_AUTHOR_NAME)Sam Thursfield2012-09-131-0/+12
| | | | | | Users in a VM may be running as 'root' so inferring from the username is often useless. Since build branches are pushed to a server it is useful to enforce useful onwership info.
* Git rid of morphlib.git methods that are not neededSam Thursfield2012-09-101-8/+0
|
* morph branch: Fail if branch already exists in the system repoSam Thursfield2012-09-101-0/+8
|
* Rewrite 'merge' commandSam Thursfield2012-09-101-1/+0
| | | | | | 'merge' now traverses every system in the branch root (baserock:morphs) and merges any chunk or stratum that was changed with 'morph edit'. It also takes care of updating the refs in the target branch.
* python scripts: pep8ize codebaseRichard Maw2012-08-011-8/+11
| | | | | | | | | This was done with the aid of the pep8 script, available by running `easy_install pep8`. It may be worth making this part of ./check, but that will require putting pep8 into the development tools stratum. This should be easy, given pep8 has no external dependencies.
* Remove optional msg argument from Submodule classLars Wirzenius2012-06-251-3/+2
| | | | | Nobody was using this, so it is obviously useless. Instead, log at the warning level.
* morph: remove dead code and replace Execute with app.runcmdRichard Maw2012-05-301-133/+20
|
* morphlib.git: fix submodule format stringRichard Maw2012-04-131-1/+1
|
* Remove final use of Treeish in submodules codeLars Wirzenius2012-04-121-3/+3
|
* Remove Treeish usage in submodule codeLars Wirzenius2012-04-121-24/+26
| | | | | This makes it possible to use the Submodule and Submodules classes in the refactored code that gets rid of Treeish completely.
* Make copy_repository get a repo instead of treeishLars Wirzenius2012-04-121-2/+2
| | | | This helps refactoring morph to get rid of treeishes.
* builder: ensure working directory matches commitRichard Maw2012-04-111-0/+7
| | | | | | | | When making a copy of the repository to build from, builder copies the .git directory then checks out the ref it wants. However sometimes this doesn't add the files to the working directory, possibly because the files it is missing weren't changed in the difference between what the .git thought HEAD was and what is now checked out.
* Refactor SystemBuilder and add helper classLars Wirzenius2012-03-021-1/+1
| | | | | | | | The helper class, Factory, has unit tests, which is why it's currently separate. It may later get integrated with BlobBuilder, or the other way around. Classes that don't have unit tests are marked out of coverage.
* Use "cp -a" and "git checkout" to unpack sources in the build tree.Jannis Pohlmann2012-02-221-11/+16
| | | | | | | | | This is done to keep the git repository intact with history and all that. cp + checkout turns out to be faster than a regular "git clone". In order to avoid roundtrips to the internet whenever a chunk tries to do something with submodules, we force the submodule URLs to point to our locally cached repos.
* Handle empty/short lines when validating submodulesLars Wirzenius2012-02-171-1/+1
|
* Switch to caching repos without checking them out.Jannis Pohlmann2012-02-141-5/+6
| | | | | | | | | | | | | This is almost as good as --bare. But unlike --bare, it properly sets up the 'origin' remote and creates the 'refs/remotes/origin/...' branches. When cloning with --bare, and you end up with "refs/heads/master", not "/refs/remotes/origin/master". You can re-add the remote 'origin', still no "refs/remotes/origin/master". Only after you update the remote (after re-adding it), the bare clone has "refs/remotes/origin/master"... and a "refs/heads/master" that is out of sync. An ugly mix, basically. So maybe this approach is better.
* Ignore submodules for which the parent repo has a non-commit object.Jannis Pohlmann2012-02-091-22/+26
| | | | | | This usually means that these submodules have been imported into the source tree and no longer need to be updated from their original repository.
* Change how git clones are created from bundlesLars Wirzenius2012-02-091-4/+4
| | | | | | For some reason, "git bundle unbundle" doesn't work for me, but "git clone" does. Then, adding a remote fails, because the cloned bundle already has one, so now we'll use "git remote set-url" and "git remote update".
* Extract chunk submodule repos into the build directory directly.Jannis Pohlmann2012-02-081-1/+11
| | | | | | | | | | | | | This change requires the latest version of cliapp to be installed. Rather than creating an archive from the cached git repositories, moving this tarball archive into the cache and then extracting it from there into the build directory, we now directly extract the sources into the build directory. We also recursively extract all submodules into the corresponding paths inside the build directory. (Recursing has not been tested yet, however.)
* Cache submodule repositories recursively.Jannis Pohlmann2012-02-081-6/+121
| | | | | | | | | | | | | | | | | | | | | | This commit introduces the following new classes: morphlib.git.Submodules: * takes a parent repo treeish * parses the .gitmodule file into morphlib.git.Submodule objects * provides iterator/container functionality for submodules morphlib.git.Submodule: * represents a single entry in a .gitmodules file * stores a Treeish for the corresponding repository In addition to this, the exception classes InvalidTreeish and SourceNotFound where renamed to InvalidReferenceError and RepositoryUpdateError. Several new exception classes were added for when resolving submodules fails. The SourceManager now resolves the Submodules and Submodule objects for the submodules of a Treeish in SourceManager.get_treeish() and also takes care of caching submodule repositories whenever necessary.
* Set GIT_DIR when running git archive. Drop --remote.Jannis Pohlmann2012-02-021-4/+4
| | | | | This change is necessary because upstream git no longer allows --remote to be combined with an arbitrary SHA1.
* Always specify '-l' when cloning in order to use hardlinking.Jannis Pohlmann2012-02-011-1/+1
|
* Bump copyright years, get rid of long lines and unused imports.Jannis Pohlmann2012-01-231-1/+0
|
* Improve the logging output of SourceManager.Jannis Pohlmann2012-01-231-12/+12
|
* Port everything to using Treeish objects instead of (repo, ref).Jannis Pohlmann2012-01-201-30/+37
| | | | | | | | | | | This affects pretty much every part of morph, so this might not be fully working and stable yet. This commit also introduces the "update-gits" command that can be used to update all cached repositories from the list of base URLs. The tree walk when resolving the Treeish objects in Builder.get_cache_id() is a bit similar to what we do in BuildDependencyGraph, maybe we can merge that one day.
* Document get_treeish() of the SourceManager.Jannis Pohlmann2012-01-191-2/+1
|
* Break long lines in git.py.Jannis Pohlmann2012-01-191-2/+5
|
View Lorry Controller Status