summaryrefslogtreecommitdiff
path: root/morphlib/localrepocache.py
Commit message (Collapse)AuthorAgeFilesLines
* Use the modern way of the GPL copyright header: URL instead real addressJavier Jardón2015-03-161-2/+1
| | | | Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
* Merge branch 'sam/fewer-submodule-updates'Sam Thursfield2015-03-031-13/+63
|\ | | | | | | | | Reviewed-By: Javier Jardón <javier.jardon@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * Only update Git submodules in cache when necessarySam Thursfield2015-02-101-14/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This saves a duplicate `git remote update origin` that was being run as part of each chunk build. For any repos that have submodules, it also avoids updating repos if the SHA1 we need to build is already present locally. As well as speeding up builds slightly, this means Morph can now build without being connected to a network, as long as the local Git cache all of the necessary repos and commits in the build, without needing the '--no-git-update' option. The code is also now in a more logical place than before.
* | sourceresolver: Factor out 'cache repo locally' code into a functionSam Thursfield2015-02-201-2/+4
|/ | | | | | | Also, move the repo.update() call into the 'fetch from tarball' code path in the localrepocache module -- there's no need to update straight after doing a `git clone`, but we do need to do one if we got the repo from a `git archive` tarball that may be out of date.
* Rework CachedRepo to use the GitDirectory class where possibleSam Thursfield2014-12-031-2/+6
| | | | | | | | | | | | | This consolidates a bunch of code paths that were previously duplicated. This also changes the API for local cached repos to match the function names GitDirectory uses. Note that the remote repo cache still uses the old names, and should be fixed when time permits. Some unit tests that use the CachedRepo module required a bit of inelegant monkey-patching in order that they continue to work. A better way to do this would be with the 'mock' library (which would need to be added to Baserock 'build' and 'devel' systems before we could use it).
* Echo stderr of subcommands that do network IO when --verbose is usedSam Thursfield2014-10-291-5/+21
| | | | | | | | | | | | Morph can appear to hang in situations where it is actually waiting on a slow network operation. This change gives users a way to see the output of the subcommands that are doing the network IO (either 'wget', 'git clone' or 'git remote update'). The status information goes onto stderr, because that is where the subcommands write it. Morph tends to put its status output on stdout, but (a) some commands are machine-parsed, such as `serialise-artifact` and (b) it's tricky to get Git to put status output on stdout.
* Don't say 'Updating git repository xxx' unless we actually areSam Thursfield2014-10-291-1/+2
| | | | | | | This fixes an issue where various branch-and-merge commands appeared to be updating Git repos even when --no-git-update was specified. The flag was actually honoured but the message made it seem as if it was being ignored.
* Fix confusing messagebaserock/ps/message-fixPaul Sherwood2014-10-271-1/+1
|
* Prevent git-replace refs affecting git operationsRichard Maw2014-08-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We assumed that the sha1 of the tree of the commit of the ref we care about was sufficient to cache, but `git replace` means that you need to know the state of other branches too, since anything in refs/replace can completely change what the tree you check-out is. This behaviour can be disabled globally by setting GIT_NO_REPLACE_OBJECTS, so we're going to do that. If we need to integrate a project that uses git-replace to change the contents of their git trees then we could support that by: if any(refs/replace/*): potentially_replacable_objects = [ `git rev-parse HEAD`, `git rev-parse HEAD^{commit}`, `git rev-parse HEAD^{tree}`] potentially_replacable_objects.extend( `git ls-tree -r HEAD | awk '{print $3}'`) # NOTE: doesn't handle submodules, and you'd need to expand this # set whenever you process a replacement for object in refs/replace/*: if basename(object) not in potentially_replacable_objects: continue cache_key['replacements'][basename(object)] = `git rev-parse $object` If we were to support this would need to traverse the tree anyway, doing replacements, so we may as well use libgit to do the checkout anyway, and list which replacements were used. However, since the expected use-case of `git replace` is as a better way to do history grafting, we're unlikely to need it, as it would only have any effect if it replaced the commit we were using with a different one. Rubber-stamped-by: Daniel Silverstone
* Refactored localrepocacheDaniel Firth2013-12-201-96/+34
|
* Add LocalRepoCache.get_updated_repo methodLars Wirzenius2013-08-061-1/+13
| | | | | This is a helper to avoid having the same code in other places. Have it in the one place it belongs.
* Avoid caching repos that are on the local machine anywaySam Thursfield2012-11-121-3/+7
| | | | | Currently the message is still displayed "Updating xxx" but no update is actually done.
* Use tarballs instead of bundlesDaniel Silverstone2012-09-251-24/+40
| | | | | | | This makes a non-backward-compatible change to morph which switches it to using tarballs instead of bundles when initialising cached git repositories. This is faster because it doesn't require index-pack --fix-thin operations on the machine running morph.
* Refactor _fetch() and log bundle fetches.Daniel Silverstone2012-09-211-11/+18
| | | | | | Log (at chatty) when we start and finish bundle downloads. If we fail for any reason, log that reason since otherwise it gets swallowed if the plain git clone succeeds.
* Rework git caches to be bare mirrors of the repos.Daniel Silverstone2012-09-141-2/+2
| | | | | | | | This reworks the code for managing and using the git caches in morph to treat the caches as bare repositories which are mirrors of where we clone from. In addition we correctly prune the branches during updates, so that we don't end up accumulating pointless branches over and over. This is even more important with branch-and-merge generating temporary build refs for things.
* python scripts: pep8ize codebaseRichard Maw2012-08-011-47/+46
| | | | | | | | | This was done with the aid of the pep8 script, available by running `easy_install pep8`. It may be worth making this part of ./check, but that will require putting pep8 into the development tools stratum. This should be easy, given pep8 has no external dependencies.
* morph: remove dead code and replace Execute with app.runcmdRichard Maw2012-05-301-6/+9
|
* Change LocalRepoCache to use the new RepoAliasResolver.Jannis Pohlmann2012-05-041-64/+6
|
* Fix LocalRepoCache to not abort on %s-less repo patternsLars Wirzenius2012-05-041-3/+17
| | | | Also, add debugging messages for debugging prefix expansion.
* Remove use of baseurls in LocalRepoCacheLars Wirzenius2012-05-041-33/+23
| | | | The tests, they still fail.
* Implement repo-alias expansionLars Wirzenius2012-05-041-3/+42
| | | | | | | | Changed the delimieter between push and pull patterns to be #, since = is reasonably frequent in URLs, and # should never be necessary in git urls. Oh my how many tests now fail.
* Add tests for repo-alias expansionLars Wirzenius2012-05-041-0/+6
|
* Get rid of the old internal morph APIsLars Wirzenius2012-05-021-5/+18
|
* Add morphlib.Error and make local repo cache exceptions be based on itLars Wirzenius2012-04-251-2/+3
| | | | | This way, cliapp will automatically print an error message, rather than a stack trace.
* Fix LocalRepoCache tests.Jannis Pohlmann2012-04-191-1/+1
|
* Clone repos with no checkout, use correct gits cache dir in cmd_build.Jannis Pohlmann2012-04-191-2/+2
|
* Remember repo objects in LocalRepoCache to always return the same ones.Jannis Pohlmann2012-04-171-3/+10
|
* localrepocache: return repo after cloningRichard Maw2012-04-111-5/+14
|
* localrepocache: fixes to remote set-urlRichard Maw2012-04-111-3/+6
| | | | | | It needed shutil, but that exception was being missed. _git needs to be able to handle a different current directory for remote set-url to work
* localrepocache: fix remote origin url to upstreamRichard Maw2012-04-111-0/+1
| | | | | | When cloning from a bundle it makes the origin remote url point to the bundle. This is not ideal as we want to update these repositories later from upstream.
* localrepocache: fix bundle pathsRichard Maw2012-04-111-1/+1
| | | | They must end with .bndl as that is the format lorry generates
* localrepocache: return error descriptionRichard Maw2012-04-111-22/+43
| | | | | | | | Also change _fetch to throw its exception as since it is more pythonic to throw exceptions than return exit status, other error handling needed to be caught, so it hsa been kept consistent. This required the tests to throw an exception as well
* localrepocache: check every base-url before cloneRichard Maw2012-04-111-5/+7
| | | | | It should check that it doesn't already have the repository cached under a different base-url before cloning, likewise with bundles.
* Create cache directory when needed in LocalRepoCacheLars Wirzenius2012-04-101-0/+12
|
* Fix unit test failures: mark out-of-coverage, and use wrapper to allow testsLars Wirzenius2012-04-101-6/+5
| | | | | | | | The self._exists method needs to be used instead of raw os.path.exists, so that unit tests can override it easily. Also, a tiny clarification so that the caller of _clone_with_bundle decides whether it's OK to try to download a bundle.
* Various small fixes to make the new update-gits work again.Jannis Pohlmann2012-04-101-15/+23
|
* Use the real CachedRepo class in LocalRepoCacheLars Wirzenius2012-04-091-9/+1
|
* Remove downloaded bundle after it has been clonedLars Wirzenius2012-04-091-0/+10
|
* Add bundle support to LocalRepoCacheLars Wirzenius2012-04-091-2/+47
|
* Add a LocalRepoCache classLars Wirzenius2012-04-091-0/+144
Jannis and I discussed a refactoring of most of morph's internals to reduce coupling and increase cohesion. This is one of the results: we'll want a couple of classes to manage locally cached git repositories. This commit adds the LocalRepoCache class to manage a the git cache directory. Later on, we'll add the CachedRepo class to represent individual repositories.
View Lorry Controller Status