| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
| |
The clone_into function is non-functional when you pass it a sha1 ref.
If you have a file:// URI then this doesn't get used, which is how it
slipped past the tests.
Reviewed-by: Lars Wirzenius +2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We assumed that the sha1 of the tree of the commit of the ref we care
about was sufficient to cache, but `git replace` means that you need to
know the state of other branches too, since anything in refs/replace can
completely change what the tree you check-out is.
This behaviour can be disabled globally by setting
GIT_NO_REPLACE_OBJECTS, so we're going to do that.
If we need to integrate a project that uses git-replace to change the
contents of their git trees then we could support that by:
if any(refs/replace/*):
potentially_replacable_objects = [
`git rev-parse HEAD`,
`git rev-parse HEAD^{commit}`,
`git rev-parse HEAD^{tree}`]
potentially_replacable_objects.extend(
`git ls-tree -r HEAD | awk '{print $3}'`)
# NOTE: doesn't handle submodules, and you'd need to expand this
# set whenever you process a replacement
for object in refs/replace/*:
if basename(object) not in potentially_replacable_objects: continue
cache_key['replacements'][basename(object)] = `git rev-parse $object`
If we were to support this would need to traverse the tree anyway, doing
replacements, so we may as well use libgit to do the checkout anyway,
and list which replacements were used.
However, since the expected use-case of `git replace` is as a better way
to do history grafting, we're unlikely to need it, as it would only have
any effect if it replaced the commit we were using with a different one.
Rubber-stamped-by: Daniel Silverstone
|
|
|
|
|
|
| |
When cloning a repository, the files stored using git-fat need to
be pulled. This situation occurs in `morph branch`, `morph edit`,
and `morph checkout`.
|
|
|
|
|
|
|
| |
It now uses a generator expression instead of a list, which makes the
operation lazy.
It also uses string.hexdigits instead of redefining it.
|
|
|
|
|
|
|
|
|
|
| |
Previously it would use `git config --get-regex`, which while more
terse, is a little arcane and the regular expression did not account
for slashes or dashes in the option names, and the names usually
correspond to a file path.
Now we use the Submodules class, which validates that the listed commit
is available.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cross-bootstrap is a way to build baserock on an architecture that
does not currently have Baserock. It can be used by `morph
cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact
that can be used as a root filesystem with a basic build environment
with a script named `native-bootstrap` which will build and install
every chunk in the system.
If done with a devel system, this will give you a suitable environment
for building a proper Baserock system.
This does not currently provide a kernel for the target architecture.
Apart from adding the cross-bootstrap plugin, it also makes the
following changes:
* Moves the lit of valid_archs into morphlib (instead of locally-scoped
in MorphologyFactory)
* BuildCommand takes an extra argument, build_env
* split BuildCommand's get_artifact_object into create_source_pool and
resolve_artifacts (plus changes things that use get_artifact_object to
use the new way)
* setup_mounts finds out whether to do so by whether build_mode is
'staging', instead of by whether the setting 'staging-chroot' is true.
* Makes ChunkBuilder's get_sources use the
morphlib.builder2.extract_sources() method, and moved
set_mtime_recursively into morphlib.builder2, since it's not currently
used anywhere else.
* moved ChunkBuilder's get_commands into the Morphology class (plus
changes to anything that used get_commands)
|
| |
|
| |
|
|
|
|
| |
morph.log will now show an INFO message rther than a ERROR
|
|
|
|
|
|
|
| |
This makes the warning messages include how to fix the problem.
morphlib.git.check_config_set will return the values of the
keys which were set, so it can be used to get git config.
|
|
|
|
|
|
|
|
|
| |
Some subcommands use git to create commits, in which case user config
needs to be set.
Others imply commits may be created, e.g. by cloning a repository, at
which point it is useful to have a reminder that the configuration needs
to be set.
|
|
|
|
|
|
|
|
|
| |
Newer versions of git have changed the behaviour when cloning with
--branch=$sha1. It used to be nearly synonamous with clone then checkout.
The clone then checkout logic is different to the clone with branch
logic, since they have different behaviour with regard to the branches
that are checked out in the workspace.
|
|\
| |
| |
| | |
into baserock/merge-queue
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously some code used `git show-ref`, which is wrong -- given two
refs named 'alpha/master' and 'master', `git show-ref master` will
return both, sorted alphabetically. This can lead to build failures,
etc. due to refs resolving to the wrong SHAs.
We should also use `git rev-parse --verify` to verify SHA1s, which
we previously did with `git rev-list`.
Finally, `git rev-parse --verify` is more than twice as fast as
`git show-ref`.
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to make releases and freeze system branches entirely, we
need to be able to 100% petrify a system branch (that is, resolve
ALL refs into SHA1s) and tag this state to be able to check it out
again later.
This is essentially what "morph tag" does. It takes a tag name
and an arbitrary amount of arguments to "git tag", petrifies all
morphologies of the current system branch behind the scenes, creates
a dangling commit and attaches an annotated tag to it.
Petrifying in this case means that all refs used for chunks are
resolved into commit SHA1s. For stratum and system morphologies,
the refs are replaced by the name of the tag that's being created.
The "tag" command also supports tagging when stratum morphologies
are spread across multiple repositories. In this case, it will
include all statum morphologies from other repos in the tag commi
in the branch root repo. The references to these morphologies are
updated so that they point to the branch root repo and the tag
being created.
This commit also adds a few tests for "morph tag" to verify that
all this works.
|
|
|
|
|
| |
Currently the message is still displayed "Updating xxx" but no
update is actually done.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We often have .gitmodules edited to contain a URI such as
upstream:gnulib, so that we can transparently mirror these in different
locations.
It would be nice to set up git url.insteadOf rules to expand these for
the submodules, but 'git submodule update' uses 'git clone' to fetch
them, which will not take into account the configuration of the
parent repository.
Instead, we set up the submodules automatically and rewrite the URLs
directly in the configuration. The user will need to recreate their
system branch checkouts if their URL configuration changes, or update
the URLs manually, but that should not happen often.
|
|
|
|
|
|
|
|
|
|
| |
The rationale here is that inside a checkout of a system branch, the
user should only be committing to the refs for that system branch,
because those are the only ones that 'morph merge' will look at.
This removes a problem where we could be confused by a ref with a
name that would be sorted before 'master' in the result of 'git
show-ref'.
|
| |
|
|
|
|
| |
'git commit' fails if there are no changes in any case.
|
|
|
|
|
|
|
|
| |
This reworks the code for managing and using the git caches in morph to treat
the caches as bare repositories which are mirrors of where we clone from. In
addition we correctly prune the branches during updates, so that we don't end
up accumulating pointless branches over and over. This is even more important
with branch-and-merge generating temporary build refs for things.
|
|
|
|
|
|
| |
Users in a VM may be running as 'root' so inferring from the username
is often useless. Since build branches are pushed to a server it is
useful to enforce useful onwership info.
|
| |
|
| |
|
|
|
|
|
|
| |
'merge' now traverses every system in the branch root (baserock:morphs)
and merges any chunk or stratum that was changed with 'morph edit'. It
also takes care of updating the refs in the target branch.
|
|
|
|
|
|
|
|
|
| |
This was done with the aid of the pep8 script, available by running
`easy_install pep8`.
It may be worth making this part of ./check, but that will require
putting pep8 into the development tools stratum.
This should be easy, given pep8 has no external dependencies.
|
|
|
|
|
| |
Nobody was using this, so it is obviously useless. Instead, log at the
warning level.
|
| |
|
| |
|
| |
|
|
|
|
|
| |
This makes it possible to use the Submodule and Submodules classes
in the refactored code that gets rid of Treeish completely.
|
|
|
|
| |
This helps refactoring morph to get rid of treeishes.
|
|
|
|
|
|
|
|
| |
When making a copy of the repository to build from, builder copies the
.git directory then checks out the ref it wants.
However sometimes this doesn't add the files to the working directory,
possibly because the files it is missing weren't changed in the difference
between what the .git thought HEAD was and what is now checked out.
|
|
|
|
|
|
|
|
| |
The helper class, Factory, has unit tests, which is why it's currently
separate. It may later get integrated with BlobBuilder, or the other
way around.
Classes that don't have unit tests are marked out of coverage.
|
|
|
|
|
|
|
|
|
| |
This is done to keep the git repository intact with history and all
that. cp + checkout turns out to be faster than a regular "git clone".
In order to avoid roundtrips to the internet whenever a chunk tries to
do something with submodules, we force the submodule URLs to point to
our locally cached repos.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is almost as good as --bare. But unlike --bare, it properly sets up
the 'origin' remote and creates the 'refs/remotes/origin/...' branches.
When cloning with --bare, and you end up with "refs/heads/master", not
"/refs/remotes/origin/master". You can re-add the remote 'origin', still
no "refs/remotes/origin/master". Only after you update the remote (after
re-adding it), the bare clone has "refs/remotes/origin/master"... and a
"refs/heads/master" that is out of sync. An ugly mix, basically.
So maybe this approach is better.
|
|
|
|
|
|
| |
This usually means that these submodules have been imported into the
source tree and no longer need to be updated from their original
repository.
|
|
|
|
|
|
| |
For some reason, "git bundle unbundle" doesn't work for me, but "git clone" does.
Then, adding a remote fails, because the cloned bundle already has one, so now
we'll use "git remote set-url" and "git remote update".
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change requires the latest version of cliapp to be installed.
Rather than creating an archive from the cached git repositories,
moving this tarball archive into the cache and then extracting it from
there into the build directory, we now directly extract the sources into
the build directory.
We also recursively extract all submodules into the corresponding paths
inside the build directory. (Recursing has not been tested yet,
however.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit introduces the following new classes:
morphlib.git.Submodules:
* takes a parent repo treeish
* parses the .gitmodule file into morphlib.git.Submodule objects
* provides iterator/container functionality for submodules
morphlib.git.Submodule:
* represents a single entry in a .gitmodules file
* stores a Treeish for the corresponding repository
In addition to this, the exception classes InvalidTreeish and
SourceNotFound where renamed to InvalidReferenceError and
RepositoryUpdateError. Several new exception classes were added for when
resolving submodules fails.
The SourceManager now resolves the Submodules and Submodule objects for
the submodules of a Treeish in SourceManager.get_treeish() and also
takes care of caching submodule repositories whenever necessary.
|
|
|
|
|
| |
This change is necessary because upstream git no longer allows --remote
to be combined with an arbitrary SHA1.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This affects pretty much every part of morph, so this might not be fully
working and stable yet.
This commit also introduces the "update-gits" command that can be used
to update all cached repositories from the list of base URLs. The tree
walk when resolving the Treeish objects in Builder.get_cache_id() is a
bit similar to what we do in BuildDependencyGraph, maybe we can merge
that one day.
|
| |
|
| |
|
| |
|
| |
|