| Commit message (Collapse) | Author | Age | Files | Lines |
|\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
ssh://git.baserock.org/baserock/baserock/morph
This includes the following fixups:
- altering the bootstrap script to install ordereddict and simplejson.
- Adding a comment to clarify that it is intentional to use simplejson
if collections does not have OrderedDict
- Amending the copyright years to include 2013
|
| |
| |
| |
| |
| | |
As we use ordererdict for python < 2.7 as well
See 37f8d84d7ab356169bf7c04bdecb39a61b85c88d
|
|/ |
|
|\
| |
| |
| | |
Fixed a copyright year to make ./check happy.
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch makes morph store its ccaches by the url's basename instead
of using the whole url. This fixes the problem of morph choking when it
tries to mount a directory with a colon in its path, and also makes
all clones of one repository use the same cache (probably). This will
cause projects with the same name to use the same cache, but that is
not likely to cause serious problems.
|
|/
|
|
|
|
|
|
| |
glob.glob() can return results in any order. In practice it varies
at least according to the file system being used. This means that test
results may be different. To avoid this, we should always use
sorted(glob.iglob()) instead, so that the code always behaves in the
same way given the same inputs.
|
|\ |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
The code took some refactoring. The core functionality is now all inside
one function with make_available() separate, as this is used other places.
The code is still far from perfect, but will hopefully be rewritten to
use the new abstractions of system branches etc. soon
|
|/
|
|
|
| |
This ensures that we don't try to build the same stratum from two different
refs.
|
|
|
|
| |
Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is set up so that each individual project repository has its own
ccache, all under one defined directory.
The top-level ccache directory is added as the setting
'compiler-cache-dir', and defaults to $cachedir/ccache.
When a build is performed, this will bind-mount a project's ccache into
the /tmp/ccache of the staging-area and set up the environment variables
so that ccache will be used (if appropriate executables are installed
to /usr/lib/ccache in the staging-area).
In addition, this removes code for ccache-remotedir, as it is unrelated
to this implementation of ccache.
Reviewed-by: Jannis Pohlmann <jannis.pohlmann@codethink.co.uk>
Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
|
|
|
|
|
|
|
|
|
|
|
| |
The committer information in the environment used to run git in morph
tag is not needed. In morph build it makes sense as morph commits without
the user knowing. With morph tag, it's the user that decides to create
the commit and tag.
There is something weird going on, where morph tag may end up generating
commits with different SHA1s on different machines. The full log output
in the morph tag tests might help investigate what happens.
|
|\
| |
| |
| | |
Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
|
| |
| |
| |
| |
| |
| |
| | |
This cuts down on the number of HTTP requests made while computing th
build order (> create source pool > traverse morphs). In an example
system I was building this patch reduced the requests from 501 to 195,
reducing the time to building the first chunk from 123s to 18s.
|
|\ \
| | |
| | |
| | | |
into baserock/merge-queue
|
| | |
| | |
| | |
| | |
| | | |
This was done to ensure tests.branching/branch-fails-if-branch-exists
always passes, but also seems like the right approach in general.
|
| |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously some code used `git show-ref`, which is wrong -- given two
refs named 'alpha/master' and 'master', `git show-ref master` will
return both, sorted alphabetically. This can lead to build failures,
etc. due to refs resolving to the wrong SHAs.
We should also use `git rev-parse --verify` to verify SHA1s, which
we previously did with `git rev-list`.
Finally, `git rev-parse --verify` is more than twice as fast as
`git show-ref`.
|
| | |
|
|\ \
| |/
|/| |
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to make releases and freeze system branches entirely, we
need to be able to 100% petrify a system branch (that is, resolve
ALL refs into SHA1s) and tag this state to be able to check it out
again later.
This is essentially what "morph tag" does. It takes a tag name
and an arbitrary amount of arguments to "git tag", petrifies all
morphologies of the current system branch behind the scenes, creates
a dangling commit and attaches an annotated tag to it.
Petrifying in this case means that all refs used for chunks are
resolved into commit SHA1s. For stratum and system morphologies,
the refs are replaced by the name of the tag that's being created.
The "tag" command also supports tagging when stratum morphologies
are spread across multiple repositories. In this case, it will
include all statum morphologies from other repos in the tag commi
in the branch root repo. The references to these morphologies are
updated so that they point to the branch root repo and the tag
being created.
This commit also adds a few tests for "morph tag" to verify that
all this works.
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| | |
This changes the cache key generation so that it will ignore unimportant
fields of the morphology, e.g. description, build-depends, chunks.
description is unimportant because it does not affect building, and
build-depends/chunks are unimportant because they are already considered
|
| |
| |
| |
| |
| | |
The lookup method will only find the first entry, but previously
would allow later entries to be found by iterating.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Before this change, if a stratum had dependencies listed, then
morph would add that stratum to the list of things to build
multiple times, resulting in more things being built.
This was not a major problem, since it would just result in more cache
lookups than necessary, but lookups are more expensive on remote
artifact caches, and anything attempting to find which artifact would
be the result of a build would get confused by there being many
top-level artifacts with no dependencies.
|
| |
| |
| |
| |
| |
| |
| |
| | |
The test relied on undesired behaviour, since it would create
the same source, since the sha1 is not used in the sourcepool's key.
Now it creates a new source pool and replaces the source it is being
compared to.
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit c859d70d86423a52bc7053abf64e9ca21f62a487, reversing
changes made to 3085a672d1e8b5177be33f0463385de72a0ef5bf.
Unfortunately, hardlinking and linux-user-chroot both break builds
in various ways. Hardlinking breaks the bootstrap process by creating
symlinks like /usr/libexec that can then not be overwritten with real
files by install scripts from morphologies like gcc.
linux-user-chroot caused problems by breaking privileged operations such
as chgrp and CAP_SETFCAP. As a consequence, chunks like util-linux and
libcap can no longer be built.
|
|
|
|
| |
This also makes coverage.py be silent on Debian squeeze.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The patch gives two things.
Improves morph build time by reusing decompressed files of
chunks/stage-fillers with hardlinks from the chroot. Rather than
decompressing each time into each chroot.
Original: real 5h 17m 47s
Hardlink: real 2h 52m 27s
It uses linux-user-chroot to create the chroot and make all but
the basics readonly.
|
|
|
|
|
|
|
| |
gzip files can store the name and date of the uncompressed original, and
this is displayed in some GUI archive managers, or when gunzip -N is used.
In our case the original filename is something like 'tmpXsgaY15', which
is useless information and serves only to confuse.
|
|
|
|
| |
Includes new test.
|
|
|
|
| |
Reported upstream as: http://bugs.python.org/issue16477
|
|
|
|
|
|
|
| |
This prevents us from leaving file handles open when code throws an
exception. When the file handle is on a loopback mount, this is a real
problem because Morph then cannot unmount the image in its exception
handler. In most cases 'with' is the best option.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
cmake has a ./configure, which accepts --prefix, but chokes on
--sysconfdir
This means that bootstrap is broken by this commit.
This reverts commit 5f22ded9711a047704fc91b68a182b057e29bc0f, reversing
changes made to e13753d4f3e26f3a4c705fb3e694ae8dea860c2d.
|
|\
| |
| |
| | |
git://git.baserock.org/baserock/morph
|
| |
| |
| |
| |
| | |
Currently the message is still displayed "Updating xxx" but no
update is actually done.
|
| |
| |
| |
| |
| | |
This is for users who prefer the old behaviour of building from the
remote repos.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This means that Morph no longer requires changes to be pushed in order
to build them.
The repos from the system branch are currently cached in the local
repo cache as part of the build process, which is far from ideal.
Tests for 'morph build' now test build without push. The build
metadata now includes a repo path that is inside the TMPDIR, so the
tests have been rewritten to avoid having any hardcoded cache keys
because the cache keys are no longer static.
|
| | |
|
|\ \
| | |
| | |
| | | |
git://git.baserock.org/baserock/morph
|
| |/ |
|
| |
| |
| |
| |
| |
| | |
Some autotools projects default their sysconfdir to $prefix/etc.
This leads to some projects trying to install files to /usr/etc, which
is not desired.
|
|\ \
| |/
|/|
| | |
Unused imports in morphlib/plugins/expand_repo_plugin.py were also removed
|
|/
|
|
|
| |
This is helpful for when the user is not sure what an aliases repo URL
actually expands to.
|
|
|
|
|
|
|
|
|
|
|
| |
The test tests.merging/rename-stratum could potentially trigger two
different errors in Morph, based on the order that the systems in the
root repo were processed.
This meant that the test would sometimes spuriously fail if TMPDIR
was manually set, because of differences in the way file systems work.
To fix the root cause requires proper 3-way merging, really.
|