| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
| |
The conversion was made so that the build command was used instead of
the build-morphology command.
Change-Id: I67c43d765ed603ecdd806bc649815526243b7b87
|
| |
|
|
|
|
| |
The build-morphology command will be removed in a future commit.
Change-Id: I1048062e5d8556622a71276c4b8fae2d3283af19
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a new 'Defaults' class to represent definitions defaults
The Python 'jsonschema' module is used to validate the contents of the
Defaults file. This module is already included in Baserock 'build' and
'devel' reference systems by way of the 'openstack-common' stratum.
This commit embeds a copy of the JSON-Schema schema for the DEFAULTS
file. I think the canonical location of this schema should be in the
reference definitions.git, for now. In future, the schemas should maybe
have their own repos. Either way, Morph should embed a copy for the time
being so that we are sure the schema matches how Morph expects to parse
the file.
Morph's automated tests are all updated to use definitions version 7.
I removed most of the tests for built-in build systems, because the
built-ins themselves are no longer part of Morph. Only the mechanism for
defining them needs to be tested now.
Change-Id: I65f8f1c967683ef605852bfae5c68518e53f9981
|
| |
|
|
| |
Change-Id: Iad95af65bd5c528d2e72f5b2ffa80a01152f50ff
|
| |
|
|
|
|
|
|
|
| |
This change, suggested in story #16, causes the testing suite to use
the user-defined tempdir in /etc/morph.conf if it exists or continue
using /tmp if not. This will prevent the test suite from filling drives
that are not intended to be used for development work.
Change-Id: I94edabe4ea1eca7a55ece7424ece7bbd4b68c513
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Improve error messages and remove redundant information
* Stop displaying the containerised settings, as they are not very
helpful to debug a build failure
* Display the error code
* Change system integration commands logging to consistent with build
commands logging.
* Provide a log file in the staging directory for each chunk build
failure or system integration failure.
* Provide a chroot script in the staging directory for each chunk build
failure or system integration build failure. This script allows the
user to enter an environment similiar to the one where the build
failed.
* Fix a bug where the system integration commands stdout was being
omitted when these commands failed.
* Fix a bug where the error_message_for_containerised_commandline() was
being called with an empty error message, as the stderr was being
directed to stdout.
Chunk build failure output before the this change and change
I16ebe9ba4ac46fef82e37d0b3e05f42d14249de8:
2015-08-03 15:55:21 [Build 258/267] [test] Running build-commands
build failed
# configure
# # echo "configure-commands stdout"
configure-commands stdout
# # echo "configure-commands stderr" >&2
configure-commands stderr
# build
# # echo "build commands stdout"
build commands stdout
# # echo "build commands stderr" >&2
build commands stderr
# # echo "some more sdout"
echo "some more stderr" >&2
echo $PATH
foo
some more sdout
some more stderr
/tools/bin:/usr/bin:/usr/lib/ccache:/sbin:/usr/sbin:/bin:/usr/bin
sh: foo: not found
ERROR: In staging area /src/tmp/failed/tmpglc5Sz: Command failed: sh -c
'echo "some more sdout"
echo "some more stderr" >&2
echo $PATH
foo
':
Containerisation settings: {'binds': (('/src/cache/ccache/nano-tarball',
'/src/tmp/staging/tmpglc5Sz/tmp/ccache'),), 'mount_proc': True,
'mounts': (('dev/shm', 'tmpfs', 'none'),), 'writable_paths':
['/src/tmp/staging/tmpglc5Sz/test.build',
'/src/tmp/staging/tmpglc5Sz/test.inst',
'/src/tmp/staging/tmpglc5Sz/dev', '/src/tmp/staging/tmpglc5Sz/proc',
'/src/tmp/staging/tmpglc5Sz/tmp'], 'root': '/src/tmp/staging/tmpglc5Sz',
'cwd': '/test.build'}
Error output:
$>
Chunk build failure output after this change and change
I16ebe9ba4ac46fef82e37d0b3e05f42d14249de8:
2015-08-03 15:53:59 [Build 258/267] [test] Running build-commands
### CONFIGURE-COMMANDS ###
+ echo configure-commands stdout
configure-commands stdout
+ echo configure-commands stderr
configure-commands stderr
### BUILD-COMMANDS ###
+ echo build commands stdout
build commands stdout
+ echo build commands stderr
build commands stderr
+ echo some more sdout
some more sdout
+ echo some more stderr
some more stderr
+ echo /tools/bin:/usr/bin:/usr/lib/ccache:/sbin:/usr/sbin:/bin:/usr/bin
/tools/bin:/usr/bin:/usr/lib/ccache:/sbin:/usr/sbin:/bin:/usr/bin
+ foo
sh: foo: not found
ERROR: In staging area /src/tmp/staging/tmpjES5_2: build failed (exi
t_code=127)
$>
System integration build failure output before this change and change
I16ebe9ba4ac46fef82e37d0b3e05f42d14249de8:
2015-08-03 15:58:25 [Build 267/267] [build-system-x86_64] Running the
system integration commands
2015-08-03 15:58:25 [Build 267/267] [build-system-x86_64] Error while
running system integration commands
2015-08-03 15:58:25 [Build 267/267] [build-system-x86_64] Error while
building system
ERROR: Command failed:
baserock/system-integration/00-test-test-misc-0002:
Containerisation settings: {'mounts': (('dev/shm', 'tmpfs', 'none'),
('tmp', 'tmpfs', 'none')), 'mount_proc': True, 'root':
'/src/tmp/staging/tmpvpwB4l/build-system-x86_64.inst'}
Error output:
+ foo
baserock/system-integration/00-test-test-misc-0002: line 1: foo: not
found
$>
System integration build failure output after this change and change
I16ebe9ba4ac46fef82e37d0b3e05f42d14249de8:
2015-08-03 16:00:06 [Build 267/267] [build-system-x86_64] Running the
system integration commands
+ echo stdout
stdout
+ echo stderr
stderr
+ foo
baserock/system-integration/00-test-test-misc-0002: line 1: foo: not
found
ERROR: In staging area /src/tmp/staging/tmpl9VNzf: system integration
commands failed (exit_code=127)
$>
Change-Id: Id992f707f69f3fa761b4c21e9904c4e5328e1c77
|
| |
|
|
|
|
|
|
|
|
|
| |
Instead of logging every command for ourselves, use the shell X-ray
mode. As side effect, each command argument will now be printed out in
its evaluated form, which aids debugging.
Also, improve the visual separation between the different build steps,
and display those steps with finer granularity.
Change-Id: I16ebe9ba4ac46fef82e37d0b3e05f42d14249de8
|
| |
|
|
|
|
|
|
|
|
|
| |
The previous release of the Baserock reference systems (baserock-15.25
tag) used Baserock definitions version 3, so this version of Morph can
still build the last release of the Baserock reference systems.
In the course of fixing up the cmdtest tests to use version 5
definitions, I removed a couple of bits of redundant code.
Change-Id: If9d480aa966d2ea5939cf04351f3d488cccea67f
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Old error:
ERROR: Git directory
/build/cache/gits/git___git_baserock_org_baserock_baserock_definitions
has no commit at ref 5046c3a0e4fd587c23f7c6a9e0d0b50d1914dcca^{commit}.
New error:
ERROR: Commit 5046c3a0e4fd587c23f7c6a9e0d0b50d1914dcca wasn't found
in the "origin" remote
git://git.baserock.org/baserock/baserock/definitions. You either need
to push your local commits on branch xxx to "origin", or use the
--local-changes=include feature.
Change-Id: I0c3658e9cd27c23f40653662ba7e4ba58b7892de
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit allows you to build only a single chunk or stratum and its
dependencies, instead of a whole system. This will be useful for
developers who want to quickly check the build commands of a chunk for
example. You can give more than one chunk or stratum to the command,
and the build will run up to the latest one, building the dependencies
of both.
An example of the usage:
morph build systems/devel-system-x86_64-generic.morph \
strata/build-essential.morph strata/core/curl.morph
This will only build up to curl in the core stratum.
Change-Id: Id55b8335fc61e8fdf847f5e383cfedd13d07e6d2
|
| |
|
|
| |
Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
|
| |
|
|
|
|
|
| |
We will use MorphologyLoader.load_from_file() to load morphologies
from a temporary file, so the output will be unpredictable. That
will break this test, so we may as well convert it into a yarn
to fix it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This consolidates a bunch of code paths that were previously duplicated.
This also changes the API for local cached repos to match the
function names GitDirectory uses. Note that the remote repo cache still
uses the old names, and should be fixed when time permits.
Some unit tests that use the CachedRepo module required a bit of
inelegant monkey-patching in order that they continue to work. A better
way to do this would be with the 'mock' library (which would need to be
added to Baserock 'build' and 'devel' systems before we could use it).
|
| |
|
|
|
|
|
| |
I've rarely needed to use it, and on those rare occasions, it would have
been easy enough to calculate it.
Let's get rid of this step, and save everyone some time in future.
|
| | |
|
| | |
|
| |
|
|
|
| |
This is tested in yarns already, and this test breaks the test suite on
any architecture but x86_64.
|
| |
|
|
|
| |
uname tends to only give us a valid morph architecture on x86_64,
this makes it work on other architectures.
|
| |
|
|
|
|
|
|
| |
This messes up the baserock-system-config-sync tool. Systemd does not
require /etc/fstab to exist in any case.
I have bumped the 'system-compatibility-version' field in this commit
to trigger rebuilding all system artifacts.
|
| |
|
|
|
| |
Refs should be completely omitted, and this is now the standard
behaviour, so there's little value in testing the behaviour separately.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If two systems with the same name (e.g. different repo/ref) depend on the
same strata, then it will collide with systems which depend on different
artifacts from that stratum, but the same number of artifacts.
For example, if you checkout an existing branch and change the artifacts
used by one of its strata, then your local changes won't be built.
This is because the 'kids' field lists artifacts it depends on by their
cache-key, which is now no longer sufficient to uniquely identify
artifacts. The same number of artifacts issue is from it listing cache
keys multiple times.
The fix for this is to include the artifact name, so the 'kids' field is
now a list of dicts, with artifact name and cache key.
This is a dict rather than a tuple so that the generated /baserock
metadata is more readable.
|
| | |
|
| | |
|
| |\
| |
| |
| | |
Changed the error (exception) to list all obsolete fields.
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cross-bootstrap is a way to build baserock on an architecture that
does not currently have Baserock. It can be used by `morph
cross-bootstrap <ARCH> <REPO> <REF> <MORPH>`, and will build an artifact
that can be used as a root filesystem with a basic build environment
with a script named `native-bootstrap` which will build and install
every chunk in the system.
If done with a devel system, this will give you a suitable environment
for building a proper Baserock system.
This does not currently provide a kernel for the target architecture.
Apart from adding the cross-bootstrap plugin, it also makes the
following changes:
* Moves the lit of valid_archs into morphlib (instead of locally-scoped
in MorphologyFactory)
* BuildCommand takes an extra argument, build_env
* split BuildCommand's get_artifact_object into create_source_pool and
resolve_artifacts (plus changes things that use get_artifact_object to
use the new way)
* setup_mounts finds out whether to do so by whether build_mode is
'staging', instead of by whether the setting 'staging-chroot' is true.
* Makes ChunkBuilder's get_sources use the
morphlib.builder2.extract_sources() method, and moved
set_mtime_recursively into morphlib.builder2, since it's not currently
used anywhere else.
* moved ChunkBuilder's get_commands into the Morphology class (plus
changes to anything that used get_commands)
|
| | |
|
| |
|
|
| |
Add tests for cmake and qmake build systems
|
| |
|
|
|
| |
Removed trailing comma, corrected copyright year, used #pragma: no cover
to exclude the exception class form coverage and allow ./check full to run.
|
| |
|
|
|
| |
Added a new test case that tests for new error from previous commit that
changed the warning to an error when building an empty stratum.
|
| |
|
|
|
|
| |
The absolute one made the test suite fail in my VM, and is not the
right thing to do anyway. $DESTDIR/usr/bin and $DESTDIR/bin are meant
to be the same directory here, not $DESTDIR/usr/bin and /bin.
|
| |
|
|
|
| |
We can no longer build strata individually because we don't know
what architecture they are for.
|
| |
|
|
|
|
|
|
|
|
|
| |
When building a stratum artifact it's easy to only include chunks that
were built in 'staging' mode. Constructing the staging area doesn't use
that code path, though, so we need an extra hack to filter out those
artifacts while building.
It may be neater to express stratum build-depends as actual
dependencies on the stratum artifact, rather than each of its
constituent chunks as we do now.
|
| |
|
|
|
| |
This is necessary so that the build-essential stratum doesn't end up
containing three different compilers.
|
| |
|
|
|
|
|
|
|
| |
Morph no longer supports setting the prefix using the --prefix
argument / setting. This was only used in tests and during bootstrap.
If a chunk build-depends on a chunk within a stratum which has
a custom prefix, that prefix is appended to the PATH in the build
environment.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Allowed values:
staging: build with a staging chroot (default)
test: build with the host's tools
bootstrap: build with the host's tools, and do not include this
chunk in the final stratum artifact
In the past, 'normal mode' has been used to describe building a chunk
with the host's tools. We don't want that mode to ever be used,
because it is a huge hole in reproducability, but we need to keep it
around to avoid making Morph's cmdtest suite depend on Baserock.
Hopefully naming it 'test' should discourage potential abusers.
It is unfortunate that the build tests now take a separate code path
compared to real-world usage of Morph. However, this is necessary to
avoid a circular dependency between Morph's test suite and the
build-essential stratum in Baserock.
We do whole-build testing of Baserock, too, so the 'staging' code path
is still tested outside of Morph. However, testing a staging area
requires populating it with at minimum a working shell, and this is a
bit too complex to go in Morph's test suite.
|
| |
|
|
| |
This is mainly to fix bootstrap.
|
| |
|
|
|
|
|
|
| |
Building invididual chunks is bad because we have no idea what
build mode to use. Building individual strata would work, but all of
the stratum's build dependencies would need to be built first so there
would be little time saved in any case. There is also no way to test
them beyond checking that the build was successful.
|
| |
|
|
|
| |
We can't rely on this being possible any more. It was mostly only used
for testing in any case.
|
| |
|
