| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
These each produce a minimal (~300MB) sysroot containing BusyBox, the
GNU C/C++ toolchain, and a couple of other components necessary for
bootstrapping Baserock reference systems from the ground up.
Morph and YBD used tools from the host to bootstrap, which usually
worked fine but was occasionally disasterous (such as when GLIBC
broke ABI between releases). BuildStream is more strict and requires
you to provide binaries to seed its sandbox.
The stage2 sysroot can only be used to build the stage3 sysroot, as
the stage2 components are configured with a non-standard /tools prefix
and the stage3 build instructions have some special casing that is
necessary to work with that.
The stage3 sysroot can be used to build pretty much anything and is
used to seed Baserock reference builds on each platform.
|
| |
|
|
|
|
|
| |
libseccomp has architecture-specific parts, and the previous version
did not support ppc64l.
|
|
|
|
|
|
| |
This is done because ppc64l is having compatibility problems.
The updated version of libffi depends on libtool.
|
|
|
|
|
|
|
|
|
|
|
|
| |
I got a seemingly random failure in CI:
https://gitlab.com/baserock/definitions/-/jobs/31214873
The log line with the actual error is cut off by BuildStream and I
can't reproduce the issue locally, but since it's clearly an
intermittent problem I can only put it down to the way BuildStream
sets MAKEFLAGS during the configure stage affecting the gnulib bootstrap
process (which does run `make` at one point).
|
| |
|
|
|
|
|
|
| |
It's better to use BuildStream arch conditionals than to use shell code,
because in the latter case any change will trigger rebuilds for all
architectures regardless of which ones it actually affects.
|
|
|
|
|
|
|
|
|
|
|
| |
We now have /tools at the *end* of the PATH rather than the start (which is
how it should have been all a long in stage 3) so glibc's configure
script will always find `bash` in /usr/bin before it looks in
/tools/bin.
This also fixes stage3 glibc building against a stage2 sysroot when there's
no symlink from /tools/bin -> /usr/bin. This is required for the current
cross-bootstrap method.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
We install the ld.so config to /etc/ld.so.conf in the stage2-glibc element,
but the ldconfig tool would look for /tools/etc/ld.so.conf. This caused no
problems for a long time, I suppose because we always built on top of
sysroots that had an existing /etc/ld.so.cache file. But if `ldconfig`
hadn't already been run in the sysroot, builds depending on stage2-glibc
would fail as various things would no longer be able to find libz.so due
the ldconfig command failing to find its config file.
|
|
|
|
|
|
|
|
|
|
|
| |
This is currently because fhs-dirs makes /lib a symlink to /usr/lib,
and buildstream mangles symlink paths from absolute paths to relative
paths.
The symlink ../tools/lib/libgcc_s.so ends up in /usr/lib, breaking
things.
This fixes that by explicitly installing the symlink to /usr/lib
|
|
|
|
| |
Needed to fix build on armv8
|
|
|
|
| |
Needed to fix build on ppc64.
|
|
|
|
| |
This means we no longer try to build syslinux on non-x86 platforms.
|
|
|
|
|
|
|
|
| |
The default behaviour of BuildStream is now for lists to overwrite
the previous value when composing them. In the fhs-dirs elements our
goal is extend the install commands in certain cases, so we now need
to use the (>) operator to cause it to append to the list instead of
overwriting.
|
|
|
|
|
|
|
|
|
|
|
|
| |
YBD scans the current working directory for definition files, which is
problematic now that we store hundreds of cached Git repos and OSTree
repos in a subdir of the definitions repo.
To avoid wasting loads of time calling stat() on tens of thousands of
directories, the cache is now in a hidden directory.
This depends on a change to YBD that causes it to efficiently ignore
hidden directories: https://gitlab.com/baserock/ybd/commit/6676c4ac0
|
| |
|
|
|
|
| |
This is hopefully what has been preventing BuildStream from pushing.
|
| |
|
| |
|
| |
|
|
|
|
|
| |
Removing since the docker image contains an installation of BuildStream
and is updated nightly (as pointed out by Sam).
|
|
|
|
|
| |
By default GitLab CI fetches and re-pushes cache, given deploy stages
won't cause the git repos to change, only fetch.
|
|
|
|
|
| |
By default, GitLab CI fetches the artifacts of all previous stages.
Be explicit about which artifacts we do and don't want.
|
| |
|
|
|
|
|
| |
Should we pick up the same runner in another job, artifacts will be
reused from this path.
|
|
|
|
| |
Allows us to only perform the tasks required for each tool.
|
| |
|
|
|
|
|
|
|
|
|
| |
With the previous attempt, storing files in the cache worked OK but
restoring from the cache did not. The symlink workaround wasn't enough
as the symlink gets created /after/ the cache is restored.
Now the whole /builds tree is on a separate Docker volume, so we should
be able to really store the cache inside the build directory.
|
|
|
|
|
|
|
|
|
|
| |
It turns out that downloaded sources have never been shared between
builds. Each container running a build has a `/cache` volume, but
a new volume is created for each container so nothing is shared.
We listed certain paths inside `/cache` to be saved by GitLab CI
between runs using the "distributed runners cache" feature, but
because the paths were outside the build directory it would just ignore
them.
|
|
|
|
|
| |
These are still needed, but are set correctly in the Docker image now so
we don't need to repeat them here.
|
|
|
|
| |
This should give a massive performance improvement.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is possible since this change: https://gitlab.com/BuildStream/buildstream/commit/00e08459c4
Older versions of BuildStream will fail to parse the project.conf file
with this change; I think it's acceptable to force everyone to update
as we are still in the development stage of the BuildStream migration.
If you want to override the configuration for whatever reason you'll
now need to put this in your ~/.config/buildstream.conf file:
projects:
baserock:
artifacts:
pull-url: ...
push-url: ...
If you set a "global" artifacts configuration by doing this...
artifacts:
pull-url: ...
... it'll be ignored because the project-specific config from this
project.conf file will take precedence.
Overriding seems to be done for the whole config block, so if you
only set push-url in your buildstream.conf file it will be interpreted
to mean "set push-url, and unset pull-url" rather than "set push-url
and use the existing value of pull-url."
|
|
|
|
| |
Change-Id: Ia72f4cc835fea6ecc72ab0704f877905f104bc40
|
|
|
|
| |
Change-Id: I767fc41d3336bb7a0fd14d74b7a7ee082ca03193
|
|
|
|
| |
Change-Id: Ifd32ee4809552bf52986431d48ed0e200d7f239d
|
|
|
|
|
|
|
|
|
|
|
| |
In order to bypass:
Cloning repository...
Cloning into '/builds/gitlab/omnibus-gitlab'...
error: RPC failed; curl 56 SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
|
|
|
|
|
| |
We don't want to error out here, otherwise builds will only pass on
protected branches.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Builds with BuildStream are intermittently failing with a message like
this:
libtool: install: /usr/bin/install -c .libs/libsvn_delta-1.a /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: install: chmod 644 /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: install: ranlib /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: install: /usr/bin/install -c .libs/libsvn_delta-1.lai /buildstream/install/usr/lib/libsvn_delta-1.la
/usr/bin/ld: cannot find -lsvn_delta-1
collect2: error: ld returned 1 exit status
libtool: install: /usr/bin/install -c .libs/libsvn_delta-1.a /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: install: chmod 644 /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: install: ranlib /buildstream/install/usr/lib/libsvn_delta-1.a
libtool: error: error: relink 'libsvn_ra_serf-1.la' with the above command before installing it
build-outputs.mk:1321: recipe for target 'install-serf-lib' failed
make: *** [install-serf-lib] Error 1
make: *** Waiting for unfinished jobs....
|
|
|
|
|
|
|
|
|
|
|
|
| |
The stage2 elements were all using the default strip-commands which
don't take into account the fact that we might be cross-compiling.
An `objcopy` build for one architecture will ignore binaries for
other architectures that it doesn't understand, so in practice no
stripping was taking place for the stage2 components when we were
doing cross-builds.
With this change, a stage2 sysroot containing just the 'runtime' and
'devel' domains has gone from 889MB to 306MB.
|
|
|
|
|
|
| |
The "downloadable" state is the one we need to special-case. The
"cached" state means that the artifact is cached *locally*, and
BuildStream will already avoid rebuilding it in this case.
|
|
|
|
|
|
|
|
|
|
|
| |
We shouldn't download artifacts to the CI workers every time somebody
pushes just to throw them away again. This should speed up no-op builds.
The functionality is implemented in two shell scripts. Context is here:
https://gitlab.com/BuildStream/buildstream/issues/77
It would be possible to do this with a single script, but I wanted to
avoid doing any argument parsing code in shell.
|
|
|
|
|
|
| |
This is a Fedora 26 image which has BuildStream's dependencies
pre-installed, which saves us waiting to `dnf install` everything
at the start of every CI job.
|
| |
|
|
|
|
|
|
|
| |
There hasn't been a release since v2.2.52, but there are fixes in
'master' which are useful. In particular the build system is now
standard Autotools, and it no longer breaks if /lib64 is a symlink
to /usr/lib64 (upstream commit cd76644ce9b9814a fixes that).
|
|
|
|
|
|
|
|
|
|
|
| |
This is required at least for armv8l64, otherwise the glibc.bst
element installs a symlink in /usr/lib/ld-linux-aarch64.so.2 that
points to a missing file (it expects /usr/lib64/ld-linux-aarch64.so.2
to exist, but if /lib64 is a directory rather than a symlink then
that file ends up only in the /lib64/ directory).
This also makes our filesystem hierarchy more consistent with other
GNU/Linux operating systems.
|
| |
|
| |
|
|
|
|
|
| |
We gained this file when we imported Tristan's gnu-toolchain project, we just
need to update it for building the Baserock reference systems.
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
sam/buildstream-autoconvert
This branch contains a manual conversion of the Baserock bootstrap
process to BuildStream. The original branch can be found here:
https://gitlab.com/BuildStream/buildstream-tests/tree/gnu-toolchain
It's not possible to automatically convert the existing Baserock
bootstrap due to differences in how BuildStream works.
|