diff options
| -rw-r--r-- | README | 91 | ||||
| -rwxr-xr-x | baserock-bootstrap | 537 | ||||
| -rwxr-xr-x | check | 2 | ||||
| -rwxr-xr-x | dump-build-times | 75 | ||||
| -rwxr-xr-x | run-bootstrap-in-chroot | 199 | ||||
| -rwxr-xr-x | scripts/assemble-stratum | 59 | ||||
| -rwxr-xr-x | scripts/bootstrap | 555 |
7 files changed, 0 insertions, 1518 deletions
@@ -254,9 +254,6 @@ more consistently across machines and developers. See the `morphlib/builder.py` file, method `setup_env` for details. -Until Baserock is properly bootstrapped, it's awkward that the `PATH` -variable gets reset. To avoid that, use the `--keep-path` option. - Hacking morph ------------- @@ -271,94 +268,6 @@ and check out the `cmdtest` utility (from <http://liw.fi/cmdtest/>). Run the checks before submitting a patch, please. -The Bootstrap Process ---------------------- - -The goal of the bootstrap process is to create an environment that has -only been built with morph. It is a long boring process that can take -a few hours. - -It can be started by running `./run-bootstrap-in-chroot $workingdir` -with some environment variables set. - -The variables are: -* `DEBIAN_MIRROR`; which must be a path suitable for passing to - debootstrap; which at the time of writing allowed http://, - file:// or ssh:// urls; **required** -* `GIT_TARBALLS`; which must be a url to a tarballs directory; - morph will try to fetch tarballs from here before trying the - git-base-url; **required** -* `snapshot`; which should be `false` (or any executable that - exits unsuccessfully) if creating a snapshot after each stage - of the bootstrap is not desirable; optional - -If snapshot is true or omitted then after each stage of the bootstrap -a tarball of the working directory is created, so that if a later stage -fails it can start from the last successful stage. - -The mirror variables allow the bootstrap to be run more quickly if the -required files are available locally. - -The mirror variables being required and snapshotting defaulting -to true are artefacts of the bootstrap process' development, where -only re-running the step that failed and using local mirrors were -essential for making fixes. - - -Bootstrap stages ----------------- - -The stages required for the bootstrap are to create a debian squeeze -chroot; build a skeleton system using the linux from scratch specified -tarballs; chroot into this system and build a system using morph with -bootstrap mode; then build the system again but instead using morph's ---staging-chroot mode and with the linux from scratch system removed. - -The squeeze chroot (stage 0) is used so that the bootstrap process does -not need to be supported on every available host system, but every host -system that can run debootstrap can be indirectly supported. - -In this stage debootstrap creates a chroot with all the packages required -to build the linux from scratch system, from then onwards every stage is -built by running baserock-bootstrap and optionally making a snapshot. - -Stage 1 builds everything in a subdirectory of the squeeze-chroot called -tree, and all the packages are built with --prefix=/tools, so the full -path for `cat` would be `$workingdir/squeeze-chroot/tree/tools/bin/cat`. - -This is done so that we have a known good system to work from, that won't -become entangled with the final system, which installed with --prefix=/usr. - -Stage 2 uses morph to build a system, while chrooted into -`$workingdir/squeeze-chroot/tree`, this naturally required PATH to include -/tools/bin. - -Morph is used to build the `foundation.morph` and `devel-bootstrap.morph` -strata in the `morphs` repository. `devel-bootstrap.morph` may not use the -same morphs as `devel.morph` because the bootstrap requires some hacks that -aren't required when a fully bootstrapped system is available. - -Morph also uses the `--bootstrap` flag to make the chunks be unpacked after -they have been built, this is required for build-dependencies to be met as -prerequisites are expected to be available on the system. - -After morph has finished a reasonably complete system should be available, -so `$workingdir/squeeze-chroot/tree/tools` is removed to save space and -make it obvious if stage 2 is missing anything needed for stage 3. - -Stage 3 builds `devel.morph` which does not need all the hacks required -in building during the bootstrap, so it should be much more like what -morph will finally build. - -Rather than using `--bootstrap`, stage 3 uses `--staging-chroot`, which -builds and extracts into a chroot, rather than using anything on the host -system. - -For this to be able to build anything `--staging-filler` is used, which -allows for specifying the environment that chunks are built in. Stage 3 -uses the strata that were build in stage 2. - - Legalese -------- diff --git a/baserock-bootstrap b/baserock-bootstrap deleted file mode 100755 index 619baeb4..00000000 --- a/baserock-bootstrap +++ /dev/null @@ -1,537 +0,0 @@ -#!/bin/bash - -set -e -set +h -set -u - -BASEDIR=$(dirname $(readlink -f $0)) - -LFS="$(pwd)/tree" -LFS="${LFS/\/\///}" -tools="$LFS/tools" - -buildwhat="$1" - -export LC_ALL=C - -case "$(uname -m)" in -*armv7*) - export LFS_TGT=$(uname -m)-lfs-linux-gnueabi - export TARGET_CFLAGS="-march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -O2" - export PASS3_STRATUM="bootstrap-pass3-imx53" - ;; -*armv5*) - export LFS_TGT=$(uname -m)-lfs-linux-gnueabi - export TARGET_CFLAGS="-march=armv5 -O2" - export PASS3_STRATUM="bootstrap-pass3-imx53" - ;; -*) - export LFS_TGT=$(uname -m)-lfs-linux-gnu - export TARGET_CFLAGS="-O2" - export PASS3_STRATUM="bootstrap-pass3-x86_64-generic" - ;; -esac - -HOST_CAT=`which cat` -HOST_CP=`which cp` -HOST_MKDIR=`which mkdir` -HOST_CHMOD=`which chmod` - -export CCACHE_DIR="/var/tmp/ccache" -export PATH="$tools/lib/ccache:$tools/bin:$tools/sbin:/usr/lib/ccache:/usr/bin:/bin:/usr/sbin:/sbin" - -pass1_directories() -{ - $HOST_MKDIR -p "$LFS" - - $HOST_MKDIR -p "$tools" - $HOST_MKDIR -p "$tools/bin" - $HOST_MKDIR -p "$tools/lib" - [ -h "$tools/sbin" ] || ln -sf "bin" "$tools/sbin" - [ -h "$tools/lib64" ] || ln -sf "lib" "$tools/lib64" - [ -h "$tools/libexec" ] || ln -sf "lib" "$tools/libexec" - - [ -e "$LFS/proc" ] || $HOST_MKDIR -p "$LFS/proc" - [ -e "$LFS/sys" ] || $HOST_MKDIR -p "$LFS/sys" - [ -e "$LFS/tmp" ] || $HOST_MKDIR -p "$LFS/tmp" - [ -e "$LFS/dev" ] || $HOST_MKDIR -p "$LFS/dev" - [ -e "$LFS/dev/console" ] || \ - mknod -m 600 "$LFS/dev/console" c 5 1 - [ -e "$LFS/dev/null" ] || mknod -m 666 "$LFS/dev/null" c 1 3 - [ -e "$LFS/dev/random" ] || mknod -m 644 "$LFS/dev/random" c 1 8 - [ -e "$LFS/dev/urandom" ] || \ - mknod -m 644 "$LFS/dev/urandom" c 1 9 -} - -pass1_get_sources_with_morph() -{ - cd $LFS/baserock/gits/morph - $HOST_MKDIR -p $LFS/baserock/cache - python ./morph --verbose update-gits \ - baserock:baserock/morphs master bootstrap-pass1 \ - --cachedir=$LFS/baserock/cache \ - --log=$LFS/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --tarball-server="$GIT_TARBALLS" -} - -pass1_build_with_morph() -{ - cd $LFS/baserock/gits/morph - $HOST_MKDIR -p $LFS/baserock/cache - $HOST_MKDIR -p $LFS/tmp - python ./morph --verbose build-morphology \ - baserock:baserock/morphs master bootstrap-pass1 \ - --no-git-update \ - --no-distcc \ - --no-ccache \ - --bootstrap \ - --cachedir=$LFS/baserock/cache \ - --tempdir=$LFS/tmp \ - --log=$LFS/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --prefix="$tools" \ - --toolchain-target="$LFS_TGT" \ - --target-cflags="$TARGET_CFLAGS" -} - -pass1_dump_build_times() -{ - echo "Dumping pass 1 build times" - cd $LFS/baserock/gits/morph - python ./dump-build-times $LFS/baserock/cache/artifacts -} - -pass2_get_morph() -{ - echo "Get morph" - if [ ! -e "$LFS/baserock/gits" ] - then - $HOST_MKDIR -p "$LFS/baserock/gits" - cp -al "$HOME/baserock/gits/morph" "$LFS/baserock/gits/morph" - fi -} - - -pass2_prepare_for_chroot() -{ - echo "Preparing $LFS for chroot" - cd "$LFS" - if [ ! -h "$LFS/$LFS" ] - then - $HOST_MKDIR -p "$LFS/$LFS" - /bin/rmdir "$LFS/$LFS" - /bin/ln -s / "$LFS/$LFS" - fi - - if [ ! -d "$LFS/etc" ] - then - mkdir -p "$LFS/etc" - chmod 777 "$LFS/etc" - touch "$LFS/etc/ld.so.conf" - chmod 666 "$LFS/etc/ld.so.conf" - cat <<EOF > "$LFS/etc/ld.so.conf" -/lib64 -/lib -/usr/lib64 -/usr/lib -EOF - fi - -# $HOST_MKDIR -p "$LFS/etc" - [ -e "$LFS/etc/passwd" ] || \ - echo 'root::0:0:root:/root:/bin/bash' > "$LFS/etc/passwd" - [ -e "$LFS/etc/group" ] || echo 'root::0:' > "$LFS/etc/group" - -# [ -e "$LFS/etc/hostname" ] || echo 'baserock-boot' | -# /usr/bin/tee "$LFS/etc/hostname" > /dev/null - - # Add symlinks for common locations of specific tools - # These are needed for #! lines in scripts - [ -e "$LFS/bin" ] || $HOST_MKDIR -p "$LFS/bin" - [ -e "$LFS/bin/sh" ] || ln -sf ../tools/bin/bash "$LFS/bin/sh" - [ -e "$LFS/bin/bash" ] || \ - ln -sf ../tools/bin/bash "$LFS/bin/bash" - [ -e "$LFS/bin/pwd" ] || ln -sf ../tools/bin/pwd "$LFS/bin/pwd" - [ -e "$LFS/bin/echo" ] || \ - ln -sf ../tools/bin/echo "$LFS/bin/echo" - if [ ! -e "$LFS/usr/bin/perl" ]; then - $HOST_MKDIR -p $LFS/usr/bin - ln -sf ../../tools/bin/perl "$LFS/usr/bin/perl" - fi -} - - -pass2_get_sources_with_morph_in_chroot() -{ - echo "Getting sources with morph" - cat <<EOF > "$LFS/baserock/build.sh" -#!/tools/bin/bash -set -e -set -x - -cd /baserock/gits/morph -mkdir -p /baserock/cache -export PATH="/usr/bin:/bin:$tools/bin:$tools/sbin" -python ./morph --verbose update-gits \ - baserock:baserock/morphs master bootstrap-pass2 \ - --bootstrap \ - --cachedir=/baserock/cache \ - --log=/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --tarball-server="$GIT_TARBALLS" -EOF - $HOST_CHMOD +x "$LFS/baserock/build.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT <<EOF >"$do_chroot" -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /tools/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin" \\ - BOOTSTRAP_TOOLS="$LFS/tools" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/build.sh -} - - -pass2_build_with_morph_in_chroot() -{ - echo "Building Baserock with morph" - cat <<EOF > "$LFS/baserock/build.sh" -#!/tools/bin/bash -set -e -set -x - -cd /baserock/gits/morph -mkdir -p /baserock/cache -export PATH="/usr/bin:/bin:$tools/bin:$tools/sbin" -export LD_LIBRARY_PATH="/usr/lib:/lib:/lib64:/tools/lib:/tools/lib64" -python ./morph --verbose build-morphology \ - baserock:baserock/morphs master bootstrap-pass2 \ - --bootstrap \ - --no-git-update \ - --cachedir=/baserock/cache \ - --log=/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --tarball-server="$GIT_TARBALLS" \ - --target-cflags="$TARGET_CFLAGS" -EOF - $HOST_CHMOD +x "$LFS/baserock/build.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT <<EOF >"$do_chroot" -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /tools/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin" \\ - BOOTSTRAP_TOOLS="$LFS/tools" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/build.sh -} - -pass2_dump_build_times() -{ - echo "Dumping pass 2 build times" - cat <<EOF > "$LFS/baserock/dump-pass2-build-times.sh" -#!/tools/bin/bash -set -e -set -x - -cd /baserock/gits/morph -python ./dump-build-times /baserock/cache/artifacts -EOF - $HOST_CHMOD +x "$LFS/baserock/dump-pass2-build-times.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT <<EOF >"$do_chroot" -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /tools/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin" \\ - BOOTSTRAP_TOOLS="$LFS/tools" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/dump-pass2-build-times.sh -} - -pass2a_cleanup_at_end() -{ - echo "Remove unnecessary stuff at the end of pass2a" - rm -f "$LFS/baserock/cache/gits/"*.bndl -} - - -pass2b_cleanup_at_end() -{ - echo "Remove unnecessary stuff at the end of pass2b" - rm -rf "$LFS/tools" -} - - -pass3_remove_tools() -{ - echo "Removing $LFS/tools" - rm -rf "$LFS/tools" -} - -pass3_get_sources_with_morph_in_chroot() -{ - echo "Getting sources with morph" - - cd "$LFS/baserock/gits/morph" - cat <<EOF > "$LFS/baserock/build.sh" -#!/bin/bash -set -e -set -x - -cd /baserock/gits/morph -python ./morph --verbose update-gits \ - baserock:baserock/morphs master $PASS3_STRATUM \ - --cachedir=/baserock/cache \ - --log=/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --tarball-server="$GIT_TARBALLS" -EOF - $HOST_CHMOD +x "$LFS/baserock/build.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT <<EOF >"$do_chroot" -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /usr/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/build.sh -} - -pass3_build_with_morph_in_chroot() -{ - echo "Building Baserock with morph" - - cat <<'EOF' >"$LFS/usr/bin/linux-user-chroot" -#!/bin/sh - -CHDIR=. - -while true; do - case "$1" in - --help) - echo 'See "man linux-user-chroot"' - exit 0 - ;; - --version) - echo 'Fake' - exit 0 - ;; - --mount-bind) - # swallow option and arguments - shift 3 - ;; - --mount-readonly) - shift 2 - ;; - --mount-proc|--unshare-ipc|--unshare-pid) - # swallow configure flag - shift - ;; - --chdir) - CHDIR="$2" - shift 2 - ;; - *) - # terminate arg processing - ROOTDIR="$1" - shift - break - ;; - esac -done - -exec chroot "$ROOTDIR" sh -c 'cd "$1" && shift && exec "$@"' -- "$CHDIR" "$@" -EOF - chmod +x "$LFS/usr/bin/linux-user-chroot" - - cat <<EOF > "$LFS/baserock/build_pass3.sh" -#!/bin/bash -set -e -set -x - -export PATH="/usr/bin:/bin:/usr/sbin:/sbin" -cd /baserock/gits/morph - -filler=\$(mktemp) -scripts/assemble-stratum --cachedir /baserock/cache \ - "$LFS/baserock/cache/artifacts/"*.stratum.bootstrap-pass2 \ - "\$filler" bootstrap-pass2 - -python ./morph --verbose build-morphology \ - baserock:baserock/morphs master $PASS3_STRATUM \ - --staging-chroot \ - --staging-filler "\$filler" \ - --no-git-update \ - --cachedir=/baserock/cache \ - --log=/baserock/morph.log \ - --dump-memory-profile=none \ - --keep-path \ - --tarball-server="$GIT_TARBALLS" \ - --target-cflags="$TARGET_CFLAGS" -rm "\$filler" -EOF - $HOST_CHMOD +x "$LFS/baserock/build_pass3.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT >"$do_chroot" <<EOF -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /usr/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/build_pass3.sh -} - -pass3_dump_build_times() -{ - echo "Dumping pass 3 build times" - - cat <<EOF > "$LFS/baserock/dump-pass3-build-times.sh" -#!/bin/bash -set -e -set -x - -export PATH="/usr/bin:/bin" -cd /baserock/gits/morph -python ./dump-build-times /baserock/cache/artifacts -EOF - $HOST_CHMOD +x "$LFS/baserock/dump-pass3-build-times.sh" - local do_chroot="$BASEDIR/do-chroot.bash" - $HOST_CAT >"$do_chroot" <<EOF -#!/bin/bash -trap "umount $LFS/proc $LFS/sys || true" INT TERM EXIT -set -e -set -x -if ! mount | grep "$LFS/proc" >/dev/null -then - mount -t proc proc "$LFS/proc" -fi -if ! mount | grep "$LFS/sys" >/dev/null -then - mount -t sysfs sysfs "$LFS/sys" -fi -$HOST_CP -f /etc/resolv.conf "$LFS/etc/resolv.conf" -/usr/sbin/chroot "$LFS" \\ - /usr/bin/env -i HOME=/baserock TERM=\$TERM \\ - PATH="/bin:/usr/bin:/sbin:/usr/sbin" \\ - "\${@-\$SHELL}" -EOF - $HOST_CHMOD +x "$do_chroot" - "$do_chroot" /baserock/dump-pass3-build-times.sh -} -echo "Bootstrapping Baserock development environment" -echo "LFS_TGT=$LFS_TGT" - -time pass1_directories - -case "$buildwhat" in - pass1a) - time pass1_get_sources_with_morph - ;; - pass1b) - time pass1_build_with_morph - time pass1_dump_build_times - ;; - pass2a) - time pass2_get_morph - time pass2_prepare_for_chroot - time pass2_get_sources_with_morph_in_chroot - time pass2a_cleanup_at_end - ;; - pass2b) - time pass2_build_with_morph_in_chroot - time pass2_dump_build_times - time pass2b_cleanup_at_end - ;; - pass3a) - time pass3_remove_tools - time pass3_get_sources_with_morph_in_chroot - ;; - pass3b) - time pass3_build_with_morph_in_chroot - time pass3_dump_build_times - ;; - *) echo "Usage! (sorry, I'm unhelpful)" 1>&2 - exit 1 - ;; -esac - -echo "$buildwhat finished OK" - @@ -107,8 +107,6 @@ then fi case "$x" in - baserock-bootstrap) - ;; *) if awk 'length > 79' "$x" | grep . > /dev/null then diff --git a/dump-build-times b/dump-build-times deleted file mode 100755 index 48fdca65..00000000 --- a/dump-build-times +++ /dev/null @@ -1,75 +0,0 @@ -#!/usr/bin/env python -# -# Copyright (C) 2012 Codethink Limited -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; version 2 of the License. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License along -# with this program; if not, write to the Free Software Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. - - -import cliapp -import glob -import json -import os -import re -import StringIO - - -class ExtractBuildTimes(cliapp.Application): - - '''Extracts build times of chunks in a morph cache directory. - - Given a morph cache directory as the first argument, this app finds all - cached chunks, loads their meta data and prints their build times. - - ''' - - def process_args(self, args): - cachedir = args[0] - - def chunk_hash(chunk): - short = re.split('\.', chunk) - return os.path.basename(short[-3]) - - def chunk_name(chunk): - short = re.split('\.', chunk) - return short[-1] - - chunks = glob.glob(os.path.join(cachedir, '*.chunk.*')) - items = [] - - for chunk in chunks: - hash = chunk_hash(chunk) - metafile = os.path.join(cachedir, '%s.meta' % hash) - with open(metafile) as f: - data = f.read() - io = StringIO.StringIO(data) - metainfo = json.load(io) - time = metainfo['build-times']['overall-build']['delta'] - minutes = float(time) / 60.0 - items.append((chunk_name(chunk), minutes)) - - items = sorted(items, key=lambda x: x[1], reverse=True) - print '%s' % (43 * '-') - print 'Build times of cached chunks in' - print '%s' % cachedir - print '%s' % (43 * '-') - sum = 0.0 - for name, time in items: - print '%30s: %6.1f mins' % (name, time) - sum += time - print '%s' % (43 * '-') - print '%30s: %6.1f mins' % ('total', sum) - - -if __name__ == '__main__': - ExtractBuildTimes().run() diff --git a/run-bootstrap-in-chroot b/run-bootstrap-in-chroot deleted file mode 100755 index fbe1e9f5..00000000 --- a/run-bootstrap-in-chroot +++ /dev/null @@ -1,199 +0,0 @@ -#!/bin/bash - -set -e - -unmount_virtual() -{ - umount "$1/proc" || true - umount "$1/sys" || true - umount "$1/tree/proc" || true - umount "$1/tree/sys" || true -} - -pass_snapshot() -{ - echo -n "$snapshotdir/$1-snapshot.tar.gz" -} - -has_pass() -{ - if [ -e $(pass_snapshot "$1") ] - then - return 0 - else - return 1 - fi -} - -update_morph() -{ - local dir="$1" - cp baserock-bootstrap "$dir/." # update bootstrap script - rm -rf "$dir/tree/baserock/gits/morph" - mkdir -p "$dir/tree/baserock/gits/morph" - - # Copy everything except the target directory into the target directory. - # The point is to be able to keep the working area for a bootstrap inside - # the morph source directory. This is useful for Jenkins jobs. - local base=$(basename $(basename "$dir")) - find . -mindepth 1 -maxdepth 1 ! -name "$base" ! -name '*.tar.gz' \ - -exec cp -a '{}' "$dir/tree/baserock/gits/morph" ';' -} - -run_pass() -{ - local dir="$1" - local passname="$2" - local tarball=$(pass_snapshot "$passname") - - if "$snapshot" && has_pass "$passname" - then - tar -C "$dir" -xhf "$tarball" - update_morph "$dir" - else - update_morph "$dir" - "./do-squeeze-chroot" bash -x baserock-bootstrap "$passname" || exit 1 - if "$snapshot" - then - tar -C "$dir" -caf "$tarball" . - fi - fi -} - -export LC_ALL=C - -if [ "x$1" = x ] -then - echo "Usage: $0 chroot-dir" 1>&2 - exit 1 -fi - -mkdir -p "$1" -dir="$1/squeeze-chroot" -: ${snapshot:=true} -snapshotdir="$1" - -cat >"./do-squeeze-chroot" <<EOF -#!/bin/sh - -# clear the temporary directory used outside the chroot -export TMPDIR= - -if mount -t proc proc "$dir/proc"; then - trap "umount \"$dir/proc\"" INT TERM EXIT - if mount -t sysfs sysfs "$dir/sys"; then - trap "umount \"$dir/proc\" \"$dir/sys\"" INT TERM EXIT - if [ "x$CCACHE_HOST_DIR" != "x" ]; then - if mount --bind "$CCACHE_HOST_DIR" "$dir/var/tmp/ccache"; then - trap "umount \"$dir/proc\" \"$dir/sys\" \"$dir/var/tmp/ccache\"" \ - INT TERM EXIT - chroot "$dir" "\$@" - fi - else - chroot "$dir" "\$@" - fi - fi -fi -EOF -chmod +x "./do-squeeze-chroot" - -if ([ "x$DEBIAN_MIRROR" = x ] && echo DEBIAN_MIRROR is unspecified >&2) || - ([ "x$GIT_TARBALLS" = x ] && echo GIT_TARBALLS is unspecified >&2) -then - echo You have to set DEBIAN_MIRROR and other environment variables 1>&2 - exit 1 -fi - -if ! which dpkg 2> /dev/null; then - echo "Warning: dpkg not found -- should debootstrap fail to autodetect " - echo "your architecture, set ARCH in the environment (probably to amd64)" -fi - -# prepare the ccache directory in the chroot, if necessary -if [ "x$CCACHE_HOST_DIR" = "x" ]; then - # print a warning if the CCACHE_HOST_DIR is not set - echo "CCACHE_HOST_DIR is unspecified, but that's ok" >&2 -fi - -unmount_virtual "$dir" -rm -rf "$dir" -mkdir "$dir" - -chrootsnapshot="$snapshotdir/squeeze.tar.gz" -if ! "$snapshot" || ! has_pass pass1a; then - if "$snapshot" && [ -e "$chrootsnapshot" ] - then - tar -C "$dir" -xhf "$chrootsnapshot" - else - EXTRAPACKAGES="build-essential,gawk,bison,flex,python,autoconf" - EXTRAPACKAGES="$EXTRAPACKAGES,autopoint,automake,gettext,libtool" - EXTRAPACKAGES="$EXTRAPACKAGES,help2man,texinfo,sudo,ccache,gperf" - EXTRAPACKAGES="$EXTRAPACKAGES,python-pip,python-simplejson,python-yaml" - - EXTRAARGS= - if [ -n $ARCH]; then - EXTRAARGS=$ARCH - fi - - debootstrap $EXTRAARGS --include="$EXTRAPACKAGES" squeeze \ - "$dir" "$DEBIAN_MIRROR" - - mkdir -p "$dir/etc" - hostname > "$dir/etc/hostname" - cat <<EOF > "$dir/etc/hosts" -127.0.0.1 localhost -127.0.1.1 `hostname` -EOF - - # create the directory for cached ccache object files - mkdir -p "$dir/var/tmp/ccache" - - # manually build and install cliapp - "./do-squeeze-chroot" mkdir -p /src - "./do-squeeze-chroot" \ - git clone git://roadtrain.codethink.co.uk/delta/cliapp /src/cliapp - "./do-squeeze-chroot" \ - sh -c 'cd /src/cliapp && "$@"' -- python setup.py build - "./do-squeeze-chroot" \ - sh -c 'cd /src/cliapp && "$@"' -- python setup.py install - - "./do-squeeze-chroot" pip install ordereddict - - if "$snapshot" - then - tar -caf "$chrootsnapshot" -C "$dir" . - fi - fi -fi - -# Unpack existing snapshot, or run pass1 of bootstrap and then make snapshot. - -if ! "$snapshot" || ! has_pass pass1b -then - run_pass "$dir" pass1a -fi - -if ! "$snapshot" || ! has_pass pass2a -then - run_pass "$dir" pass1b -fi - -if ! "$snapshot" || ! has_pass pass2b -then - run_pass "$dir" pass2a -fi - -if ! "$snapshot" || ! has_pass pass3a -then - run_pass "$dir" pass2b -fi - -if ! "$snapshot" || ! has_pass pass3b -then - run_pass "$dir" pass3a -fi - -snapshot=false run_pass "$dir" pass3b - -echo "Passes 1, 2, and 3 of bootstrap done (possibly cached)." - diff --git a/scripts/assemble-stratum b/scripts/assemble-stratum deleted file mode 100755 index 9addd36d..00000000 --- a/scripts/assemble-stratum +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/env python -# -# Copyright (C) 2011-2012 Codethink Limited -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; version 2 of the License. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License along -# with this program; if not, write to the Free Software Foundation, Inc., -# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. - -# This is a program to convert the json dump of the overlaps between artifacts -# in a format more suited to shell programs, or human reading - -import json -import tarfile -import os - -import cliapp - - -class AssembleStratum(cliapp.Application): - - def add_settings(self): - self.settings.string(['cachedir'], - 'Where the cache basedir is') - self.settings.string(['tarformat'], - 'What format to write tar to', - default='') - - def process_args(self, args): - chunklist = json.load(open(args[0])) - tarformat = 'w' - if self.settings['tarformat'] != "": - tarformat += self.settings['tarformat'] - outfile = tarfile.open(args[1], tarformat) - # concatenate chunk tarballs - for chunk in chunklist: - path = os.path.join(self.settings['cachedir'], 'artifacts', chunk) - chunktar = tarfile.open(path, mode='r:*') - for tarinfo in chunktar: - if tarinfo.isfile(): - outfile.addfile(tarinfo, chunktar.extractfile(tarinfo)) - else: - outfile.addfile(tarinfo) - chunktar.close() - # add the stratum's metadata - if os.path.exists(args[0] + '.meta'): - outfile.add(args[0] + '.meta', - os.path.join('baserock', '%s.meta' % args[2])) - outfile.close() - -AssembleStratum().run() diff --git a/scripts/bootstrap b/scripts/bootstrap deleted file mode 100755 index 37ce6952..00000000 --- a/scripts/bootstrap +++ /dev/null @@ -1,555 +0,0 @@ -#!/bin/sh - -# Build an intermediate build-essential stratum using the host's tools -# -- everything is installed into $PREFIX inside a chroot -# -- a "cross-compiler" toolchain is built first, as suggested by LFS, and -# this builds the actual build-essential tools. -# -- unlike LFS, the initial toolchain goes into a separate prefix. This is -# because programs in the target chroot cannot be executed outside of the -# chroot without creating a symlink in the root directory of the host -# system, which is not pleasant. -# -- this also means we effectively always cross-compile, because nothing in -# the chroot is executable during the build process. This is good, because -# build-essential must be cross-compilable so that we can port Baserock -# to new architectures. -# -- we can't build g++, because since it's effectively a cross build we can't -# bootstrap the compiler, and although theoretically we could build one in -# the temporary toolchain, that doesn't actually work. Since the next step -# is a native build we can get a c++ compiler in stage 2 of the bootstrap. - -# In future Morph will handle the build of build-essential that is currently -# done by this script. - -# Disable hashing, so that newly built tools are picked up in PATH immediately -set +h - -set -eu - -## Configuration - -# Installing to a different sysroot (e.g. /tools) is supported, but in order -# to use the result as a staging filler Morph needs to know to create /bin and -# /lib{64} symlinks in the staging area if not present (creating the symlinks -# in the filler itself will conflict with the fhs-dirs chunk) -PREFIX=/usr - -export MAKEFLAGS="-j 4" -export CFLAGS="-O2" - -# i686 -#TARGET="i686-baserock-linux-gnu" -#TARGET_GCC_CONFIG= -#export ARCH=i386 - -# x86_64 -TARGET="x86_64-baserock-linux-gnu" -TARGET_GCC_CONFIG= -export ARCH="x86_64" - -# Little-endian ARM -#TARGET="armv7l-baserock-linux-gnueabi" -#TARGET_GCC_CONFIG="--with-arch=armv7-a" -#export ARCH="arm" - -# Big-endian ARM -# No stable GCC release as of January 2013 can default to big-endian linking. -# You must backport a patch from trunk to make this work. See: -# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16350 -# Correct linking also depends on the correct --with-arch= flag being given. -#TARGET="armv7leb-baserock-linux-gnueabi" -#TARGET_GCC_CONFIG="--with-arch=armv7-a" -#export ARCH="arm" - - -## Setup - -if [ "$#" -ne "1" ]; then - echo "Usage: $0 BASE_DIR" - echo - echo " Compiles a build-essential chroot in BASE_DIR. The directory" - echo " BASE_DIR/gits is expected to contain checkouts of all source" - echo " code required for the builds; you will currently have to" - echo " assemble this yourself based on the source code of this script." - exit 1 -fi - -BASE_DIR="$1" - -if [ ! -d "$BASE_DIR" ]; then - echo "$BASE_DIR does not exist" - exit 1 -fi - -BASE_DIR="$(readlink -f $BASE_DIR)" - -TOOLCHAIN_DIR="$BASE_DIR"/toolchain -CHROOT_DIR="$BASE_DIR"/chroot - -if [ ! -d "$BASE_DIR" ]; then - echo "$BASE_DIR does not exist" - exit 1 -fi - -if [ ! -d "$BASE_DIR/gits" ]; then - echo "$BASE_DIR/gits does not exist" - exit 1 -fi - -if [ ! -d "$TOOLCHAIN_DIR" ]; then - mkdir -p "$TOOLCHAIN_DIR" -fi - -if [ ! -d "$CHROOT_DIR" ]; then - mkdir -p "$CHROOT_DIR" -fi - - -## Architecture-specific hacks - -fix_chroot_for_target() { - case "$TARGET" in - x86_64*) - # eglibc 2.15 is belligerant about putting things into /lib64, - # especially when prefix is /usr. Currently if PREFIX is NOT /usr - # we force ld.so to be in PREFIX/lib/ld.so; but eglibc installs it - # in /lib64 anyway. - if [ ! -e $CHROOT_DIR/lib64/ld-linux-x86-64.so.2 ]; then - mkdir -p $CHROOT_DIR$PREFIX/lib64 - ln -sf $CHROOT_DIR$PREFIX/lib/ld-linux-x86-64.so.2 \ - $CHROOT_DIR$PREFIX/lib64 - fi ;; - esac -} - -## Build process - -assert_branch() { - branch="$1" - if which git > /dev/null && - [ $(git rev-parse HEAD) != $(git rev-parse "$branch") ]; then - echo "Expected to be building '$branch' branch in $(basename $(pwd))" \ - >&2 - exit 1 - fi -} - -touch_tree() { - # git doesn't preserve mtimes, but this causes rebuilds of things that - # haven't changed which breaks builds if the correct tools are not - # available - #date=$(date '+%C%y%m%d%H%M.%S') - #find -type f -exec touch -m -t $date \{} \; - return 0 -} - -# build-essential - -build_binutils() { - pass="$1" - - source_dir="$BASE_DIR"/gits/binutils-redhat - build_dir="$BASE_DIR"/builds/binutils-"$pass" - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - # Note for shell escape purists: quotes around --with-lib-path's argument - # do not work correctly - if [ "$pass" == "pass1" ]; then - extra_config="--target=$TARGET --with-sysroot=\"$CHROOT_DIR\" \ - --with-lib-path=$CHROOT_DIR$PREFIX/lib" - elif [ "$pass" == "pass2" ]; then - extra_config="--host=$TARGET --with-lib-path=$PREFIX/lib " - else - echo "Invalid pass for binutils: $pass" - exit 1 - fi - - # Note that the root configure script's --help doesn't display the - # arguments for sub-configure scripts, but it does pass on arguments to - # them - "$source_dir"/configure \ - --prefix=$PREFIX --disable-nls --disable-werror \ - $extra_config - make - make DESTDIR="$DESTDIR" install -} - -build_busybox() { - source_dir="$BASE_DIR"/gits/busybox - build_dir="$BASE_DIR"/builds/busybox - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - # Busybox's default config includes basically everything - make KBUILD_SRC="$source_dir" -f "$source_dir"/Makefile defconfig - - if [ "$PREFIX" != "/usr" ]; then - # Install everything into DESTDIR instead of putting some of it into - # DESTDIR/usr/. For compatibility with the old Baserock staging - # fillers, if the prefix is /usr we still do the traditional /bin vs. - # /usr/bin split for now. - sed -e 's/.*CONFIG_INSTALL_NO_USR.*/CONFIG_INSTALL_NO_USR=y/' -i \ - .config - fi - - sed -e 's/.*CONFIG_STATIC.*/CONFIG_STATIC=y/' -i .config - - # Requires stuff that was removed after eglibc 2.14 - sed -e 's/.*CONFIG_INETD.*/# CONFIG_INETD is not set/' -i .config - - # Disable all module tools. BusyBox's depmod isn't sufficient for Linux - # builds, but Linux will build OK if depmod isn't present at all. Also, we - # have kmod in Baserock which can do all this stuff anyway. - sed -e 's/.*MODPROBE_SMALL=.*/# CONFIG_MODPROBE_SMALL is not set/' \ - -i .config - sed -e 's/.*INSMOD=.*/CONFIG_INSMOD=y/' -i .config - sed -e 's/.*RMMOD=.*/CONFIG_RMMOD=y/' -i .config - sed -e 's/.*LSMOD=.*/CONFIG_LSMOD=y/' -i .config - sed -e 's/.*DEPMOD=.*/CONFIG_DEPMOD=y/' -i .config - sed -e 's/.*MODPROBE=.*/CONFIG_MODPROBE=y/' -i .config - - sed -e 's/.*CONFIG_INETD.*/# CONFIG_INETD is not set/' -i .config - - - make CROSS_COMPILE="$TARGET-" - - if [ "$PREFIX" != "/usr" ]; then - make CROSS_COMPILE="$TARGET-" CONFIG_PREFIX="$DESTDIR$PREFIX" install - else - make CROSS_COMPILE="$TARGET-" CONFIG_PREFIX="$DESTDIR" install - fi -} - -build_eglibc() { - source_dir="$BASE_DIR"/gits/eglibc2 - build_dir="$BASE_DIR"/builds/eglibc - - # Necessary for ARM port. - if [ ! -e "$source_dir/libc/ports" ]; then - ln -s "$source_dir/ports" "$source_dir/libc/ports" - fi - - cd "$source_dir" - assert_branch "baserock/2.15-build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - # If prefix is set to /usr, eglibc otherwise decides to install its - # libraries in /usr/lib64 on some Linux - extra_config="--libdir=$PREFIX/lib" - - # Suggested by Linux From Scratch. Is the default really to build with - # profiling?? - extra_config="--disable-profile $extra_config" - - # Minimum kernel version that this eglibc will be usable with. - extra_config="--enable-kernel=2.6.25 $extra_config" - - # Location of headers - extra_config="--with-headers=$CHROOT_DIR$PREFIX/include $extra_config" - - # Force configure flags of certain things that can't be detected in a - # cross-compile. - extra_config="$extra_config \ - libc_cv_c_cleanup=yes libc_cv_ctors_header=yes \ - libc_cv_forced_unwind=yes libc_cv_ssp=no" - - # + --without-fp for ARM - "$source_dir"/libc/configure --prefix=$PREFIX --host="$TARGET" \ - --build=$("$source_dir"/libc/scripts/config.guess) \ - --enable-add-ons=nptl,ports $extra_config - make - make "install_root=$DESTDIR" install -} - -build_gcc() { - pass="$1" - source_dir="$BASE_DIR"/gits/gcc-tarball - build_dir="$BASE_DIR"/builds/gcc-"$pass" - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - # This hack is to prevent the multilib configuration of the host OS - # leaking into Baserock, which does not use multilib at all. Without this, - # the pass2 gcc will install its libraries into $PREFIX/lib64 instead of - # $PREFIX/lib, because its configure script decides ${toolexeclibdir} - # based on the output of gcc -print-multi-os-directory - if [ "$(echo $TARGET | cut -c -6)" = "x86_64" ]; then - sed -i "$source_dir/gcc/config/i386/t-linux64" \ - -e "/^MULTILIB_OSDIRNAMES/ c\ - MULTILIB_OSDIRNAMES = ." - fi - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - if [ "$pass" == "pass1" ]; then - # In pass 1 we build a crosscompiler for $TARGET. - extra_config="--target=$TARGET" - - # The pass 1 compiler needs to find the libraries we build in pass 2. - # Include path must be set explicility, because it defaults to - # $CHROOT_DIR/usr/include. - extra_config="$extra_config \ - --with-sysroot="$CHROOT_DIR" \ - --with-native-system-header-dir=\"$CHROOT_DIR$PREFIX/include\"" - - # Disable stuff that doesn't work when building a cross compiler - # without an existing libc, and generally try to keep this build as - # simple as possible. - extra_config="$extra_config \ - --enable-languages=c \ - --disable-decimal-float --disable-libmudflap \ - --disable-libquadmath --disable-libssp --disable-shared \ - --disable-threads --disable-target-libiberty \ - --disable-target-zlib --without-headers --with-newlib \ - --with-system-zlib" - - # way too slow with this, but it maybe a good idea / required - extra_config="--disable-bootstrap $extra_config" - elif [ "$pass" == "pass2" ]; then - # Pass 1 gcc's fixincludes process created a limits.h before there was - # a real limits.h available for the target. This step (taken from - # Linux From Scratch) creates a better one so gcc can compile. - libgcc_dir=$(dirname $($TARGET-gcc -print-libgcc-file-name)) - cat "$source_dir/gcc/limitx.h" "$source_dir/gcc/glimits.h" \ - "$source_dir/gcc/limity.h" \ - > "$libgcc_dir/include-fixed/limits.h" - - # This time we are creating a native compiler running on $TARGET - extra_config="--host=$TARGET" - - # I'm not sure why this is needed. target should default to host, - # but if we don't pass this explicitly some of the files that should - # go in $PREFIX/lib/gcc/$TARGET/4.6.2 end up in - # $PREFIX/lib/gcc/$TARGET/4.6.3 instead. Weird. - extra_config="--target=$TARGET $extra_config" - - # The two-step compiler process means we don't need to bootstrap now - # (and couldn't anyway, since build != host for pass 2). - extra_config="--disable-bootstrap $extra_config" - - # C++ doesn't build without a C++ compiler, I think, so we still - # can't have that yet. We don't need one anyway. - extra_config="$extra_config \ - --enable-clocale=gnu \ - --enable-languages=c --enable-shared \ - --enable-threads=posix" - else - echo "Invalid pass for gcc: $pass" - exit 1 - fi - - # An attempt to stop anything going in $PREFIX/lib64 - extra_config=" --libdir=$PREFIX/lib $extra_config" - - # Disable searching /usr/local/include for headers - extra_config="--with-local-prefix=$PREFIX $extra_config" - - # It makes no sense for Baserock to use multilib. - extra_config="--disable-multilib $extra_config" - - # General stuff that we don't need / won't work right now - extra_config="$extra_config \ - --disable-libgomp --without-cloog --without-ppl" - - # mpfr is built as part of gcc, but we need to point latter components - # to the result of this build. - extra_config="$extra_config \ - --with-mpfr-include="$source_dir"/mpfr/src \ - --with-mpfr-lib="$build_dir"/mpfr/src/.libs" - - "$source_dir"/configure --prefix=$PREFIX --disable-nls \ - $TARGET_GCC_CONFIG $extra_config - make - make DESTDIR="$DESTDIR" install -} - -build_linux_api_headers() { - source_dir="$BASE_DIR"/gits/linux - build_dir="$BASE_DIR"/builds/linux-api-headers - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - - # We don't achieve a real out of tree build here :( - make O="$build_dir" mrproper - #make O="$build_dir" headers_check - make O="$build_dir" INSTALL_HDR_PATH="$DESTDIR$PREFIX" headers_install -} - -# build-essential-plus - -build_gawk() { - source_dir="$BASE_DIR"/gits/gawk - build_dir="$BASE_DIR"/builds/gawk - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - $source_dir/configure --prefix=$PREFIX --host=$TARGET --disable-nls - make - make DESTDIR="$DESTDIR" install -} - -build_make() { - source_dir="$BASE_DIR"/gits/make - build_dir="$BASE_DIR"/builds/make - - cd "$source_dir" - assert_branch "baserock/build-essential" - touch_tree - - rm -Rf "$build_dir" && mkdir -p "$build_dir" - cd "$build_dir" - - $source_dir/configure --prefix=$PREFIX --host=$TARGET --disable-nls - make - make DESTDIR="$DESTDIR" install -} - - -## 1. Build a "cross-compiler" -# -# We always do two builds of the compiler for now; hopefully the rebootstrap -# process in Baserock will allow us to optimise this out when not -# cross-compiling (we will never be cross-compiling except when bootstrapping -# a new architecture). - -DESTDIR="$TOOLCHAIN_DIR" -export CC="gcc" -export PATH="$TOOLCHAIN_DIR$PREFIX/bin":"$PATH" - -build_binutils pass1 -build_gcc pass1 - -if ! [ -x "$(which $TARGET-gcc)" ]; then - echo "Missing $TARGET-gcc in PATH: something went wrong" - exit 1 -fi - -# A hack so that we can build eglibc with a no-shared-libs gcc; libgcc_eh is -# referenced in the eglibc build, but its static version contains all of the -# necessary symbols anyway. -for f in `find "$TOOLCHAIN_DIR" -name libgcc.a`; do \ - EH="`echo "$f" | sed 's/libgcc/&_eh/'`" && if [ ! -e "$EH" ]; then - ln -s libgcc.a "$EH"; - fi; -done - -## 2. Build the actual intermediate build-essential chroot -## -DESTDIR="$CHROOT_DIR" -export CC= - -# We don't want to customise the specs until we actually have eglibc installed. -specs_dir="$(dirname $($TARGET-gcc --print-libgcc-file-name))" -rm -f "$specs_dir/specs" - -build_linux_api_headers -build_eglibc - -# Passing --with-lib-path to our pass1 linker gives it the correct library -# search path, but this doesn't work for the startup files. The startup files -# (crt*.o) are searched for by gcc itself, and passed to ld as absolute paths -# if found or just filenames if not (in which case, ld will not search for -# them either because they are .o files, not libraries). -# -# One alternative to this hack is to pass -B $CHROOT_DIR$PREFIX/lib to gcc, but -# that option often gets eaten by libtool and by gcc's nested configure -# scripts so it's not fully effective. -$TARGET-gcc -dumpspecs | \ -sed -e "s@\(crt1\|gcrt1\|Scrt1\|crti\|crtn\)\.o@$CHROOT_DIR$PREFIX/lib/&@g" \ - > "$specs_dir/specs" - -if [ "$PREFIX" != "/usr" ]; then - # eglibc (in sysdeps/unix/sysv/linux/configure) puts some files in /lib64 - # if PREFIX is /usr, so for now we go with this and avoid doing any fixups. - # For other prefixes, everything is installed correctly into the sysroot. - # - # In the future we should fix eglibc to put all files into PREFIX/lib at - # which point this fix will always be necessary, until GCC grows a better - # way to specify the location of ld.so. - sed -i "$specs_dir/specs" -e "s@/lib\(64\)\?/ld@$PREFIX/&@g" -fi - -fix_chroot_for_target - -echo -echo "Testing pass 1 compiler and chroot ..." - -cat <<EOF > $CHROOT_DIR/build-test.c -#include <stdio.h> -int main() { printf("OK"); return 0; } -EOF -$TARGET-gcc $CFLAGS "$CHROOT_DIR/build-test.c" -o "$CHROOT_DIR/build-test" - -if [ "$PREFIX" != "/usr" ]; then - # Check that we successfully forced use of the ld.so inside the sysroot - if ! readelf -l "$CHROOT_DIR"/build-test | grep -q "interpreter: $PREFIX" ; - then - echo "Wrong interpreter in output of pass 1 C compiler" - exit 1 - fi -fi - -rm "$CHROOT_DIR/build-test" "$CHROOT_DIR/build-test.c" - -export CC=$TARGET-gcc -export AR=$TARGET-ar -export RANLIB=$TARGET-ranlib - -build_binutils pass2 -build_busybox - -# This is hardcoded into gcc as NATIVE_SYSTEM_HEADERS. It's used only when -# running fixincludes, which only makes sense when cross-compiling anyway. -# Compilation fails if this directory is missing anyway, unfortunately. -created_native_system_header_dir="no" -if [ ! -d /usr/include ]; then - mkdir -p /usr/include - created_native_system_header_dir="yes" -fi - -build_gcc pass2 - -if [ ! -e "$DESTDIR$PREFIX/bin/cc" ]; then - ln -s gcc "$DESTDIR$PREFIX/bin/cc" -fi - -if [ "$created_native_system_header_dir" == "yes" ]; then - rm /usr/include - rm /usr -fi - -build_gawk -build_make - -echo -echo "Complete! You now have a build-essential for $TARGET in " -echo "$CHROOT_DIR." -echo -echo "Before it can be used as a staging filler for Morph, you need to " -echo "manually run the install-commands from the 'fhs-dirs' chunk as root to " -echo "set up the necessary files and device nodes." |