diff options
Diffstat (limited to 'hack')
-rw-r--r-- | hack/MAINTAINERS.md | 80 | ||||
-rw-r--r-- | hack/PACKAGERS.md | 278 | ||||
-rw-r--r-- | hack/RELEASE-CHECKLIST.md | 12 | ||||
-rwxr-xr-x | hack/make.sh | 37 | ||||
-rw-r--r-- | hack/make/binary | 9 | ||||
-rw-r--r-- | hack/make/cross | 2 | ||||
-rw-r--r-- | hack/make/dynbinary | 47 | ||||
-rw-r--r-- | hack/make/dyntest | 4 | ||||
-rw-r--r-- | hack/make/dyntest-integration | 4 | ||||
-rw-r--r-- | hack/make/test | 17 | ||||
-rw-r--r-- | hack/make/test-integration | 9 | ||||
-rw-r--r-- | hack/make/ubuntu | 131 |
12 files changed, 412 insertions, 218 deletions
diff --git a/hack/MAINTAINERS.md b/hack/MAINTAINERS.md index 8944fbee1a..be3117c864 100644 --- a/hack/MAINTAINERS.md +++ b/hack/MAINTAINERS.md @@ -1,22 +1,24 @@ -# The Docker maintainer manual +# The Docker Maintainer manual ## Introduction -Dear maintainer. Thank you for investing the time and energy to help make Docker as -useful as possible. Maintaining a project is difficult, sometimes unrewarding work. -Sure, you will get to contribute cool features to the project. But most of your time -will be spent reviewing, cleaning up, documenting, answering questions, justifying -design decisions - while everyone has all the fun! But remember - the quality of the -maintainers work is what distinguishes the good projects from the great. -So please be proud of your work, even the unglamourous parts, and encourage a culture -of appreciation and respect for *every* aspect of improving the project - not just the -hot new features. +Dear maintainer. Thank you for investing the time and energy to help +make Docker as useful as possible. Maintaining a project is difficult, +sometimes unrewarding work. Sure, you will get to contribute cool +features to the project. But most of your time will be spent reviewing, +cleaning up, documenting, answering questions, justifying design +decisions - while everyone has all the fun! But remember - the quality +of the maintainers work is what distinguishes the good projects from the +great. So please be proud of your work, even the unglamourous parts, +and encourage a culture of appreciation and respect for *every* aspect +of improving the project - not just the hot new features. -This document is a manual for maintainers old and new. It explains what is expected of -maintainers, how they should work, and what tools are available to them. - -This is a living document - if you see something out of date or missing, speak up! +This document is a manual for maintainers old and new. It explains what +is expected of maintainers, how they should work, and what tools are +available to them. +This is a living document - if you see something out of date or missing, +speak up! ## What are a maintainer's responsibility? @@ -24,19 +26,26 @@ It is every maintainer's responsibility to: * 1) Expose a clear roadmap for improving their component. * 2) Deliver prompt feedback and decisions on pull requests. -* 3) Be available to anyone with questions, bug reports, criticism etc. on their component. This includes irc, github requests and the mailing list. -* 4) Make sure their component respects the philosophy, design and roadmap of the project. - +* 3) Be available to anyone with questions, bug reports, criticism etc. + on their component. This includes IRC, GitHub requests and the mailing + list. +* 4) Make sure their component respects the philosophy, design and + roadmap of the project. ## How are decisions made? Short answer: with pull requests to the docker repository. -Docker is an open-source project with an open design philosophy. This means that the repository is the source of truth for EVERY aspect of the project, -including its philosophy, design, roadmap and APIs. *If it's part of the project, it's in the repo. It's in the repo, it's part of the project.* +Docker is an open-source project with an open design philosophy. This +means that the repository is the source of truth for EVERY aspect of the +project, including its philosophy, design, roadmap and APIs. *If it's +part of the project, it's in the repo. It's in the repo, it's part of +the project.* -As a result, all decisions can be expressed as changes to the repository. An implementation change is a change to the source code. An API change is a change to -the API specification. A philosophy change is a change to the philosophy manifesto. And so on. +As a result, all decisions can be expressed as changes to the +repository. An implementation change is a change to the source code. An +API change is a change to the API specification. A philosophy change is +a change to the philosophy manifesto. And so on. All decisions affecting docker, big and small, follow the same 3 steps: @@ -49,25 +58,36 @@ All decisions affecting docker, big and small, follow the same 3 steps: ## Who decides what? -So all decisions are pull requests, and the relevant maintainer makes the decision by accepting or refusing the pull request. -But how do we identify the relevant maintainer for a given pull request? +So all decisions are pull requests, and the relevant maintainer makes +the decision by accepting or refusing the pull request. But how do we +identify the relevant maintainer for a given pull request? -Docker follows the timeless, highly efficient and totally unfair system known as [Benevolent dictator for life](http://en.wikipedia.org/wiki/Benevolent_Dictator_for_Life), -with yours truly, Solomon Hykes, in the role of BDFL. -This means that all decisions are made by default by me. Since making every decision myself would be highly un-scalable, in practice decisions are spread across multiple maintainers. +Docker follows the timeless, highly efficient and totally unfair system +known as [Benevolent dictator for +life](http://en.wikipedia.org/wiki/Benevolent_Dictator_for_Life), with +yours truly, Solomon Hykes, in the role of BDFL. This means that all +decisions are made by default by Solomon. Since making every decision +myself would be highly un-scalable, in practice decisions are spread +across multiple maintainers. The relevant maintainer for a pull request is assigned in 3 steps: -* Step 1: Determine the subdirectory affected by the pull request. This might be src/registry, docs/source/api, or any other part of the repo. +* Step 1: Determine the subdirectory affected by the pull request. This + might be `src/registry`, `docs/source/api`, or any other part of the repo. -* Step 2: Find the MAINTAINERS file which affects this directory. If the directory itself does not have a MAINTAINERS file, work your way up the repo hierarchy until you find one. +* Step 2: Find the `MAINTAINERS` file which affects this directory. If the + directory itself does not have a `MAINTAINERS` file, work your way up + the repo hierarchy until you find one. -* Step 3: The first maintainer listed is the primary maintainer. The pull request is assigned to him. He may assign it to other listed maintainers, at his discretion. +* Step 3: The first maintainer listed is the primary maintainer. The + pull request is assigned to him. He may assign it to other listed + maintainers, at his discretion. ### I'm a maintainer, should I make pull requests too? -Yes. Nobody should ever push to master directly. All changes should be made through a pull request. +Yes. Nobody should ever push to master directly. All changes should be +made through a pull request. ### Who assigns maintainers? diff --git a/hack/PACKAGERS.md b/hack/PACKAGERS.md index b50306430b..5dcb120689 100644 --- a/hack/PACKAGERS.md +++ b/hack/PACKAGERS.md @@ -1,65 +1,94 @@ -Dear packager. +# Dear Packager, -If you are looking to make docker available on your favorite software distribution, -this document is for you. It summarizes the requirements for building and running -docker. +If you are looking to make Docker available on your favorite software +distribution, this document is for you. It summarizes the requirements for +building and running the Docker client and the Docker daemon. -## Getting started +## Getting Started -We really want to help you package Docker successfully. Before anything, a good first step -is to introduce yourself on the [docker-dev mailing list](https://groups.google.com/forum/?fromgroups#!forum/docker-dev) -, explain what you''re trying to achieve, and tell us how we can help. Don''t worry, we don''t bite! -There might even be someone already working on packaging for the same distro! +We want to help you package Docker successfully. Before doing any packaging, a +good first step is to introduce yourself on the [docker-dev mailing +list](https://groups.google.com/d/forum/docker-dev), explain what you're trying +to achieve, and tell us how we can help. Don't worry, we don't bite! There might +even be someone already working on packaging for the same distro! -You can also join the IRC channel - #docker and #docker-dev on Freenode are both active and friendly. +You can also join the IRC channel - #docker and #docker-dev on Freenode are both +active and friendly. -## Package name +We like to refer to Tianon ("@tianon" on GitHub and "tianon" on IRC) as our +"Packagers Relations", since he's always working to make sure our packagers have +a good, healthy upstream to work with (both in our communication and in our +build scripts). If you're having any kind of trouble, feel free to ping him +directly. He also likes to keep track of what distributions we have packagers +for, so feel free to reach out to him even just to say "Hi!" -If possible, your package should be called "docker". If that name is already taken, a second -choice is "lxc-docker". +## Package Name -## Official build vs distro build +If possible, your package should be called "docker". If that name is already +taken, a second choice is "lxc-docker", but with the caveat that "LXC" is now an +optional dependency (as noted below). Another possible choice is "docker.io". -The Docker project maintains its own build and release toolchain. It is pretty neat and entirely -based on Docker (surprise!). This toolchain is the canonical way to build Docker, and the only -method supported by the development team. We encourage you to give it a try, and if the circumstances +## Official Build vs Distro Build + +The Docker project maintains its own build and release toolchain. It is pretty +neat and entirely based on Docker (surprise!). This toolchain is the canonical +way to build Docker. We encourage you to give it a try, and if the circumstances allow you to use it, we recommend that you do. -You might not be able to use the official build toolchain - usually because your distribution has a -toolchain and packaging policy of its own. We get it! Your house, your rules. The rest of this document -should give you the information you need to package Docker your way, without denaturing it in -the process. +You might not be able to use the official build toolchain - usually because your +distribution has a toolchain and packaging policy of its own. We get it! Your +house, your rules. The rest of this document should give you the information you +need to package Docker your way, without denaturing it in the process. -## System build dependencies +## Build Dependencies -To build docker, you will need the following system dependencies +To build Docker, you will need the following: -* An amd64 machine * A recent version of git and mercurial * Go version 1.2 or later +* A clean checkout of the source added to a valid [Go + workspace](http://golang.org/doc/code.html#Workspaces) under the path + *src/github.com/dotcloud/docker* (unless you plan to use `AUTO_GOPATH`, + explained in more detail below). + +To build the Docker daemon, you will additionally need: + +* An amd64/x86_64 machine running Linux * SQLite version 3.7.9 or later -* libdevmapper version 1.02.68-cvs (2012-01-26) or later from lvm2 version 2.02.89 or later -* btrfs-progs version 3.8 or later (including commit e5cb128 from 2013-01-07) for the necessary btrfs headers -* A clean checkout of the source must be added to a valid Go [workspace](http://golang.org/doc/code.html#Workspaces) -under the path *src/github.com/dotcloud/docker*. +* libdevmapper version 1.02.68-cvs (2012-01-26) or later from lvm2 version + 2.02.89 or later +* btrfs-progs version 3.8 or later (including commit e5cb128 from 2013-01-07) + for the necessary btrfs headers -## Go dependencies +Be sure to also check out Docker's Dockerfile for the most up-to-date list of +these build-time dependencies. -All Go dependencies are vendored under ./vendor. They are used by the official build, -so the source of truth for the current version is whatever is in ./vendor. +### Go Dependencies -To use the vendored dependencies, simply make sure the path to ./vendor is included in $GOPATH. +All Go dependencies are vendored under "./vendor". They are used by the official +build, so the source of truth for the current version of each dependency is +whatever is in "./vendor". -If you would rather package these dependencies yourself, take a look at ./hack/vendor.sh for an -easy-to-parse list of the exact version for each. +To use the vendored dependencies, simply make sure the path to "./vendor" is +included in `GOPATH` (or use `AUTO_GOPATH`, as explained below). -NOTE: if you''re not able to package the exact version (to the exact commit) of a given dependency, -please get in touch so we can remediate! Who knows what discrepancies can be caused by even the -slightest deviation. We promise to do our best to make everybody happy. +If you would rather (or must, due to distro policy) package these dependencies +yourself, take a look at "./hack/vendor.sh" for an easy-to-parse list of the +exact version for each. + +NOTE: if you're not able to package the exact version (to the exact commit) of a +given dependency, please get in touch so we can remediate! Who knows what +discrepancies can be caused by even the slightest deviation. We promise to do +our best to make everybody happy. ## Stripping Binaries -Please, please, please do not strip any compiled binaries. This is really important. +Please, please, please do not strip any compiled binaries. This is really +important. + +In our own testing, stripping the resulting binaries sometimes results in a +binary that appears to work, but more often causes random panics, segfaults, and +other issues. Even if the binary appears to work, please don't strip. See the following quotes from Dave Cheney, which explain this position better from the upstream Golang perspective. @@ -94,79 +123,172 @@ from the upstream Golang perspective. ## Building Docker -To build the docker binary, run the following command with the source checkout as the -working directory: +Please use our build script ("./hack/make.sh") for all your compilation of +Docker. If there's something you need that it isn't doing, or something it could +be doing to make your life as a packager easier, please get in touch with Tianon +and help us rectify the situation. Chances are good that other packagers have +probably run into the same problems and a fix might already be in the works, but +none of us will know for sure unless you harass Tianon about it. :) + +All the commands listed within this section should be run with the Docker source +checkout as the current working directory. + +### `AUTO_GOPATH` + +If you'd rather not be bothered with the hassles that setting up `GOPATH` +appropriately can be, and prefer to just get a "build that works", you should +add something similar to this to whatever script or process you're using to +build Docker: ```bash -./hack/make.sh binary +export AUTO_GOPATH=1 ``` -This will create a static binary under *./bundles/$VERSION/binary/docker-$VERSION*, where -*$VERSION* is the contents of the file *./VERSION*. +This will cause the build scripts to set up a reasonable `GOPATH` that +automatically and properly includes both dotcloud/docker from the local +directory, and the local "./vendor" directory as necessary. -You are encouraged to use ./hack/make.sh without modification. If you must absolutely write -your own script (are you really, really sure you need to? make.sh is really not that complicated), -then please take care the respect the following: +### `DOCKER_BUILDTAGS` -* In *./hack/make.sh*: $LDFLAGS, $BUILDFLAGS, $VERSION and $GITCOMMIT -* In *./hack/make/binary*: the exact build command to run +If you're building a binary that may need to be used on platforms that include +AppArmor, you will need to set `DOCKER_BUILDTAGS` as follows: -You may be tempted to tweak these settings. In particular, being a rigorous maintainer, you may want -to disable static linking. Please don''t! Docker *needs* to be statically linked to function properly. -You would do the users of your distro a disservice and "void the docker warranty" by changing the flags. +```bash +export DOCKER_BUILDTAGS='apparmor' +``` -A good comparison is Busybox: all distros package it as a statically linked binary, because it just -makes sense. Docker is the same way. +### Static Daemon -If you *must* have a non-static Docker binary, please use: +If it is feasible within the constraints of your distribution, you should +seriously consider packaging Docker as a single static binary. A good comparison +is Busybox, which is often packaged statically as a feature to enable mass +portability. Because of the unique way Docker operates, being similarly static +is a "feature". + +To build a static Docker daemon binary, run the following command (first +ensuring that all the necessary libraries are available in static form for +linking - see the "Build Dependencies" section above, and the relevant lines +within Docker's own Dockerfile that set up our official build environment): + +```bash +./hack/make.sh binary +``` + +This will create a static binary under +"./bundles/$VERSION/binary/docker-$VERSION", where "$VERSION" is the contents of +the file "./VERSION". This binary is usually installed somewhere like +"/usr/bin/docker". + +### Dynamic Daemon / Client-only Binary + +If you are only interested in a Docker client binary, set `DOCKER_CLIENTONLY` to a non-empty value using something similar to the following: (which will prevent the extra step of compiling dockerinit) + +```bash +export DOCKER_CLIENTONLY=1 +``` + +If you need to (due to distro policy, distro library availability, or for other +reasons) create a dynamically compiled daemon binary, or if you are only +interested in creating a client binary for Docker, use something similar to the +following: ```bash ./hack/make.sh dynbinary ``` -This will create *./bundles/$VERSION/dynbinary/docker-$VERSION* and *./bundles/$VERSION/binary/dockerinit-$VERSION*. -The first of these would usually be installed at */usr/bin/docker*, while the second must be installed -at */usr/libexec/docker/dockerinit*. +This will create "./bundles/$VERSION/dynbinary/docker-$VERSION", which for +client-only builds is the important file to grab and install as appropriate. -## Testing Docker +For daemon builds, you will also need to grab and install +"./bundles/$VERSION/dynbinary/dockerinit-$VERSION", which is created from the +minimal set of Docker's codebase that _must_ be compiled statically (and is thus +a pure static binary). The acceptable locations Docker will search for this file +are as follows (in order): -Before releasing your binary, make sure to run the tests! Run the following command with the source -checkout as the working directory: +* as "dockerinit" in the same directory as the daemon binary (ie, if docker is + installed at "/usr/bin/docker", then "/usr/bin/dockerinit" will be the first + place this file is searched for) +* "/usr/libexec/docker/dockerinit" or "/usr/local/libexec/docker/dockerinit" + ([FHS 3.0 Draft](http://www.linuxbase.org/betaspecs/fhs/fhs.html#usrlibexec)) +* "/usr/lib/docker/dockerinit" or "/usr/local/lib/docker/dockerinit" ([FHS + 2.3](http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA)) + +If (and please, only if) one of the paths above is insufficient due to distro +policy or similar issues, you may use the `DOCKER_INITPATH` environment variable +at compile-time as follows to set a different path for Docker to search: ```bash -./hack/make.sh test +export DOCKER_INITPATH=/usr/lib/docker.io/dockerinit ``` -The test suite includes both live integration tests and unit tests, so you will need all runtime -dependencies to be installed (see below). +If you find yourself needing this, please don't hesitate to reach out to Tianon +to see if it would be reasonable or helpful to add more paths to Docker's list, +especially if there's a relevant standard worth referencing (such as the FHS). + +Also, it goes without saying, but for the purposes of the daemon please consider +these two binaries ("docker" and "dockerinit") as if they were a single unit. +Mixing and matching can cause undesired consequences, and will fail to run +properly. -The test suite will also download a small test container, so you will need internet connectivity. +## System Dependencies -## Runtime dependencies +### Runtime Dependencies -To run properly, docker needs the following software to be installed at runtime: +To function properly, the Docker daemon needs the following software to be +installed and available at runtime: -* iproute2 version 3.5 or later (build after 2012-05-21), and specifically the "ip" utility * iptables version 1.4 or later -* The LXC utility scripts (http://lxc.sourceforge.net) version 0.8 or later +* XZ Utils version 4.9 or later + +Additionally, the Docker client needs the following software to be installed and +available at runtime: + * Git version 1.7 or later -* XZ Utils 4.9 or later -## Kernel dependencies +### Kernel Requirements + +The Docker daemon has very specific kernel requirements. Most pre-packaged +kernels already include the necessary options enabled. If you are building your +own kernel, you will either need to discover the options necessary via trial and +error, or check out the [Gentoo +ebuild](https://github.com/tianon/docker-overlay/blob/master/app-emulation/docker/docker-9999.ebuild), +in which a list is maintained (and if there are any issues or discrepancies in +that list, please contact Tianon so they can be rectified). + +Note that in client mode, there are no specific kernel requirements, and that +the client will even run on alternative platforms such as Mac OS X / Darwin. -Docker in daemon mode has specific kernel requirements. For details, see -http://docs.docker.io/en/latest/installation/kernel/ +### Optional Dependencies -Note that Docker also has a client mode, which can run on virtually any linux kernel (it even builds -on OSX!). +Some of Docker's features are activated by using optional command-line flags or +by having support for them in the kernel or userspace. A few examples include: -## Init script +* LXC execution driver (requires version 0.8 or later of the LXC utility scripts) +* AUFS graph driver (requires AUFS patches/support enabled in the kernel, and at + least the "auplink" utility from aufs-tools) +* experimental BTRFS graph driver (requires BTRFS support enabled in the kernel) -Docker expects to run as a daemon at machine startup. Your package will need to include a script -for your distro''s process supervisor of choice. +## Daemon Init Script -Docker should be run as root, with the following arguments: +Docker expects to run as a daemon at machine startup. Your package will need to +include a script for your distro's process supervisor of choice. Be sure to +check out the "contrib/init" folder in case a suitable init script already +exists (and if one does not, contact Tianon about whether it might be +appropriate for your distro's init script to live there too!). + +In general, Docker should be run as root, similar to the following: ```bash docker -d ``` + +Generally, a `DOCKER_OPTS` variable of some kind is available for adding more +flags (such as changing the graph driver to use BTRFS, switching the location of +"/var/lib/docker", etc). + +## Communicate + +As a final note, please do feel free to reach out to Tianon at any time for +pretty much anything. He really does love hearing from our packagers and wants +to make sure we're not being a "hostile upstream". As should be a given, we +appreciate the work our packagers do to make sure we have broad distribution! diff --git a/hack/RELEASE-CHECKLIST.md b/hack/RELEASE-CHECKLIST.md index a7ae45f2ff..84a0ff70e1 100644 --- a/hack/RELEASE-CHECKLIST.md +++ b/hack/RELEASE-CHECKLIST.md @@ -173,9 +173,13 @@ git push origin $VERSION It's very important that we don't make the tag until after the official release is uploaded to get.docker.io! -### 10. Go to github to merge the `bump_$VERSION` into release +### 10. Go to github to merge the `bump_$VERSION` branch into release -Merging the pull request to the release branch will automatically +Don't delete the leftover branch just yet, as we will need it for the next step. + +### 11. Go to github to merge the `bump_$VERSION` branch into docs + +Merging the pull request to the docs branch will automatically update the documentation on the "latest" revision of the docs. You should see the updated docs 5-10 minutes after the merge. The docs will appear on http://docs.docker.io/. For more information about @@ -184,7 +188,7 @@ documentation releases, see `docs/README.md`. Don't forget to push that pretty blue button to delete the leftover branch afterwards! -### 11. Create a new pull request to merge release back into master +### 12. Create a new pull request to merge release back into master ```bash git checkout master @@ -202,7 +206,7 @@ echo "https://github.com/dotcloud/docker/compare/master...merge_release_$VERSION Again, get two maintainers to validate, then merge, then push that pretty blue button to delete your branch. -### 12. Rejoice and Evangelize! +### 13. Rejoice and Evangelize! Congratulations! You're done. diff --git a/hack/make.sh b/hack/make.sh index c0092b106f..63edca4d4c 100755 --- a/hack/make.sh +++ b/hack/make.sh @@ -16,7 +16,7 @@ set -e # in the Dockerfile at the root of the source. In other words: # DO NOT CALL THIS SCRIPT DIRECTLY. # - The right way to call this script is to invoke "make" from -# your checkout of the Docker repository. +# your checkout of the Docker repository. # the Makefile will do a "docker build -t docker ." and then # "docker run hack/make.sh" in the resulting container image. # @@ -82,9 +82,23 @@ if [ ! "$GOPATH" ]; then fi # Use these flags when compiling the tests and final binary -LDFLAGS='-X github.com/dotcloud/docker/dockerversion.GITCOMMIT "'$GITCOMMIT'" -X github.com/dotcloud/docker/dockerversion.VERSION "'$VERSION'" -w' -LDFLAGS_STATIC='-X github.com/dotcloud/docker/dockerversion.IAMSTATIC true -linkmode external -extldflags "-lpthread -static -Wl,--unresolved-symbols=ignore-in-object-files"' -BUILDFLAGS='-tags netgo -a' +LDFLAGS=' + -w + -X github.com/dotcloud/docker/dockerversion.GITCOMMIT "'$GITCOMMIT'" + -X github.com/dotcloud/docker/dockerversion.VERSION "'$VERSION'" +' +LDFLAGS_STATIC='-linkmode external' +EXTLDFLAGS_STATIC='-static' +BUILDFLAGS=( -a -tags "netgo $DOCKER_BUILDTAGS" ) + +# A few more flags that are specific just to building a completely-static binary (see hack/make/binary) +# PLEASE do not use these anywhere else. +EXTLDFLAGS_STATIC_DOCKER="$EXTLDFLAGS_STATIC -lpthread -Wl,--unresolved-symbols=ignore-in-object-files" +LDFLAGS_STATIC_DOCKER=" + $LDFLAGS_STATIC + -X github.com/dotcloud/docker/dockerversion.IAMSTATIC true + -extldflags \"$EXTLDFLAGS_STATIC_DOCKER\" +" HAVE_GO_TEST_COVER= if \ @@ -101,21 +115,32 @@ fi # go_test_dir() { dir=$1 + coverpkg=$2 testcover=() if [ "$HAVE_GO_TEST_COVER" ]; then # if our current go install has -cover, we want to use it :) mkdir -p "$DEST/coverprofiles" coverprofile="docker${dir#.}" coverprofile="$DEST/coverprofiles/${coverprofile//\//-}" - testcover=( -cover -coverprofile "$coverprofile" ) + testcover=( -cover -coverprofile "$coverprofile" $coverpkg ) fi ( set -x cd "$dir" - go test ${testcover[@]} -ldflags "$LDFLAGS" $BUILDFLAGS $TESTFLAGS + go test ${testcover[@]} -ldflags "$LDFLAGS" "${BUILDFLAGS[@]}" $TESTFLAGS ) } +# This helper function walks the current directory looking for directories +# holding certain files ($1 parameter), and prints their paths on standard +# output, one per line. +find_dirs() { + find -not \( \ + \( -wholename './vendor' -o -wholename './integration' -o -wholename './contrib' -o -wholename './pkg/mflag/example' \) \ + -prune \ + \) -name "$1" -print0 | xargs -0n1 dirname | sort -u +} + bundle() { bundlescript=$1 bundle=$(basename $bundlescript) diff --git a/hack/make/binary b/hack/make/binary index 93e99fee8f..7272b1ede0 100644 --- a/hack/make/binary +++ b/hack/make/binary @@ -2,5 +2,12 @@ DEST=$1 -go build -o $DEST/docker-$VERSION -ldflags "$LDFLAGS $LDFLAGS_STATIC" $BUILDFLAGS ./docker +go build \ + -o $DEST/docker-$VERSION \ + "${BUILDFLAGS[@]}" \ + -ldflags " + $LDFLAGS + $LDFLAGS_STATIC_DOCKER + " \ + ./docker echo "Created binary: $DEST/docker-$VERSION" diff --git a/hack/make/cross b/hack/make/cross index a67ab6c28a..e8f90e29b7 100644 --- a/hack/make/cross +++ b/hack/make/cross @@ -17,7 +17,7 @@ for platform in $DOCKER_CROSSPLATFORMS; do mkdir -p "$DEST/$platform" # bundles/VERSION/cross/GOOS/GOARCH/docker-VERSION export GOOS=${platform%/*} export GOARCH=${platform##*/} - export LDFLAGS_STATIC="" # we just need a simple client for these platforms (TODO this might change someday) + export LDFLAGS_STATIC_DOCKER="" # we just need a simple client for these platforms (TODO this might change someday) source "$(dirname "$BASH_SOURCE")/binary" "$DEST/$platform" ) done diff --git a/hack/make/dynbinary b/hack/make/dynbinary index d5ea6ebe54..d4f583fb62 100644 --- a/hack/make/dynbinary +++ b/hack/make/dynbinary @@ -2,27 +2,40 @@ DEST=$1 -# dockerinit still needs to be a static binary, even if docker is dynamic -CGO_ENABLED=0 go build -o $DEST/dockerinit-$VERSION -ldflags "$LDFLAGS -d" $BUILDFLAGS ./dockerinit -echo "Created binary: $DEST/dockerinit-$VERSION" -ln -sf dockerinit-$VERSION $DEST/dockerinit - -sha1sum= -if command -v sha1sum &> /dev/null; then - sha1sum=sha1sum -elif command -v shasum &> /dev/null; then - # Mac OS X - why couldn't they just use the same command name and be happy? - sha1sum=shasum +if [ -z "$DOCKER_CLIENTONLY" ]; then + # dockerinit still needs to be a static binary, even if docker is dynamic + go build \ + -o $DEST/dockerinit-$VERSION \ + "${BUILDFLAGS[@]}" \ + -ldflags " + $LDFLAGS + $LDFLAGS_STATIC + -extldflags \"$EXTLDFLAGS_STATIC\" + " \ + ./dockerinit + echo "Created binary: $DEST/dockerinit-$VERSION" + ln -sf dockerinit-$VERSION $DEST/dockerinit + + sha1sum= + if command -v sha1sum &> /dev/null; then + sha1sum=sha1sum + elif command -v shasum &> /dev/null; then + # Mac OS X - why couldn't they just use the same command name and be happy? + sha1sum=shasum + else + echo >&2 'error: cannot find sha1sum command or equivalent' + exit 1 + fi + + # sha1 our new dockerinit to ensure separate docker and dockerinit always run in a perfect pair compiled for one another + export DOCKER_INITSHA1="$($sha1sum $DEST/dockerinit-$VERSION | cut -d' ' -f1)" else - echo >&2 'error: cannot find sha1sum command or equivalent' - exit 1 + # DOCKER_CLIENTONLY must be truthy, so we don't need to bother with dockerinit :) + export DOCKER_INITSHA1="" fi - -# sha1 our new dockerinit to ensure separate docker and dockerinit always run in a perfect pair compiled for one another -export DOCKER_INITSHA1="$($sha1sum $DEST/dockerinit-$VERSION | cut -d' ' -f1)" # exported so that "dyntest" can easily access it later without recalculating it ( - export LDFLAGS_STATIC="-X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\" -X github.com/dotcloud/docker/dockerversion.INITPATH \"$DOCKER_INITPATH\"" + export LDFLAGS_STATIC_DOCKER="-X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\" -X github.com/dotcloud/docker/dockerversion.INITPATH \"$DOCKER_INITPATH\"" source "$(dirname "$BASH_SOURCE")/binary" ) diff --git a/hack/make/dyntest b/hack/make/dyntest index 555517fd05..744db3e999 100644 --- a/hack/make/dyntest +++ b/hack/make/dyntest @@ -12,6 +12,8 @@ fi ( export TEST_DOCKERINIT_PATH="$INIT" - export LDFLAGS_STATIC="-X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\"" + export LDFLAGS_STATIC_DOCKER=" + -X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\" + " source "$(dirname "$BASH_SOURCE")/test" ) diff --git a/hack/make/dyntest-integration b/hack/make/dyntest-integration index b3a2d7be63..ef7e6a5a41 100644 --- a/hack/make/dyntest-integration +++ b/hack/make/dyntest-integration @@ -12,6 +12,8 @@ fi ( export TEST_DOCKERINIT_PATH="$INIT" - export LDFLAGS_STATIC="-X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\"" + export LDFLAGS_STATIC_DOCKER=" + -X github.com/dotcloud/docker/dockerversion.INITSHA1 \"$DOCKER_INITSHA1\" + " source "$(dirname "$BASH_SOURCE")/test-integration" ) diff --git a/hack/make/test b/hack/make/test index 760c5a5fc6..39ba5cd3a5 100644 --- a/hack/make/test +++ b/hack/make/test @@ -4,9 +4,9 @@ DEST=$1 set -e -TEXTRESET=$'\033[0m' # reset the foreground colour RED=$'\033[31m' GREEN=$'\033[32m' +TEXTRESET=$'\033[0m' # reset the foreground colour # Run Docker's test suite, including sub-packages, and store their output as a bundle # If $TESTFLAGS is set in the environment, it is passed as extra arguments to 'go test'. @@ -19,10 +19,10 @@ bundle_test() { date TESTS_FAILED=() - for test_dir in $(find_test_dirs); do + for test_dir in $(find_dirs '*_test.go'); do echo - if ! LDFLAGS="$LDFLAGS $LDFLAGS_STATIC" go_test_dir "$test_dir"; then + if ! LDFLAGS="$LDFLAGS $LDFLAGS_STATIC_DOCKER" go_test_dir "$test_dir"; then TESTS_FAILED+=("$test_dir") echo echo "${RED}Tests failed: $test_dir${TEXTRESET}" @@ -48,15 +48,4 @@ bundle_test() { } 2>&1 | tee $DEST/test.log } - -# This helper function walks the current directory looking for directories -# holding Go test files, and prints their paths on standard output, one per -# line. -find_test_dirs() { - find -not \( \ - \( -wholename './vendor' -o -wholename './integration' \) \ - -prune \ - \) -name '*_test.go' -print0 | xargs -0n1 dirname | sort -u -} - bundle_test diff --git a/hack/make/test-integration b/hack/make/test-integration index f1ab0b99c3..0af4c23c48 100644 --- a/hack/make/test-integration +++ b/hack/make/test-integration @@ -5,7 +5,12 @@ DEST=$1 set -e bundle_test_integration() { - LDFLAGS="$LDFLAGS $LDFLAGS_STATIC" go_test_dir ./integration + LDFLAGS="$LDFLAGS $LDFLAGS_STATIC_DOCKER" go_test_dir ./integration \ + "-coverpkg $(find_dirs '*.go' | sed 's,^\.,github.com/dotcloud/docker,g' | paste -d, -s)" } -bundle_test_integration 2>&1 | tee $DEST/test.log +# this "grep" hides some really irritating warnings that "go test -coverpkg" +# spews when it is given packages that aren't used +bundle_test_integration 2>&1 \ + | grep --line-buffered -v '^warning: no packages being tested depend on ' \ + | tee $DEST/test.log diff --git a/hack/make/ubuntu b/hack/make/ubuntu index 23af0ff7e0..ebc12f27ec 100644 --- a/hack/make/ubuntu +++ b/hack/make/ubuntu @@ -29,42 +29,36 @@ bundle_ubuntu() { cp contrib/udev/80-docker.rules $DIR/etc/udev/rules.d/ # Include our init scripts - mkdir -p $DIR/etc - cp -R contrib/init/upstart $DIR/etc/init - cp -R contrib/init/sysvinit $DIR/etc/init.d - mkdir -p $DIR/lib/systemd - cp -R contrib/init/systemd $DIR/lib/systemd/system - + mkdir -p $DIR/etc/init + cp contrib/init/upstart/docker.conf $DIR/etc/init/ + mkdir -p $DIR/etc/init.d + cp contrib/init/sysvinit-debian/docker $DIR/etc/init.d/ mkdir -p $DIR/etc/default - cat > $DIR/etc/default/docker <<'EOF' -# Docker Upstart and SysVinit configuration file - -# Customize location of Docker binary (especially for development testing). -#DOCKER="/usr/local/bin/docker" - -# Use DOCKER_OPTS to modify the daemon startup options. -#DOCKER_OPTS="-dns 8.8.8.8" - -# If you need Docker to use an HTTP proxy, it can also be specified here. -#export http_proxy=http://127.0.0.1:3128/ -EOF + cp contrib/init/sysvinit-debian/docker.default $DIR/etc/default/docker + mkdir -p $DIR/lib/systemd/system + cp contrib/init/systemd/docker.service $DIR/lib/systemd/system/ # Copy the binary # This will fail if the binary bundle hasn't been built mkdir -p $DIR/usr/bin - # Copy the binary - # This will fail if the binary bundle hasn't been built cp $DEST/../binary/docker-$VERSION $DIR/usr/bin/docker # Generate postinst/prerm/postrm scripts - cat > /tmp/postinst <<'EOF' + cat > $DEST/postinst <<'EOF' #!/bin/sh set -e set -u -getent group docker > /dev/null || groupadd --system docker || true +if [ "$1" = 'configure' ] && [ -z "$2" ]; then + if ! getent group docker > /dev/null; then + groupadd --system docker + fi +fi -update-rc.d docker defaults > /dev/null || true +if ! { [ -x /sbin/initctl ] && /sbin/initctl version 2>/dev/null | grep -q upstart; }; then + # we only need to do this if upstart isn't in charge + update-rc.d docker defaults > /dev/null || true +fi if [ -n "$2" ]; then _dh_action=restart else @@ -74,7 +68,7 @@ service docker $_dh_action 2>/dev/null || true #DEBHELPER# EOF - cat > /tmp/prerm <<'EOF' + cat > $DEST/prerm <<'EOF' #!/bin/sh set -e set -u @@ -83,7 +77,7 @@ service docker stop 2>/dev/null || true #DEBHELPER# EOF - cat > /tmp/postrm <<'EOF' + cat > $DEST/postrm <<'EOF' #!/bin/sh set -e set -u @@ -101,50 +95,61 @@ fi #DEBHELPER# EOF # TODO swaths of these were borrowed from debhelper's auto-inserted stuff, because we're still using fpm - we need to use debhelper instead, and somehow reconcile Ubuntu that way - chmod +x /tmp/postinst /tmp/prerm + chmod +x $DEST/postinst $DEST/prerm $DEST/postrm ( + # switch directories so we create *.deb in the right folder cd $DEST + + # create lxc-docker-VERSION package fpm -s dir -C $DIR \ - --name lxc-docker-$VERSION --version $PKGVERSION \ - --after-install /tmp/postinst \ - --before-remove /tmp/prerm \ - --after-remove /tmp/postrm \ - --architecture "$PACKAGE_ARCHITECTURE" \ - --prefix / \ - --depends lxc \ - --depends aufs-tools \ - --depends iptables \ - --deb-recommends ca-certificates \ - --deb-recommends xz-utils \ - --description "$PACKAGE_DESCRIPTION" \ - --maintainer "$PACKAGE_MAINTAINER" \ - --conflicts docker \ - --conflicts docker.io \ - --conflicts lxc-docker-virtual-package \ - --provides lxc-docker \ - --provides lxc-docker-virtual-package \ - --replaces lxc-docker \ - --replaces lxc-docker-virtual-package \ - --url "$PACKAGE_URL" \ - --license "$PACKAGE_LICENSE" \ - --config-files /etc/udev/rules.d/80-docker.rules \ - --config-files /etc/init/docker.conf \ - --config-files /etc/init.d/docker \ - --config-files /etc/default/docker \ - --deb-compression gz \ - -t deb . + --name lxc-docker-$VERSION --version $PKGVERSION \ + --after-install $DEST/postinst \ + --before-remove $DEST/prerm \ + --after-remove $DEST/postrm \ + --architecture "$PACKAGE_ARCHITECTURE" \ + --prefix / \ + --depends iptables \ + --deb-recommends aufs-tools \ + --deb-recommends ca-certificates \ + --deb-recommends git \ + --deb-recommends xz-utils \ + --deb-suggests cgroup-lite \ + --description "$PACKAGE_DESCRIPTION" \ + --maintainer "$PACKAGE_MAINTAINER" \ + --conflicts docker \ + --conflicts docker.io \ + --conflicts lxc-docker-virtual-package \ + --provides lxc-docker \ + --provides lxc-docker-virtual-package \ + --replaces lxc-docker \ + --replaces lxc-docker-virtual-package \ + --url "$PACKAGE_URL" \ + --license "$PACKAGE_LICENSE" \ + --config-files /etc/udev/rules.d/80-docker.rules \ + --config-files /etc/init/docker.conf \ + --config-files /etc/init.d/docker \ + --config-files /etc/default/docker \ + --deb-compression gz \ + -t deb . + # TODO replace "Suggests: cgroup-lite" with "Recommends: cgroupfs-mount | cgroup-lite" once cgroupfs-mount is available + + # create empty lxc-docker wrapper package fpm -s empty \ - --name lxc-docker --version $PKGVERSION \ - --architecture "$PACKAGE_ARCHITECTURE" \ - --depends lxc-docker-$VERSION \ - --description "$PACKAGE_DESCRIPTION" \ - --maintainer "$PACKAGE_MAINTAINER" \ - --url "$PACKAGE_URL" \ - --license "$PACKAGE_LICENSE" \ - --deb-compression gz \ - -t deb + --name lxc-docker --version $PKGVERSION \ + --architecture "$PACKAGE_ARCHITECTURE" \ + --depends lxc-docker-$VERSION \ + --description "$PACKAGE_DESCRIPTION" \ + --maintainer "$PACKAGE_MAINTAINER" \ + --url "$PACKAGE_URL" \ + --license "$PACKAGE_LICENSE" \ + --deb-compression gz \ + -t deb ) + + # clean up after ourselves so we have a clean output directory + rm $DEST/postinst $DEST/prerm $DEST/postrm + rm -r $DIR } bundle_ubuntu |