diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/README.md | 12 | ||||
| -rw-r--r-- | doc/api/merge_requests.md | 54 | ||||
| -rw-r--r-- | doc/ci/README.md | 12 | ||||
| -rw-r--r-- | doc/ci/docker/using_docker_images.md | 285 | ||||
| -rw-r--r-- | doc/ci/languages/README.md | 7 | ||||
| -rw-r--r-- | doc/ci/languages/php.md | 284 | ||||
| -rw-r--r-- | doc/ci/services/README.md | 9 | ||||
| -rw-r--r-- | doc/ci/services/docker-services.md | 5 | ||||
| -rw-r--r-- | doc/ci/services/mysql.md | 118 | ||||
| -rw-r--r-- | doc/ci/services/postgres.md | 114 | ||||
| -rw-r--r-- | doc/ci/services/redis.md | 69 | ||||
| -rw-r--r-- | doc/ci/ssh_keys/README.md | 109 | ||||
| -rw-r--r-- | doc/update/patch_versions.md | 2 | ||||
| -rw-r--r-- | doc/workflow/README.md | 1 | ||||
| -rw-r--r-- | doc/workflow/merge_when_build_succeeds.md | 15 | ||||
| -rw-r--r-- | doc/workflow/merge_when_build_succeeds/enable.png | bin | 0 -> 151112 bytes | |||
| -rw-r--r-- | doc/workflow/merge_when_build_succeeds/status.png | bin | 0 -> 180318 bytes |
17 files changed, 978 insertions, 118 deletions
diff --git a/doc/README.md b/doc/README.md index 58ab5dd08e0..a3098094210 100644 --- a/doc/README.md +++ b/doc/README.md @@ -24,9 +24,21 @@ - [Using Docker Images](ci/docker/using_docker_images.md) - [Using Docker Build](ci/docker/using_docker_build.md) - [Using Variables](ci/variables/README.md) +- [Using SSH keys](ci/ssh_keys/README.md) - [User permissions](ci/permissions/README.md) - [API](ci/api/README.md) +### CI Languages + ++ [Testing PHP](ci/languages/php.md) + +### CI Services + ++ [Using MySQL](ci/services/mysql.md) ++ [Using PostgreSQL](ci/services/postgres.md) ++ [Using Redis](ci/services/redis.md) ++ [Using Other Services](ci/docker/using_docker_images.md#how-to-use-other-images-as-services) + ### CI Examples - [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md) diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 82f2cef969f..366a1f8abec 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -335,9 +335,57 @@ PUT /projects/:id/merge_request/:merge_request_id/merge Parameters: -- `id` (required) - The ID of a project -- `merge_request_id` (required) - ID of MR -- `merge_commit_message` (optional) - Custom merge commit message +- `id` (required) - The ID of a project +- `merge_request_id` (required) - ID of MR +- `merge_commit_message` (optional) - Custom merge commit message +- `should_remove_source_branch` (optional) - if `true` removes the source branch +- `merged_when_build_succeeds` (optional) - if `true` the MR is merge when the build succeeds + +```json +{ + "id": 1, + "target_branch": "master", + "source_branch": "test1", + "project_id": 3, + "title": "test1", + "state": "merged", + "upvotes": 0, + "downvotes": 0, + "author": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + }, + "assignee": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + } +} +``` + +## Cancel Merge When Build Succeeds + +If successful you'll get `200 OK`. + +If you don't have permissions to accept this merge request - you'll get a 401 + +If the merge request is already merged or closed - you get 405 and error message 'Method Not Allowed' + +In case the merge request is not set to be merged when the build succeeds, you'll also get a 406 error. +``` +PUT /projects/:id/merge_request/:merge_request_id/cancel_merge_when_build_succeeds +``` +Parameters: + +- `id` (required) - The ID of a project +- `merge_request_id` (required) - ID of MR ```json { diff --git a/doc/ci/README.md b/doc/ci/README.md index 97325069ceb..5d9d7a81db3 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -9,6 +9,18 @@ + [Using Docker Images](docker/using_docker_images.md) + [Using Docker Build](docker/using_docker_build.md) + [Using Variables](variables/README.md) ++ [Using SSH keys](ssh_keys/README.md) + +### Languages + ++ [Testing PHP](languages/php.md) + +### Services + ++ [Using MySQL](services/mysql.md) ++ [Using PostgreSQL](services/postgres.md) ++ [Using Redis](services/redis.md) ++ [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services) ### Examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 1feae62b1c7..8d4bd44053e 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,19 +1,29 @@ # Using Docker Images -GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. -Docker is an open-source project that allows to use predefined images to run applications -in independent "containers" that are run within a single Linux instance. -[Docker Hub](https://registry.hub.docker.com/) have rich database of built images that can be used to build applications. +GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use +[Docker Engine](https://www.docker.com/) to test and build any application. -Docker when used with GitLab CI runs each build in separate and isolated container using predefined image and always from scratch. -It makes it easier to have simple and reproducible build environment that can also be run on your workstation. -This allows you to test all commands from your shell, rather than having to test them on a CI server. +Docker is an open-source project that allows you to use predefined images to +run applications in independent "containers" that are run within a single Linux +instance. [Docker Hub][hub] has a rich database of prebuilt images that can be +used to test and build your applications. -### Register Docker runner -To use GitLab Runner with Docker you need to register new runner to use `docker` executor: +Docker, when used with GitLab CI, runs each build in a separate and isolated +container using the predefined image that is set up in +[`.gitlab-ci.yml`](../yaml/README.md). + +This makes it easier to have a simple and reproducible build environment that +can also run on your workstation. The added benefit is that you can test all +the commands that we will explore later from your shell, rather than having to +test them on a dedicated CI server. + +## Register docker runner + +To use GitLab Runner with docker you need to register a new runner to use the +`docker` executor: ```bash -gitlab-ci-multi-runner register \ +gitlab-runner register \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ @@ -23,101 +33,79 @@ gitlab-ci-multi-runner register \ --docker-mysql latest ``` -**The registered runner will use `ruby:2.1` image and will run two services (`postgres:latest` and `mysql:latest`) that will be accessible for time of the build.** +The registered runner will use the `ruby:2.1` docker image and will run two +services, `postgres:latest` and `mysql:latest`, both of which will be +accessible during the build process. -### What is image? -The image is the name of any repository that is present in local Docker Engine or any repository that can be found at [Docker Hub](https://registry.hub.docker.com/). -For more information about the image and Docker Hub please read the [Docker Fundamentals](https://docs.docker.com/introduction/understanding-docker/). +## What is image -### What is service? -Service is just another image that is run for time of your build and is linked to your build. This allows you to access the service image during build time. -The service image can run any application, but most common use case is to run some database container, ie.: `mysql`. -It's easier and faster to use existing image, run it as additional container than install `mysql` every time project is built. +The `image` keyword is the name of the docker image that is present in the +local Docker Engine (list all images with `docker images`) or any image that +can be found at [Docker Hub][hub]. For more information about images and Docker +Hub please read the [Docker Fundamentals][] documentation. -#### How is service linked to the build? -There's good document that describes how Docker linking works: [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). -To summarize: if you add `mysql` as service to your application, the image will be used to create container that is linked to build container. -The service container for MySQL will be accessible under hostname `mysql`. -So, **to access your database service you have to connect to host: `mysql` instead of socket or `localhost`**. +In short, with `image` we refer to the docker image, which will be used to +create a container on which your build will run. -### How to use other images as services? -You are not limited to have only database services. -You can hand modify `config.toml` to add any image as service found at [Docker Hub](https://registry.hub.docker.com/). -Look for `[runners.docker]` section: -``` -[runners.docker] - image = "ruby:2.1" - services = ["mysql:latest", "postgres:latest"] -``` +## What is service -For example you need `wordpress` instance to test some API integration with `Wordpress`. -You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). -This is image that have fully preconfigured `wordpress` and have `MySQL` server built-in: -``` -[runners.docker] - image = "ruby:2.1" - services = ["mysql:latest", "postgres:latest", "tutum/wordpress:latest"] -``` +The `services` keyword defines just another docker image that is run during +your build and is linked to the docker image that the `image` keyword defines. +This allows you to access the service image during build time. -Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum__wordpress`. +The service image can run any application, but the most common use case is to +run a database container, eg. `mysql`. It's easier and faster to use an +existing image and run it as an additional container than install `mysql` every +time the project is built. -Alias hostname for the service is made from the image name: -1. Everything after `:` is stripped, -2. '/' is replaced with `__`. +You can see some widely used services examples in the relevant documentation of +[CI services examples](../services/README.md). -### Configuring services -Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. +### How is service linked to the build -GitLab Runner 0.5.0 and up passes all YAML-defined variables to created service containers. +To better understand how the container linking works, read +[Linking containers together](https://docs.docker.com/userguide/dockerlinks/). -1. To configure database name for [postgres](https://registry.hub.docker.com/u/library/postgres/) service, -you need to set POSTGRES_DB. +To summarize, if you add `mysql` as service to your application, the image will +then be used to create a container that is linked to the build container. - ```yaml - services: - - postgres - - variables: - POSTGRES_DB: gitlab - ``` +The service container for MySQL will be accessible under the hostname `mysql`. +So, in order to access your database service you have to connect to the host +named `mysql` instead of a socket or `localhost`. -1. To use [mysql](https://registry.hub.docker.com/u/library/mysql/) service with empty password for time of build, -you need to set MYSQL_ALLOW_EMPTY_PASSWORD. +## Overwrite image and services - ```yaml - services: - - mysql - - variables: - MYSQL_ALLOW_EMPTY_PASSWORD: "yes" - ``` +See [How to use other images as services](#how-to-use-other-images-as-services). -For other possible configuration variables check the -https://registry.hub.docker.com/u/library/mysql/ or https://registry.hub.docker.com/u/library/postgres/ -or README page for any other Docker image. +## How to use other images as services -**Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** +You are not limited to have only database services. You can add as many +services you need to `.gitlab-ci.yml` or manually modify `config.toml`. +Any image found at [Docker Hub][hub] can be used as a service. -### Overwrite image and services -It's possible to overwrite `docker-image` and specify services from `.gitlab-ci.yml`. -If you add to your YAML the `image` and the `services` these parameters -be used instead of the ones that were specified during runner's registration. -``` +## Define image and services from `.gitlab-ci.yml` + +You can simply define an image that will be used for all jobs and a list of +services that you want to use during build time. + +```yaml image: ruby:2.2 + services: - postgres:9.3 -before_install: + +before_script: - bundle install - + test: script: - bundle exec rake spec ``` -It's possible to define image and service per-job: -``` -before_install: +It is also possible to define different images and services per job: + +```yaml +before_script: - bundle install test:2.1: @@ -135,34 +123,73 @@ test:2.2: - bundle exec rake spec ``` -#### How to enable overwriting? -To enable overwriting you have to **enable it first** (it's disabled by default for security reasons). -You can do that by hand modifying runner configuration: `config.toml`. -Please go to section where is `[runners.docker]` definition for your runner. -Add `allowed_images` and `allowed_services` to specify what images are allowed to be picked from `.gitlab-ci.yml`: +## Define image and services in `config.toml` + +Look for the `[runners.docker]` section: + ``` [runners.docker] image = "ruby:2.1" - allowed_images = ["ruby:*", "python:*"] - allowed_services = ["mysql:*", "redis:*"] + services = ["mysql:latest", "postgres:latest"] ``` -This enables you to use in your `.gitlab-ci.yml` any image that matches above wildcards. -You will be able to pick only `ruby` and `python` images. -The same rule can be applied to limit services. -If you are courageous enough, you can make it fully open and accept everything: -``` -[runners.docker] - image = "ruby:2.1" - allowed_images = ["*", "*/*"] - allowed_services = ["*", "*/*"] +The image and services defined this way will be added to all builds run by +that runner. + +## Accessing the services + +Let's say that you need a Wordpress instance to test some API integration with +your application. + +You can then use for example the [tutum/wordpress][] image in your +`.gitlab-ci.yml`: + +```yaml +services: +- tutum/wordpress:latest ``` -**It the feature is not enabled, or image isn't allowed the error message will be put into the build log.** +When the build is run, `tutum/wordpress` will be started and you will have +access to it from your build container under the hostname `tutum__wordpress`. + +The alias hostname for the service is made from the image name following these +rules: + +1. Everything after `:` is stripped +2. Backslash (`/`) is replaced with double underscores (`__`) + +## Configuring services + +Many services accept environment variables which allow you to easily change +database names or set account names depending on the environment. + +GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created +service containers. + +For all possible configuration variables check the documentation of each image +provided in their corresponding Docker hub page. + +*Note: All variables will be passed to all services containers. It's not +designed to distinguish which variable should go where.* + +### PostgreSQL service example + +See the specific documentation for +[using PostgreSQL as a service](../services/postgres.md). + +### MySQL service example + +See the specific documentation for +[using MySQL as a service](../services/mysql.md). + +## How Docker integration works + +Below is a high level overview of the steps performed by docker during build +time. -### How Docker integration works 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. -1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). +1. Create cache container to store all volumes as defined in `config.toml` and + `Dockerfile` of build image (`ruby:2.1` as in above example). 1. Create build container and link any service container to build container. 1. Start build container and send build script to the container. 1. Run build script. @@ -171,33 +198,63 @@ If you are courageous enough, you can make it fully open and accept everything: 1. Check exit status of build script. 1. Remove build container and all created service containers. -### How to debug a build locally -1. Create a file with build script: +## How to debug a build locally + +*Note: The following commands are run without root privileges. You should be +able to run docker with your regular user account.* + +First start with creating a file named `build script`: + ```bash -$ cat <<EOF > build_script +cat <<EOF > build_script git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner cd /builds/gitlab-org/gitlab-ci-multi-runner -make <- or any other build step +make EOF ``` -1. Create service containers: +Here we use as an example the GitLab Runner repository which contains a +Makefile, so running `make` will execute the commands defined in the Makefile. +Your mileage may vary, so instead of `make` you could run the command which +is specific to your project. + +Then create some service containers: + ``` -$ docker run -d -n service-mysql mysql:latest -$ docker run -d -n service-postgres postgres:latest +docker run -d -n service-mysql mysql:latest +docker run -d -n service-postgres postgres:latest ``` -This will create two service containers (MySQL and PostgreSQL). -1. Create a build container and execute script in its context: +This will create two service containers, named `service-mysql` and +`service-postgres` which use the latest MySQL and PostgreSQL images +respectively. They will both run in the background (`-d`). + +Finally, create a build container by executing the `build_script` file we +created earlier: + ``` -$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script +docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` -This will create build container that has two service containers linked. -The build_script is piped using STDIN to bash interpreter which executes the build script in container. -1. At the end remove all containers: +The above command will create a container named `build` that is spawned from +the `ruby:2.1` image and has two services linked to it. The `build_script` is +piped using STDIN to the bash interpreter which in turn executes the +`build_script` in the `build` container. + +When you finish testing and no longer need the containers, you can remove them +with: + ``` docker rm -f -v build service-mysql service-postgres ``` -This will forcefully (the `-f` switch) remove build container and service containers -and all volumes (the `-v` switch) that were created with the container creation. + +This will forcefully (`-f`) remove the `build` container, the two service +containers as well as all volumes (`-v`) that were created with the container +creation. + +[Docker Fundamentals]: https://docs.docker.com/engine/introduction/understanding-docker/ +[hub]: https://hub.docker.com/ +[linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/ +[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ +[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ +[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ diff --git a/doc/ci/languages/README.md b/doc/ci/languages/README.md new file mode 100644 index 00000000000..54b2343e08b --- /dev/null +++ b/doc/ci/languages/README.md @@ -0,0 +1,7 @@ +### Languages + +This is a list of languages you can test with GitLab CI. Each section has +comprehensive documentation and comes with a test repository hosted on +GitLab.com + ++ [Testing PHP](php.md) diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md new file mode 100644 index 00000000000..dacb67fa3ff --- /dev/null +++ b/doc/ci/languages/php.md @@ -0,0 +1,284 @@ +# Testing PHP projects + +This guide covers basic building instructions for PHP projects. + +There are covered two cases: testing using the Docker executor and testing +using the Shell executor. + +## Test PHP projects using the Docker executor + +While it is possible to test PHP apps on any system, this would require manual +configuration from the developer. To overcome this we will be using the +official [PHP docker image][php-hub] that can be found in Docker Hub. + +This will allow us to test PHP projects against different versions of PHP. +However, not everything is plug 'n' play, you still need to onfigure some +things manually. + +As with every build, you need to create a valid `.gitlab-ci.yml` describing the +build environment. + +Let's first specify the PHP image that will be used for the build process +(you can read more about what an image means in the Runner's lingo reading +about [Using Docker images](../docker/using_docker_images.md#what-is-image)). + +Start by adding the image to your `.gitlab-ci.yml`: + +```yaml +image: php:5.6 +``` + +The official images are great, but they lack a few useful tools for testing. +We need to first prepare the build environment. A way to overcome this is to +create a script which installs all prerequisites prior the actual testing is +done. + +Let's create a `ci/docker_install.sh` file in the root directory of our +repository with the following content: + +```bash +#!/bin/bash + +# We need to install dependencies only for Docker +[[ ! -e /.dockerinit ]] && exit 0 + +set -xe + +# Install git (the php image doesn't have it) which is required by composer +apt-get update -yqq +apt-get install git -yqq + +# Install phpunit, the tool that we will use for testing +curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar +chmod +x /usr/local/bin/phpunit + +# Install mysql driver +# Here you can install any other extension that you need +docker-php-ext-install pdo_mysql +``` + +You might wonder what `docker-php-ext-install` is. In short, it is a script +provided by the official php docker image that you can use to easilly install +extensions. For more information read the the documentation at +<https://hub.docker.com/_/php/>. + +Now that we created the script that contains all prerequisites for our build +environment, let's add it in `.gitlab-ci.yml`: + +```yaml +... + +before_script: +- bash ci/docker_install.sh > /dev/null + +... +``` + +Last step, run the actual tests using `phpunit`: + +```yaml +... + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml + +... +``` + +Finally, commit your files and push them to GitLab to see your build succeeding +(or failing). + +The final `.gitlab-ci.yml` should look similar to this: + +```yaml +# Select image from https://hub.docker.com/_/php/ +image: php:5.6 + +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Test against different PHP versions in Docker builds + +Testing against multiple versions of PHP is super easy. Just add another job +with a different docker image version and the runner will do the rest: + +```yaml +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +# We test PHP5.6 +test:5.6: + image: php:5.6 + script: + - phpunit --configuration phpunit_myapp.xml + +# We test PHP7.0 (good luck with that) +test:7.0: + image: php:7.0 + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Custom PHP configuration in Docker builds + +There are times where you will need to customise your PHP environment by +putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose +add a `before_script` action: + +```yaml +before_script: +- cp my_php.ini /usr/local/etc/php/conf.d/test.ini +``` + +Of course, `my_php.ini` must be present in the root directory of your repository. + +## Test PHP projects using the Shell executor + +The shell executor runs your builds in a terminal session on your server. +Thus, in order to test your projects you first need to make sure that all +dependencies are installed. + +For example, in a VM running Debian 8 we first update the cache, then we +install `phpunit` and `php5-mysql`: + +```bash +sudo apt-get update -y +sudo apt-get install -y phpunit php5-mysql +``` + +Next, add the following snippet to your `.gitlab-ci.yml`: + +```yaml +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` + +Finally, push to GitLab and let the tests begin! + +### Test against different PHP versions in Shell builds + +The [phpenv][] project allows you to easily manage different versions of PHP +each with its own config. This is specially usefull when testing PHP projects +with the Shell executor. + +You will have to install it on your build machine under the `gitlab-runner` +user following [the upstream installation guide][phpenv-installation]. + +Using phpenv also allows to easily configure the PHP environment with: + +``` +phpenv config-add my_config.ini +``` + +*__Important note:__ It seems `phpenv/phpenv` + [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork + at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring + the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also + seems like a good alternative. Picking any of the mentioned tools will work + with the basic phpenv commands. Guiding you to choose the right phpenv is out + of the scope of this tutorial.* + +### Install custom extensions + +Since this is a pretty bare installation of the PHP environment, you may need +some extensions that are not currently present on the build machine. + +To install additional extensions simply execute: + +```bash +pecl install <extension> +``` + +It's not advised to add this to `.gitlab-ci.yml`. You should execute this +command once, only to setup the build environment. + +## Extend your tests + +### Using atoum + +Instead of PHPUnit, you can use any other tool to run unit tests. For example +you can use [atoum](https://github.com/atoum/atoum): + +```yaml +before_script: +- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar + +test:atoum: + script: + - php mageekguy.atoum.phar +``` + +### Using Composer + +The majority of the PHP projects use Composer for managing their PHP packages. +In order to execute Composer before running your tests, simply add the +following in your `.gitlab-ci.yml`: + +```yaml +... + +# Composer stores all downloaded packages in the vendor/ directory. +# Do not use the following if the vendor/ directory is commited to +# your git repository. +cache: + paths: + - vendor/ + +before_script: +# Install composer dependencies +- curl -sS https://getcomposer.org/installer | php +- php composer.phar install + +... +``` + +## Access private packages / dependencies + +If your test suite needs to access a private repository, you need to configure +[the SSH keys](../ssh_keys/README.md) in order to be able to clone it. + +## Use databases or other services + +Most of the time you will need a running database in order for your tests to +run. If you are using the Docker executor you can leverage Docker's ability to +link to other containers. In GitLab Runner lingo, this can be achieved by +defining a `service`. + +This functionality is covered in [the CI services](../services/README.md) +documentation. + +## Testing things locally + +With GitLab Runner 1.0 you can also test any changes locally. From your +terminal execute: + +```bash +# Check using docker executor +gitlab-runner exec docker test:app + +# Check using shell executor +gitlab-runner exec shell test:app +``` + +## Example project + +We have set up an [Example PHP Project][php-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[php-hub]: https://hub.docker.com/_/php/ +[phpenv]: https://github.com/phpenv/phpenv +[phpenv-installation]: https://github.com/phpenv/phpenv#installation +[php-example-repo]: https://gitlab.com/gitlab-examples/php diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md new file mode 100644 index 00000000000..1ebb0a4a250 --- /dev/null +++ b/doc/ci/services/README.md @@ -0,0 +1,9 @@ +## GitLab CI Services + +GitLab CI uses the `services` keyword to define what docker containers should be +linked with your base image. Below is a list of examples you may use. + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) ++ [Using Other Services](../docker/using_docker_images.md#how-to-use-other-images-as-services) diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md new file mode 100644 index 00000000000..df36ebaf7d4 --- /dev/null +++ b/doc/ci/services/docker-services.md @@ -0,0 +1,5 @@ +## GitLab CI Services + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md new file mode 100644 index 00000000000..c66d77122b2 --- /dev/null +++ b/doc/ci/services/mysql.md @@ -0,0 +1,118 @@ +# Using MySQL + +As many applications depend on MySQL as their database, you will eventually +need it in order for your tests to run. Below you are guided how to do this +with the Docker and Shell executors of GitLab Runner. + +## Use MySQL with the Docker executor + +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. + +First, in your `.gitlab-ci.yml` add: + +```yaml +services: + - mysql:latest + +variables: + # Configure mysql environment variables (https://hub.docker.com/_/mysql/) + MYSQL_DATABASE: el_duderino + MYSQL_ROOT_PASSWORD: mysql_strong_password +``` + +And then configure your application to use the database, for example: + +```yaml +Host: mysql +User: root +Password: mysql_strong_password +Database: el_duderino +``` + +If you are wondering why we used `mysql` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + +You can also use any other docker image available on [Docker Hub][hub-mysql]. +For example, to use MySQL 5.5 the service becomes `mysql:5.5`. + +The `mysql` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-mysql]. + +## Use MySQL with the Shell executor + +You can also use MySQL on manually configured servers that are using +GitLab Runner with the Shell executor. + +First install the MySQL server: + +```bash +sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev +``` + +Pick a MySQL root password (can be anything), and type it twice when asked. + +*Note: As a security measure you can run `mysql_secure_installation` to +remove anonymous users, drop the test database and disable remote logins with +the root user.* + +The next step is to create a user, so login to MySQL as root: + +```bash +mysql -u root -p +``` + +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. + +*Note: Do not type `mysql>`, this is part of the MySQL prompt.* + +```bash +mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; +``` + +Create the database: + +```bash +mysql> CREATE DATABASE IF NOT EXISTS `el_duderino` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; +``` + +Grant the necessary permissions on the database: + +```bash +mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `el_duderino`.* TO 'runner'@'localhost'; +``` + +If all went well you can now quit the database session: + +```bash +mysql> \q +``` + +Now, try to connect to the newly created database to check that everything is +in place: + +```bash +mysql -u runner -p -D el_duderino +``` + +As a final step, configure your application to use the database, for example: + +```bash +Host: localhost +User: runner +Password: $password +Database: el_duderino +``` + +## Example project + +We have set up an [Example MySQL Project][mysql-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-mysql]: https://hub.docker.com/_/mysql/ +[mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md new file mode 100644 index 00000000000..17d21dbda1c --- /dev/null +++ b/doc/ci/services/postgres.md @@ -0,0 +1,114 @@ +# Using PostgreSQL + +As many applications depend on PostgreSQL as their database, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. + +## Use PostgreSQL with the Docker executor + +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. + +First, in your `.gitlab-ci.yml` add: + +```yaml +services: + - postgres:latest + +variables: + POSTGRES_DB: nice_marmot + POSTGRES_USER: runner + POSTGRES_PASSWORD: "" +``` + +And then configure your application to use the database, for example: + +```yaml +Host: postgres +User: runner +Password: +Database: nice_marmot +``` + +If you are wondering why we used `postgres` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + +You can also use any other docker image available on [Docker Hub][hub-pg]. +For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`. + +The `postgres` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-pg]. + +## Use PostgreSQL with the Shell executor + +You can also use PostgreSQL on manually configured servers that are using +GitLab Runner with the Shell executor. + +First install the PostgreSQL server: + +```bash +sudo apt-get install -y postgresql postgresql-client libpq-dev +``` + +The next step is to create a user, so login to PostgreSQL: + +```bash +sudo -u postgres psql -d template1 +``` + +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. + +*__Note:__ Do not type `template1=#`, this is part of the PostgreSQL prompt.* + +```bash +template1=# CREATE USER runner WITH PASSWORD '$password' CREATEDB; +``` + +*__Note:__ Notice that we created the user with the privilege to be able to +create databases (`CREATEDB`). In the following steps we will create a database +explicitly for that user but having that privilege can be useful if in your +testing framework you have tools that drop and create databases.* + +Create the database and grant all privileges on it for the user `runner`: + +```bash +template1=# CREATE DATABASE nice_marmot OWNER runner; +``` + +If all went well you can now quit the database session: + +```bash +template1=# \q +``` + +Now, try to connect to the newly created database with the user `runner` to +check that everything is in place. + +```bash +psql -U runner -h localhost -d nice_marmot -W +``` + +*__Note:__ We are explicitly telling `psql` to connect to localhost in order +to use the md5 authentication. If you omit this step you will be denied access.* + +Finally, configure your application to use the database, for example: + +```yaml +Host: localhost +User: runner +Password: $password +Database: nice_marmot +``` + +## Example project + +We have set up an [Example PostgreSQL Project][postgres-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-pg]: https://hub.docker.com/_/postgres/ +[postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md new file mode 100644 index 00000000000..b281e8f9f60 --- /dev/null +++ b/doc/ci/services/redis.md @@ -0,0 +1,69 @@ +# Using Redis + +As many applications depend on Redis as their key-value store, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. + +## Use Redis with the Docker executor + +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. + +First, in your `.gitlab-ci.yml` add: + +```yaml +services: + - redis:latest +``` + +Then you need to configure your application to use the Redis database, for +example: + +```yaml +Host: redis +``` + +And that's it. Redis will now be available to be used within your testing +framework. + +You can also use any other docker image available on [Docker Hub][hub-redis]. +For example, to use Redis 2.8 the service becomes `redis:2.8`. + +## Use Redis with the Shell executor + +Redis can also be used on manually configured servers that are using GitLab +Runner with the Shell executor. + +In your build machine install the Redis server: + +```bash +sudo apt-get install redis-server +``` + +Verify that you can connect to the server with the `gitlab-runner` user: + +```bash +# Try connecting the the Redis server +sudo -u gitlab-runner -H redis-cli + +# Quit the session +127.0.0.1:6379> quit +``` + +Finally, configure your application to use the database, for example: + +```yaml +Host: localhost +``` + +## Example project + +We have set up an [Example Redis Project][redis-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-redis]: https://hub.docker.com/_/redis/ +[redis-example-repo]: https://gitlab.com/gitlab-examples/redis diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md new file mode 100644 index 00000000000..210f9c3e849 --- /dev/null +++ b/doc/ci/ssh_keys/README.md @@ -0,0 +1,109 @@ +# Using SSH keys + +GitLab currently doesn't have built-in support for managing SSH keys in a build +environment. + +The SSH keys can be useful when: + +1. You want to checkout internal submodules +2. You want to download private packages using your package manager (eg. bundler) +3. You want to deploy your application to eg. Heroku or your own server +4. You want to execute SSH commands from the build server to the remote server +5. You want to rsync files from your build server to the remote server + +If anything of the above rings a bell, then you most likely need an SSH key. + +## Inject keys in your build server + +The most widely supported method is to inject an SSH key into your build +environment by extending your `.gitlab-ci.yml`. + +This is the universal solution which works with any type of executor +(docker, shell, etc.). + +### How it works + +1. Create a new SSH key pair with [ssh-keygen][] +2. Add the private key as a **Secret Variable** to the project +3. Run the [ssh-agent][] during build to load the private key. + +## SSH keys when using the Docker executor + +You will first need to create an SSH key pair. For more information, follow the +instructions to [generate an SSH key](../ssh/README.md). + +Then, create a new **Secret Variable** in your project settings on GitLab +following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` +and in the **Value** field paste the content of your _private_ key that you +created earlier. + +Next you need to modify your `.gitlab-ci.yml` with a `before_script` action. +Add it to the top: + +``` +before_script: + # Install ssh-agent if not already installed, it is required by Docker. + # (change apt-get to yum if you use a CentOS-based image) + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + + # Run ssh-agent (inside the build environment) + - eval $(ssh-agent -s) + + # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + - ssh-add <(echo "$SSH_PRIVATE_KEY") + + # For Docker builds disable host key checking. Be aware that by adding that + # you are suspectible to man-in-the-middle attacks. + # WARNING: Use this only with the Docker executor, if you use it with shell + # you will overwrite your user's SSH config. + - mkdir -p ~/.ssh + - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` +``` + +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). + +That's it! You can now have access to private servers or repositories in your +build environment. + +## SSH keys when using the Shell executor + +If you are using the Shell executor and not Docker, it is easier to set up an +SSH key. + +You can generate the SSH key from the machine that GitLab Runner is installed +on, and use that key for all projects that are run on this machine. + +First, you need to login to the server that runs your builds. + +Then from the terminal login as the `gitlab-runner` user and generate the SSH +key pair as described in the [SSH keys documentation](../ssh/README.md). + +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). + +Once done, try to login to the remote server in order to accept the fingerprint: + +```bash +ssh <address-of-my-server> +``` + +For accessing repositories on GitLab.com, the `<address-of-my-server>` would be +`git@gitlab.com`. + +## Example project + +We have set up an [Example SSH Project][ssh-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[ssh-keygen]: http://linux.die.net/man/1/ssh-keygen +[ssh-agent]: http://linux.die.net/man/1/ssh-agent +[ssh-example-repo]: https://gitlab.com/gitlab-examples/ssh-private-key/ diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 957354decb7..c19ee49f9e0 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -47,7 +47,7 @@ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`ca ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch -sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` +sudo -u git -H git checkout `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` ``` ### 5. Install libs, migrations, etc. diff --git a/doc/workflow/README.md b/doc/workflow/README.md index a6b4d951188..d2642495c9a 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -17,4 +17,5 @@ - [Milestones](milestones.md) - [Merge Requests](merge_requests.md) - ["Work In Progress" Merge Requests](wip_merge_requests.md) +- [Merge When Build Succeeds](merge_when_build_succeeds.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) diff --git a/doc/workflow/merge_when_build_succeeds.md b/doc/workflow/merge_when_build_succeeds.md new file mode 100644 index 00000000000..75e1fdff2b2 --- /dev/null +++ b/doc/workflow/merge_when_build_succeeds.md @@ -0,0 +1,15 @@ +# Merge When Build Succeeds + +When reviewing a merge request that looks ready to merge but still has one or more CI builds running, you can set it to be merged automatically when all builds succeed. This way, you don't have to wait for the builds to finish and remember to merge the request manually. + + + +When you hit the "Merge When Build Succeeds" button, the status of the merge request will be updated to represent the impending merge. If you cannot wait for the build to succeed and want to merge immediately, this option is available in the dropdown menu on the right of the main button. + +Both team developers and the author of the merge request have the option to cancel the automatic merge if they find a reason why it shouldn't be merged after all. + + + +When the build succeeds, the merge request will automatically be merged. When the build fails, the author gets a chance to retry any failed builds, or to push new commits to fix the failure. + +When the builds are retried and succeed on the second try, the merge request will automatically be merged after all. When the merge request is updated with new commits, the automatic merge is automatically canceled to allow the new changes to be reviewed. diff --git a/doc/workflow/merge_when_build_succeeds/enable.png b/doc/workflow/merge_when_build_succeeds/enable.png Binary files differnew file mode 100644 index 00000000000..633efa1246f --- /dev/null +++ b/doc/workflow/merge_when_build_succeeds/enable.png diff --git a/doc/workflow/merge_when_build_succeeds/status.png b/doc/workflow/merge_when_build_succeeds/status.png Binary files differnew file mode 100644 index 00000000000..c856c7d14dc --- /dev/null +++ b/doc/workflow/merge_when_build_succeeds/status.png |
