diff options
Diffstat (limited to 'doc/ci')
40 files changed, 1093 insertions, 153 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 10ce4ac8940..545cc72682d 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,6 +1,6 @@ -## GitLab CI Documentation +# GitLab CI Documentation -### CI User documentation +## CI User documentation - [Get started with GitLab CI](quick_start/README.md) - [CI examples for various languages](examples/README.md) @@ -16,5 +16,12 @@ - [Trigger builds through the API](triggers/README.md) - [Build artifacts](../user/project/builds/artifacts.md) - [User permissions](../user/permissions.md#gitlab-ci) +- [Build permissions](../user/permissions.md#build-permissions) - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) +- [CI/CD pipelines settings](../user/project/pipelines/settings.md) +- [Review Apps](review_apps/index.md) + +## Breaking changes + +- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 0f64137a8a9..89088cf9b83 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -44,7 +44,8 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. 2. Install Docker Engine on server. - For more information how to install Docker Engine on different systems checkout the [Supported installations](https://docs.docker.com/engine/installation/). + For more information how to install Docker Engine on different systems + checkout the [Supported installations](https://docs.docker.com/engine/installation/). 3. Add `gitlab-runner` user to `docker` group: @@ -122,11 +123,17 @@ In order to do that, follow the steps: Insecure = false ``` -1. You can now use `docker` in the build script (note the inclusion of the `docker:dind` service): +1. You can now use `docker` in the build script (note the inclusion of the + `docker:dind` service): ```yaml image: docker:latest + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + variables: + DOCKER_DRIVER: overlay + services: - docker:dind @@ -140,15 +147,21 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: -* By enabling `--docker-privileged`, you are effectively disabling all of -the security mechanisms of containers and exposing your host to privilege -escalation which can lead to container breakout. For more information, check out the official Docker documentation on -[Runtime privilege and Linux capabilities][docker-cap]. -* Using docker-in-docker, each build is in a clean environment without the past -history. Concurrent builds work fine because every build gets it's own instance of docker engine so they won't conflict with each other. But this also means builds can be slower because there's no caching of layers. -* By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form -offered. +Docker-in-Docker works well, and is the recommended configuration, but it is +not without its own challenges: + +- By enabling `--docker-privileged`, you are effectively disabling all of + the security mechanisms of containers and exposing your host to privilege + escalation which can lead to container breakout. For more information, check + out the official Docker documentation on + [Runtime privilege and Linux capabilities][docker-cap]. +- Using docker-in-docker, each build is in a clean environment without the past + history. Concurrent builds work fine because every build gets it's own + instance of Docker engine so they won't conflict with each other. But this + also means builds can be slower because there's no caching of layers. +- By default, `docker:dind` uses `--storage-driver vfs` which is the slowest + form offered. To use a different driver, see + [Using the overlayfs driver](#using-the-overlayfs-driver). An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. @@ -188,7 +201,7 @@ In order to do that, follow the steps: image = "docker:latest" privileged = false disable_cache = false - volumes = ["/var/run/docker.sock", "/cache"] + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] [runners.cache] Insecure = false ``` @@ -221,12 +234,46 @@ work as expected since volume mounting is done in the context of the host machine, not the build container. e.g. `docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests` +## Using the OverlayFS driver + +By default, when using `docker:dind`, Docker uses the `vfs` storage driver which +copies the filesystem on every run. This is a very disk-intensive operation +which can be avoided if a different driver is used, for example `overlay`. + +1. Make sure a recent kernel is used, preferably `>= 4.2`. +1. Check whether the `overlay` module is loaded: + + ``` + sudo lsmod | grep overlay + ``` + + If you see no result, then it isn't loaded. To load it use: + + ``` + sudo modprobe overlay + ``` + + If everything went fine, you need to make sure module is loaded on reboot. + On Ubuntu systems, this is done by editing `/etc/modules`. Just add the + following line into it: + + ``` + overlay + ``` + +1. Use the driver by defining a variable at the top of your `.gitlab-ci.yml`: + + ``` + variables: + DOCKER_DRIVER: overlay + ``` + ## Using the GitLab Container Registry > **Note:** This feature requires GitLab 8.8 and GitLab Runner 1.2. -Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../container_registry/README.md). For example, if you're using +Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../user/project/container_registry.md). For example, if you're using docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look: @@ -242,10 +289,10 @@ docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look: - docker push registry.example.com/group/project:latest ``` -You have to use the credentials of the special `gitlab-ci-token` user with its -password stored in `$CI_BUILD_TOKEN` in order to push to the Registry connected -to your project. This allows you to automate building and deployment of your -Docker images. +You have to use the special `gitlab-ci-token` user created for you in order to +push to the Registry connected to your project. Its password is provided in the +`$CI_BUILD_TOKEN` variable. This allows you to automate building and deployment +of your Docker images. Here's a more elaborate example that splits up the tasks into 4 pipeline stages, including two tests that run in parallel. The build is stored in the container diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index a849905ac6b..aba77490915 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -37,7 +37,7 @@ The registered runner will use the `ruby:2.1` docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. -## What is image +## What is an image The `image` keyword is the name of the docker image that is present in the local Docker Engine (list all images with `docker images`) or any image that @@ -47,7 +47,7 @@ Hub please read the [Docker Fundamentals][] documentation. In short, with `image` we refer to the docker image, which will be used to create a container on which your build will run. -## What is service +## What is a service The `services` keyword defines just another docker image that is run during your build and is linked to the docker image that the `image` keyword defines. @@ -61,7 +61,7 @@ time the project is built. You can see some widely used services examples in the relevant documentation of [CI services examples](../services/README.md). -### How is service linked to the build +### How services are linked to the build To better understand how the container linking works, read [Linking containers together][linking-containers]. @@ -221,7 +221,7 @@ time. *Note: The following commands are run without root privileges. You should be able to run docker with your regular user account.* -First start with creating a file named `build script`: +First start with creating a file named `build_script`: ```bash cat <<EOF > build_script diff --git a/doc/ci/environments.md b/doc/ci/environments.md index d85b8a34ced..9dd84a5ff81 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,56 +3,523 @@ >**Note:** Introduced in GitLab 8.9. -## Environments +During the development of software, there can be many stages until it's ready +for public consumption. You sure want to first test your code and then deploy it +in a testing or staging environment before you release it to the public. That +way you can prevent bugs not only in your software, but in the deployment +process as well. -Environments are places where code gets deployed, such as staging or production. -CI/CD [Pipelines] usually have one or more [jobs] that deploy to an environment. -Defining environments in a project's `.gitlab-ci.yml` lets developers track -[deployments] to these environments. +GitLab CI is capable of not only testing or building your projects, but also +deploying them in your infrastructure, with the added benefit of giving you a +way to track your deployments. In other words, you can always know what is +currently being deployed or has been deployed on your servers. -## Deployments +## Overview -Deployments are created when [jobs] deploy versions of code to [environments]. +With environments, you can control the Continuous Deployment of your software +all within GitLab. All you need to do is define them in your project's +[`.gitlab-ci.yml`][yaml] as we will explore below. GitLab provides a full +history of your deployments per every environment. + +Environments are like tags for your CI jobs, describing where code gets deployed. +Deployments are created when [jobs] deploy versions of code to environments, +so every environment can have one or more deployments. GitLab keeps track of +your deployments, so you always know what is currently being deployed on your +servers. + +To better understand how environments and deployments work, let's consider an +example. We assume that you have already created a project in GitLab and set up +a Runner. The example will cover the following: + +- We are developing an application +- We want to run tests and build our app on all branches +- Our default branch is `master` +- We deploy the app only when a pipeline on `master` branch is run + +Let's see how it all ties together. ## Defining environments -You can create and delete environments manually in the web interface, but we -recommend that you define your environments in `.gitlab-ci.yml` first, which -will automatically create environments for you after the first deploy. +Let's consider the following `.gitlab-ci.yml` example: -The `environment` is just a hint for GitLab that this job actually deploys to -this environment. Each time the job succeeds, a deployment is recorded, -remembering the git SHA and environment. +```yaml +stages: + - test + - build + - deploy -Add something like this to your `.gitlab-ci.yml`: +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master ``` -production: + +We have defined 3 [stages](yaml/README.md#stages): + +- test +- build +- deploy + +The jobs assigned to these stages will run in this order. If a job fails, then +the builds that are assigned to the next stage won't run, rendering the pipeline +as failed. In our case, the `test` job will run first, then the `build` and +lastly the `deploy_staging`. With this, we ensure that first the tests pass, +then our app is able to be built successfully, and lastly we deploy to the +staging server. + +The `environment` keyword is just a hint for GitLab that this job actually +deploys to this environment's `name`. It can also have a `url` which, as we +will later see, is exposed in various places within GitLab. Each time a job that +has an environment specified and succeeds, a deployment is recorded, remembering +the Git SHA and environment name. + +To sum up, with the above `.gitlab-ci.yml` we have achieved that: + +- All branches will run the `test` and `build` jobs. +- The `deploy_staging` job will run [only](yaml/README.md#only) on the `master` + branch which means all merge requests that are created from branches don't + get to deploy to the staging server +- When a merge request is merged, all jobs will run and the `deploy_staging` + in particular will deploy our code to a staging server while the deployment + will be recorded in an environment named `staging`. + +Let's now see how that information is exposed within GitLab. + +## Viewing the current status of an environment + +The environment list under your project's **Pipelines ➔ Environments**, is +where you can find information of the last deployment status of an environment. + +Here's how the Environments page looks so far. + + + +There's a bunch of information there, specifically you can see: + +- The environment's name with a link to its deployments +- The last deployment ID number and who performed it +- The build ID of the last deployment with its respective job name +- The commit information of the last deployment such as who committed, to what + branch and the Git SHA of the commit +- The exact time the last deployment was performed +- A button that takes you to the URL that you have defined under the + `environment` keyword in `.gitlab-ci.yml` +- A button that re-deploys the latest deployment, meaning it runs the job + defined by the environment name for that specific commit + +>**Notes:** +- While you can create environments manually in the web interface, we recommend + that you define your environments in `.gitlab-ci.yml` first. They will + be automatically created for you after the first deploy. +- The environments page can only be viewed by Reporters and above. For more + information on the permissions, see the [permissions documentation][permissions]. +- Only deploys that happen after your `.gitlab-ci.yml` is properly configured + will show up in the "Environment" and "Last deployment" lists. + +The information shown in the Environments page is limited to the latest +deployments, but as you may have guessed an environment can have multiple +deployments. + +## Viewing the deployment history of an environment + +GitLab keeps track of your deployments, so you always know what is currently +being deployed on your servers. That way you can have the full history of your +deployments per every environment right in your browser. Clicking on an +environment will show the history of its deployments. Assuming you have deployed +multiple times already, here's how a specific environment's page looks like. + + + +We can see the same information as when in the Environments page, but this time +all deployments are shown. As you may have noticed, apart from the **Re-deploy** +button there are now **Rollback** buttons for each deployment. Let's see how +that works. + +## Rolling back changes + +You can't control everything, so sometimes things go wrong. When that unfortunate +time comes GitLab has you covered. Simply by clicking the **Rollback** button +that can be found in the deployments page +(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the +job with the commit associated with it. + +>**Note:** +Bare in mind that your mileage will vary and it's entirely up to how you define +the deployment process in the job's `script` whether the rollback succeeds or not. +GitLab CI is just following orders. + +Thankfully that was the staging server that we had to rollback, and since we +learn from our mistakes, we decided to not make the same again when we deploy +to the production server. Enter manual actions for deployments. + +## Manually deploying to environments + +Turning a job from running automatically to a manual action is as simple as +adding `when: manual` to it. To expand on our previous example, let's add +another job that this time deploys our app to a production server and is +tracked by a `production` environment. The `.gitlab-ci.yml` looks like this +so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: stage: deploy - script: dpl... - environment: production + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -See full [documentation](yaml/README.md#environment). +The `when: manual` action exposes a play button in GitLab's UI and the +`deploy_prod` job will only be triggered if and when we click that play button. +You can find it in the pipeline, build, environment, and deployment views. -## Seeing environment status +| Pipelines | Single pipeline | Environments | Deployments | Builds | +| --------- | ----------------| ------------ | ----------- | -------| +|  |  |  |  |  | -You can find the environment list under **Pipelines > Environments** for your -project. You'll see the git SHA and date of the last deployment to each -environment defined. +Clicking on the play button in either of these places will trigger the +`deploy_prod` job, and the deployment will be recorded under a new +environment named `production`. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Remember that if your environment's name is `production` (all lowercase), then +it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). +Double the benefit! + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. + +## Dynamic environments + +As the name suggests, it is possible to create environments on the fly by just +declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is +the basis of [Review apps](review_apps/index.md). + +GitLab Runner exposes various [environment variables][variables] when a job runs, +and as such, you can use them as environment names. Let's add another job in +our example which will deploy to all branches except `master`: -## Seeing deployment history +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master +``` + +Let's break it down in pieces. The job's name is `deploy_review` and it runs +on the `deploy` stage. The `script` at this point is fictional, you'd have to +use your own based on your deployment. Then, we set the `environment` with the +`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting +one. Since the [environment name][env-name] can contain also slashes (`/`), we +can use this pattern to distinguish between dynamic environments and the regular +ones. + +So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` +which takes the value of the branch name. We also use the same +`$CI_BUILD_REF_NAME` value in the `environment:url` so that the environment +can get a specific and distinct URL for each branch. Again, the way you set up +the webserver to serve these requests is based on your setup. -Clicking on an environment will show the history of deployments. +Last but not least, we tell the job to run [`only`][only] on branches +[`except`][only] master. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +You are not bound to use the same prefix or only slashes in the dynamic +environments' names (`/`), but as we will see later, this will enable the +[grouping similar environments](#grouping-similar-environments) feature. + +The whole `.gitlab-ci.yml` looks like this so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master +``` + +A more realistic example would include copying files to a location where a +webserver (NGINX) could then read and serve. The example below will copy the +`public` directory to `/srv/nginx/$CI_BUILD_REF_NAME/public`: + +```yaml +review_app: + stage: deploy + script: + - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_NAME + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com +``` + +It is assumed that the user has already setup NGINX and GitLab Runner in the +server this job will run on. + +>**Note:** +Be sure to check out the [limitations](#limitations) section for some edge +cases regarding naming of you branches and Review Apps. + +--- + +The development workflow would now be: + +- Developer creates a branch locally +- Developer makes changes, commits and pushes the branch to GitLab +- Developer creates a merge request + +Behind the scenes: + +- GitLab Runner picks up the changes and starts running the jobs +- The jobs run sequentially as defined in `stages` + - First, the tests pass + - Then, the build begins and successfully also passes + - Lastly, the app is deployed to an environment with a name specific to the + branch + +So now, every branch gets its own environment and is deployed to its own place +with the added benefit of having a [history of deployments](#viewing-the-deployment-history-of-an-environment) +and also being able to [rollback changes](#rolling-back-changes) if needed. +Let's briefly see where URL that's defined in the environments is exposed. + +## Making use of the environment URL + +The environment URL is exposed in a few places within GitLab. + +| In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | +| -------------------- | ------------ | ----------- | +|  |  |  | + +If a merge request is eventually merged to the default branch (in our case +`master`) and that branch also deploys to an environment (in our case `staging` +and/or `production`) you can see this information in the merge request itself. + + + +--- + +We now have a full development cycle, where our app is tested, built, deployed +as a Review app, deployed to a staging server once the merge request is merged, +and finally manually deployed to the production server. What we just described +is a single workflow, but imagine tens of developers working on a project +at the same time. They each push to their branches, and dynamic environments are +created all the time. In that case, we probably need to do some clean up. Read +next how environments can be stopped. + +## Stopping an environment + +By stopping an environment, you are effectively terminating its recording of the +deployments that happen in it. + +A branch is associated with an environment when the CI pipeline that is created +for this branch, was recently deployed to this environment. You can think of +the CI pipeline as the glue between the branch and the environment: +`branch ➔ CI pipeline ➔ environment`. + +There is a special case where environments can be manually stopped. That can +happen if you provide another job for that matter. The syntax is a little +tricky since a job calls another job to do the job. + +Consider the following example where the `deploy_review` calls the `stop_review` +to clean up and stop the environment: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + on_stop: stop_review + only: + - branches + except: + - master + +stop_review: + variables: + GIT_STRATEGY: none + script: + - echo "Remove review app" + when: manual + environment: + name: review/$CI_BUILD_REF_NAME + action: stop +``` + +Setting the [`GIT_STRATEGY`][git-strategy] to `none` is necessary on the +`stop_review` job so that the [GitLab Runner] won't try to checkout the code +after the branch is deleted. + +>**Note:** +Starting with GitLab 8.14, dynamic environments will be stopped automatically +when their associated branch is deleted. + +When you have an environment that has a stop action defined (typically when +the environment describes a review app), GitLab will automatically trigger a +stop action when the associated branch is deleted. + +You can read more in the [`.gitlab-ci.yml` reference][onstop]. + +## Grouping similar environments + +> [Introduced][ce-7015] in GitLab 8.14. + +As we've seen in the [dynamic environments](#dynamic-environments), you can +prepend their name with a word, then followed by a `/` and finally the branch +name which is automatically defined by the `CI_BUILD_REF_NAME` variable. + +In short, environments that are named like `type/foo` are presented under a +group named `type`. + +In our minimal example, we name the environments `review/$CI_BUILD_REF_NAME` +where `$CI_BUILD_REF_NAME` is the branch name: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME +``` + +In that case, if you visit the Environments page, and provided the branches +exist, you should see something like: + + + +## Checkout deployments locally + +Since 8.13, a reference in the git repository is saved for each deployment. So +knowing what the state is of your current environments is only a `git fetch` +away. + +In your git config, append the `[remote "<your-remote>"]` block with an extra +fetch line: + +``` +fetch = +refs/environments/*:refs/remotes/origin/environments/* +``` + +## Limitations + +1. If the branch name contains special characters (`/`), and you use the + `$CI_BUILD_REF_NAME` variable to dynamically create environments, there might + be complications during your Review Apps deployment. Follow the + [issue 22849][ce-22849] for more information. +1. You are limited to use only the [CI predefined variables][variables] in the + `environment: name`. If you try to re-use variables defined inside `script` + as part of the environment name, it will not work. + +## Further reading + +Below are some links you may find interesting: + +- [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment) +- [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +- [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs +[yaml]: yaml/README.md [environments]: #environments [deployments]: #deployments +[permissions]: ../user/permissions.md +[variables]: variables/README.md +[env-name]: yaml/README.md#environment-name +[only]: yaml/README.md#only-and-except +[onstop]: yaml/README.md#environment-on_stop +[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 +[ce-22849]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22849 +[gitlab runner]: https://docs.gitlab.com/runner/ +[git-strategy]: yaml/README.md#git-strategy diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 406396deaaa..ffc310ec8c7 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -11,9 +11,11 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - [Test and deploy a Python application to Heroku](test-and-deploy-python-application-to-heroku.md) - [Test a Clojure application](test-clojure-application.md) - [Test a Scala application](test-scala-application.md) +- [Test a Phoenix application](test-phoenix-application.md) - [Using `dpl` as deployment tool](deployment/README.md) +- [Example project that shows how to use Review Apps](https://gitlab.com/gitlab-examples/review-apps-nginx/) - [Blog post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) -- [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples) +- [Repositories with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -[gitlab-ci-templates][https://gitlab.com/gitlab-org/gitlab-ci-yml] +[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md new file mode 100644 index 00000000000..150698ca04b --- /dev/null +++ b/doc/ci/examples/test-phoenix-application.md @@ -0,0 +1,56 @@ +## Test a Phoenix application + +This example demonstrates the integration of Gitlab CI with Phoenix, Elixir and +Postgres. + +### Add `.gitlab-ci.yml` file to project + +The following `.gitlab-ci.yml` should be added in the root of your +repository to trigger CI: + +```yaml +image: elixir:1.3 + +services: + - postgres:9.6 + +variables: + MIX_ENV: "test" + +before_script: + # Setup phoenix dependencies + - apt-get update + - apt-get install -y postgresql-client + - mix local.hex --force + - mix deps.get --only test + - mix ecto.reset + +test: + script: + - mix test +``` + +The variables will set the Mix environment to "test". The +`before_script` will install `psql`, some Phoenix dependencies, and will also +run your migrations. + +Finally, the test `script` will run your tests. + +### Update the Config Settings + +In `config/test.exs`, update the database hostname: + +```elixir +config :my_app, MyApp.Repo, + hostname: if(System.get_env("CI"), do: "postgres", else: "localhost"), +``` + +### Add the Migrations Folder + +If you do not have any migrations yet, you will need to create an empty +`.gitkeep` file in `priv/repo/migrations`. + +### Sources + +- https://medium.com/@nahtnam/using-phoenix-on-gitlab-ci-5a51eec81142 +- https://davejlong.com/ci-with-phoenix-and-gitlab/ diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index 7412fdbbc78..85f8849fa99 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -1,11 +1,11 @@ -## Test a Scala application +# Test and deploy to Heroku a Scala application This example demonstrates the integration of Gitlab CI with Scala applications using SBT. Checkout the example [project](https://gitlab.com/gitlab-examples/scala-sbt) and [build status](https://gitlab.com/gitlab-examples/scala-sbt/builds). -### Add `.gitlab-ci.yml` file to project +## Add `.gitlab-ci.yml` file to project The following `.gitlab-ci.yml` should be added in the root of your repository to trigger CI: @@ -13,10 +13,14 @@ repository to trigger CI: ``` yaml image: java:8 +stages: + - test + - deploy + before_script: - apt-get update -y - apt-get install apt-transport-https -y - # Install SBT + ## Install SBT - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 - apt-get update -y @@ -24,8 +28,17 @@ before_script: - sbt sbt-version test: + stage: test script: - sbt clean coverage test coverageReport + +deploy: + stage: deploy + script: + - apt-get update -yq + - apt-get install rubygems ruby-dev -y + - gem install dpl + - dpl --provider=heroku --app=gitlab-play-sample-app --api-key=$HEROKU_API_KEY ``` The `before_script` installs [SBT](http://www.scala-sbt.org/) and @@ -33,15 +46,31 @@ displays the version that is being used. The `test` stage executes SBT to compile and test the project. [scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT plugin to measure test coverage. +The `deploy` stage automatically deploys the project to Heroku using dpl. You can use other versions of Scala and SBT by defining them in `build.sbt`. -### Display test coverage in build +## Display test coverage in build Add the `Coverage was \[\d+.\d+\%\]` regular expression in the -**Settings > Edit Project > Test coverage parsing** project setting to -retrieve the test coverage rate from the build trace and have it +**Settings ➔ Edit Project ➔ Test coverage parsing** project setting to +retrieve the [test coverage] rate from the build trace and have it displayed with your builds. **Builds** must be enabled for this option to appear. + +## Heroku application + +A Heroku application is required. You can create one through the +[Dashboard](https://dashboard.heroku.com/). Substitute `gitlab-play-sample-app` +in the `.gitlab-ci.yml` file with your application's name. + +## Heroku API key + +You can look up your Heroku API key in your +[account](https://dashboard.heroku.com/account). Add a secure [variable] with +this value in **Project ➔ Variables** with key `HEROKU_API_KEY`. + +[variable]: ../variables/README.md#user-defined-variables-secure-variables +[test coverage]: ../../user/project/pipelines/settings.md#test-coverage-report-badge diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png Binary files differindex 35780e277ae..2d7eec8a949 100644 --- a/doc/ci/img/builds_tab.png +++ b/doc/ci/img/builds_tab.png diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differnew file mode 100644 index 00000000000..7ded0c97b72 --- /dev/null +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differnew file mode 100644 index 00000000000..5c031ad0d9d --- /dev/null +++ b/doc/ci/img/environments_available_staging.png diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differnew file mode 100644 index 00000000000..0f42b368c5b --- /dev/null +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differnew file mode 100644 index 00000000000..44010f6aa6f --- /dev/null +++ b/doc/ci/img/environments_link_url.png diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differnew file mode 100644 index 00000000000..4f90143527a --- /dev/null +++ b/doc/ci/img/environments_link_url_deployments.png diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differnew file mode 100644 index 00000000000..64f134e0b0d --- /dev/null +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differnew file mode 100644 index 00000000000..e7cf63a1031 --- /dev/null +++ b/doc/ci/img/environments_manual_action_builds.png diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differnew file mode 100644 index 00000000000..2b3f6f3edad --- /dev/null +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differnew file mode 100644 index 00000000000..e0c07604e7f --- /dev/null +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differnew file mode 100644 index 00000000000..82bbae88027 --- /dev/null +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differnew file mode 100644 index 00000000000..36337cb1870 --- /dev/null +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differnew file mode 100644 index 00000000000..7bff84362a3 --- /dev/null +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differnew file mode 100644 index 00000000000..821352188ef --- /dev/null +++ b/doc/ci/img/environments_view.png diff --git a/doc/ci/img/features_settings.png b/doc/ci/img/features_settings.png Binary files differindex 38d7036f606..c159253d1c9 100644 --- a/doc/ci/img/features_settings.png +++ b/doc/ci/img/features_settings.png diff --git a/doc/ci/img/pipelines.png b/doc/ci/img/pipelines.png Binary files differnew file mode 100644 index 00000000000..5937e9d99c8 --- /dev/null +++ b/doc/ci/img/pipelines.png diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index ca9b986a060..03b9c4bb444 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -5,12 +5,14 @@ Introduced in GitLab 8.8. ## Pipelines -A pipeline is a group of [builds] that get executed in [stages] \(batches). All -of the builds in a stage are executed in parallel (if there are enough -concurrent [runners]), and if they all succeed, the pipeline moves on to the +A pipeline is a group of [builds][] that get executed in [stages][](batches). +All of the builds in a stage are executed in parallel (if there are enough +concurrent [Runners]), and if they all succeed, the pipeline moves on to the next stage. If one of the builds fails, the next stage is not (usually) executed. + + ## Builds Builds are individual runs of [jobs]. Not to be confused with a `build` job or @@ -25,51 +27,22 @@ See full [documentation](yaml/README.md#jobs). ## Seeing pipeline status -You can find the current and historical pipeline runs under **Pipelines** for your -project. +You can find the current and historical pipeline runs under **Pipelines** for +your project. ## Seeing build status Clicking on a pipeline will show the builds that were run for that pipeline. +Clicking on an individual build will show you its build trace, and allow you to +cancel the build, retry it, or erase the build trace. ## Badges -There are build status and test coverage report badges available. - -Go to pipeline settings to see available badges and code you can use to embed -badges in the `README.md` or your website. - -### Build status badge - -You can access a build status badge image using following link: - -``` -http://example.gitlab.com/namespace/project/badges/branch/build.svg -``` - -### Test coverage report badge - -GitLab makes it possible to define the regular expression for coverage report, -that each build log will be matched against. This means that each build in the -pipeline can have the test coverage percentage value defined. - -You can access test coverage badge using following link: - -``` -http://example.gitlab.com/namespace/project/badges/branch/coverage.svg -``` - -If you would like to get the coverage report from the specific job, you can add -a `job=coverage_job_name` parameter to the URL. For example, it is possible to -use following Markdown code to embed the est coverage report into `README.md`: - -```markdown - -``` - -The latest successful pipeline will be used to read the test coverage value. +Build status and test coverage report badges are available. You can find their +respective link in the [Pipelines settings] page. [builds]: #builds [jobs]: yaml/README.md#jobs [stages]: yaml/README.md#stages -[runners]: runners/README.md +[runners]: runners/READM +[pipelines settings]: ../user/project/pipelines/settings.md diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png Binary files differindex b53a6cd86b0..87643d62d58 100644 --- a/doc/ci/quick_start/img/build_log.png +++ b/doc/ci/quick_start/img/build_log.png diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png Binary files differindex 47862761ffe..d287ae3064f 100644 --- a/doc/ci/quick_start/img/builds_status.png +++ b/doc/ci/quick_start/img/builds_status.png diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png Binary files differindex a53562ce328..29c2fea5d6d 100644 --- a/doc/ci/quick_start/img/new_commit.png +++ b/doc/ci/quick_start/img/new_commit.png diff --git a/doc/ci/quick_start/img/pipelines_status.png b/doc/ci/quick_start/img/pipelines_status.png Binary files differindex 6bc97bb739c..53ccc49bd66 100644 --- a/doc/ci/quick_start/img/pipelines_status.png +++ b/doc/ci/quick_start/img/pipelines_status.png diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png Binary files differindex 23261123b18..5ce6fe8e17c 100644 --- a/doc/ci/quick_start/img/runners_activated.png +++ b/doc/ci/quick_start/img/runners_activated.png diff --git a/doc/ci/quick_start/img/single_commit_status_pending.png b/doc/ci/quick_start/img/single_commit_status_pending.png Binary files differindex ccf3ac957bb..91fc9011847 100644 --- a/doc/ci/quick_start/img/single_commit_status_pending.png +++ b/doc/ci/quick_start/img/single_commit_status_pending.png diff --git a/doc/ci/quick_start/img/status_pending.png b/doc/ci/quick_start/img/status_pending.png Binary files differindex 9feacf0c961..cbd44a189d3 100644 --- a/doc/ci/quick_start/img/status_pending.png +++ b/doc/ci/quick_start/img/status_pending.png diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr.png b/doc/ci/review_apps/img/review_apps_preview_in_mr.png Binary files differnew file mode 100644 index 00000000000..0300392f24b --- /dev/null +++ b/doc/ci/review_apps/img/review_apps_preview_in_mr.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md new file mode 100644 index 00000000000..a66165dc973 --- /dev/null +++ b/doc/ci/review_apps/index.md @@ -0,0 +1,124 @@ +# Getting started with Review Apps + +> +- [Introduced][ce-21971] in GitLab 8.12. Further additions were made in GitLab + 8.13 and 8.14. +- Inspired by [Heroku's Review Apps][heroku-apps] which itself was inspired by + [Fourchette]. + +The basis of Review Apps is the [dynamic environments] which allow you to create +a new environment (dynamically) for each one of your branches. + +A Review App can then be visible as a link when you visit the [merge request] +relevant to the branch. That way, you are able to see live all changes introduced +by the merge request changes. Reviewing anything, from performance to interface +changes, becomes much easier with a live environment and as such, Review Apps +can make a huge impact on your development flow. + +They mostly make sense to be used with web applications, but you can use them +any way you'd like. + +## Overview + +Simply put, a Review App is a mapping of a branch with an environment as there +is a 1:1 relation between them. + +Here's an example of what it looks like when viewing a merge request with a +dynamically set environment. + + + +In the image above you can see that the `add-new-line` branch was successfully +built and deployed under a dynamic environment and can be previewed with an +also dynamically URL. + +The details of the Review Apps implementation depend widely on your real +technology stack and on your deployment process. The simplest case it to +deploy a simple static HTML website, but it will not be that straightforward +when your app is using a database for example. To make a branch be deployed +on a temporary instance and booting up this instance with all required software +and services automatically on the fly is not a trivial task. However, it is +doable, especially if you use Docker, or at least a configuration management +tool like Chef, Puppet, Ansible or Salt. + +## Prerequisites + +To get a better understanding of Review Apps, you must first learn how +environments and deployments work. The following docs will help you grasp that +knowledge: + +1. First, learn about [environments][] and their role in the development workflow. +1. Then make a small stop to learn about [CI variables][variables] and how they + can be used in your CI jobs. +1. Next, explore the [`environment` syntax][yaml-env] as defined in `.gitlab-ci.yml`. + This will be your primary reference when you are finally comfortable with + how environments work. +1. Additionally, find out about [manual actions][] and how you can use them to + deploy to critical environments like production with the push of a button. +1. And as a last step, follow the [example tutorials](#examples) which will + guide you step by step to set up the infrastructure and make use of + Review Apps. + +## Configuration + +The configuration of Review apps depends on your technology stack and your +infrastructure. Read the [dynamic environments] documentation to understand +how to define and create them. + +## Creating and destroying Review Apps + +The creation and destruction of a Review App is defined in `.gitlab-ci.yml` +at a job level under the `environment` keyword. + +Check the [environments] documentation how to do so. + +## A simple workflow + +The process of adding Review Apps in your workflow would look like: + +1. Set up the infrastructure to host and deploy the Review Apps. +1. [Install][install-runner] and [configure][conf-runner] a Runner that does + the deployment. +1. Set up a job in `.gitlab-ci.yml` that uses the predefined + [predefined CI environment variable][variables] `${CI_BUILD_REF_NAME}` to + create dynamic environments and restrict it to run only on branches. +1. Optionally set a job that [manually stops][manual-env] the Review Apps. + +From there on, you would follow the branched Git flow: + +1. Push a branch and let the Runner deploy the Review App based on the `script` + definition of the dynamic environment job. +1. Wait for the Runner to build and/or deploy your web app. +1. Click on the link that's present in the MR related to the branch and see the + changes live. + +## Limitations + +Check the [environments limitations](../environments.md#limitations). + +## Examples + +A list of examples used with Review Apps can be found below: + +- [Use with NGINX][app-nginx] - Use NGINX and the shell executor of GitLab Runner + to deploy a simple HTML website. + +And below is a soon to be added examples list: + +- Use with Amazon S3 +- Use on Heroku with dpl +- Use with OpenShift/kubernetes + +[app-nginx]: https://gitlab.com/gitlab-examples/review-apps-nginx +[ce-21971]: https://gitlab.com/gitlab-org/gitlab-ce/issues/21971 +[dynamic environments]: ../environments.md#dynamic-environments +[environments]: ../environments.md +[fourchette]: https://github.com/rainforestapp/fourchette +[heroku-apps]: https://devcenter.heroku.com/articles/github-integration-review-apps +[manual actions]: ../environments.md#manual-actions +[merge request]: ../../user/project/merge_requests.md +[variables]: ../variables/README.md +[yaml-env]: ../yaml/README.md#environment +[install-runner]: https://docs.gitlab.com/runner/install/ +[conf-runner]: https://docs.gitlab.com/runner/commands/ +[manual-env]: ../environments.md#stopping-an-environment diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 6c6767fea0b..cf7c55f75f2 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -2,6 +2,10 @@ > [Introduced][ci-229] in GitLab CE 7.14. +> **Note**: +GitLab 8.12 has a completely redesigned build permissions system. +Read all about the [new model and its implications](../../user/project/new_ci_build_permissions_model.md#build-triggers). + Triggers can be used to force a rebuild of a specific branch, tag or commit, with an API call. @@ -54,6 +58,22 @@ below. See the [Examples](#examples) section for more details on how to actually trigger a rebuild. +## Trigger a build from webhook + +> Introduced in GitLab 8.14. + +To trigger a build from webhook of another project you need to add the following +webhook url for Push and Tag push events: + +``` +https://gitlab.example.com/api/v3/projects/:id/ref/:ref/trigger/builds?token=TOKEN +``` + +> **Note**: +- `ref` should be passed as part of url in order to take precedence over `ref` + from webhook body that designates the branchref that fired the trigger in the source repository. +- `ref` should be url encoded if contains slashes. + ## Pass build variables to a trigger You can pass any number of arbitrary variables in the trigger API call and they @@ -165,6 +185,14 @@ curl --request POST \ https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` +### Using webhook to trigger builds + +You can add the following webhook to another project in order to trigger a build: + +``` +https://gitlab.example.com/api/v3/projects/9/ref/master/trigger/builds?token=TOKEN&variables[UPLOAD_TO_S3]=true +``` + ### Using cron to trigger nightly builds Whether you craft a script or just run cURL directly, you can trigger builds diff --git a/doc/ci/triggers/img/builds_page.png b/doc/ci/triggers/img/builds_page.png Binary files differindex c2cf4b1852c..fded5839f76 100644 --- a/doc/ci/triggers/img/builds_page.png +++ b/doc/ci/triggers/img/builds_page.png diff --git a/doc/ci/triggers/img/trigger_single_build.png b/doc/ci/triggers/img/trigger_single_build.png Binary files differindex fa86f0fee3d..c4a5550d640 100644 --- a/doc/ci/triggers/img/trigger_single_build.png +++ b/doc/ci/triggers/img/trigger_single_build.png diff --git a/doc/ci/triggers/img/trigger_variables.png b/doc/ci/triggers/img/trigger_variables.png Binary files differindex b2fcc65d304..65fe1ea9ab6 100644 --- a/doc/ci/triggers/img/trigger_variables.png +++ b/doc/ci/triggers/img/trigger_variables.png diff --git a/doc/ci/triggers/img/triggers_page.png b/doc/ci/triggers/img/triggers_page.png Binary files differindex 438f285ae2d..56d13905ce6 100644 --- a/doc/ci/triggers/img/triggers_page.png +++ b/doc/ci/triggers/img/triggers_page.png diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 6a971c3ae87..a4c3a731a20 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -44,10 +44,11 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`. | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the build is run | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | -| **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returnes the address of the registry tied to the specific project | +| **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | **CI_RUNNER_ID** | 8.10 | 0.5 | The unique id of runner being used | | **CI_RUNNER_DESCRIPTION** | 8.10 | 0.5 | The description of the runner as saved in GitLab | | **CI_RUNNER_TAGS** | 8.10 | 0.5 | The defined runner tags | +| **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | | **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the build | | **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the build | @@ -105,6 +106,39 @@ Variables can be defined at a global level, but also at a job level. More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md). +#### Debug tracing + +> **WARNING:** Enabling debug tracing can have severe security implications. The + output **will** contain the content of all your secure variables and any other + secrets! The output **will** be uploaded to the GitLab server and made visible + in build traces! + +By default, GitLab Runner hides most of the details of what it is doing when +processing a job. This behaviour keeps build traces short, and prevents secrets +from being leaked into the trace unless your script writes them to the screen. + +If a job isn't working as expected, this can make the problem difficult to +investigate; in these cases, you can enable debug tracing in `.gitlab-ci.yml`. +Available on GitLab Runner v1.7+, this feature enables the shell's execution +trace, resulting in a verbose build trace listing all commands that were run, +variables that were set, etc. + +Before enabling this, you should ensure builds are visible to +[team members only](../../../user/permissions.md#project-features). You should +also [erase](../pipelines.md#seeing-build-traces) all generated build traces +before making them visible again. + +To enable debug traces, set the `CI_DEBUG_TRACE` variable to `true`: + +```yaml +job1: + variables: + CI_DEBUG_TRACE: "true" +``` + +The [example project](https://gitlab.com/gitlab-examples/ci-debug-trace) +demonstrates a working configuration, including build trace examples. + ### User-defined variables (Secure Variables) **This feature requires GitLab Runner 0.4.0 or higher** diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index ff4c8ddc54b..338c9a27789 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -90,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string. ### after_script ->**Note:** -Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 +> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 `after_script` is used to define the command that will be run after for all builds. This has to be an array or a multi-line string. @@ -135,8 +134,7 @@ Alias for [stages](#stages). ### variables ->**Note:** -Introduced in GitLab Runner v0.5.0. +> Introduced in GitLab Runner v0.5.0. GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the build environment. The variables are stored in the Git repository and are meant @@ -148,21 +146,25 @@ variables: ``` These variables can be later used in all executed commands and scripts. - The YAML-defined variables are also set to all created service containers, -thus allowing to fine tune them. +thus allowing to fine tune them. Variables can be also defined on a +[job level](#job-variables). -Variables can be also defined on [job level](#job-variables). +Except for the user defined variables, there are also the ones set up by the +Runner itself. One example would be `CI_BUILD_REF_NAME` which has the value of +the branch or tag name for which project is built. Apart from the variables +you can set in `.gitlab-ci.yml`, there are also the so called secret variables +which can be set in GitLab's UI. -[Learn more about variables.](../variables/README.md) +[Learn more about variables.][variables] ### cache ->**Note:** -Introduced in GitLab Runner v0.7.0. +> Introduced in GitLab Runner v0.7.0. `cache` is used to specify a list of files and directories which should be -cached between builds. +cached between builds. You can only use paths that are within the project +workspace. **By default the caching is enabled per-job and per-branch.** @@ -220,8 +222,7 @@ will be always present. For implementation details, please check GitLab Runner. #### cache:key ->**Note:** -Introduced in GitLab Runner v1.0.0. +> Introduced in GitLab Runner v1.0.0. The `key` directive allows you to define the affinity of caching between jobs, allowing to have a single cache for all jobs, @@ -531,8 +532,7 @@ The above script will: #### Manual actions ->**Note:** -Introduced in GitLab 8.10. +> Introduced in GitLab 8.10. Manual actions are a special type of job that are not executed automatically; they need to be explicitly started by a user. Manual actions can be started @@ -541,35 +541,184 @@ same manual action multiple times. An example usage of manual actions is deployment to production. +Read more at the [environments documentation][env-manual]. + ### environment ->**Note:** -Introduced in GitLab 8.9. +> Introduced in GitLab 8.9. -`environment` is used to define that a job deploys to a specific environment. -This allows easy tracking of all deployments to your environments straight from -GitLab. +> You can read more about environments and find more examples in the +[documentation about environments][environment]. +`environment` is used to define that a job deploys to a specific environment. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name must contain only letters, digits, '-' and '_'. Common -names are `qa`, `staging`, and `production`, but you can use whatever name works -with your workflow. +In its simplest form, the `environment` keyword can be defined like: ---- +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: + name: production +``` -**Example configurations** +In the above example, the `deploy to production` job will be marked as doing a +deployment to the `production` environment. + +#### environment:name + +> Introduced in GitLab 8.11. + +>**Note:** +Before GitLab 8.11, the name of an environment could be defined as a string like +`environment: production`. The recommended way now is to define it under the +`name` keyword. + +The `environment` name can contain: + +- letters +- digits +- spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` + +Common names are `qa`, `staging`, and `production`, but you can use whatever +name works with your workflow. + +Instead of defining the name of the environment right after the `environment` +keyword, it is also possible to define it as a separate value. For that, use +the `name` keyword under `environment`: ``` deploy to production: stage: deploy script: git push production HEAD:master - environment: production + environment: + name: production +``` + +#### environment:url + +> Introduced in GitLab 8.11. + +>**Note:** +Before GitLab 8.11, the URL could be added only in GitLab's UI. The +recommended way now is to define it in `.gitlab-ci.yml`. + +This is an optional value that when set, it exposes buttons in various places +in GitLab which when clicked take you to the defined URL. + +In the example below, if the job finishes successfully, it will create buttons +in the merge requests and in the environments/deployments pages which will point +to `https://prod.example.com`. + +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: + name: production + url: https://prod.example.com +``` + +#### environment:on_stop + +> +**Notes:** +- [Introduced][ce-6669] in GitLab 8.13. +- Starting with GitLab 8.14, when you have an environment that has a stop action + defined, GitLab will automatically trigger a stop action when the associated + branch is deleted. + +Closing (stoping) environments can be achieved with the `on_stop` keyword defined under +`environment`. It declares a different job that runs in order to close +the environment. + +Read the `environment:action` section for an example. + +#### environment:action + +> [Introduced][ce-6669] in GitLab 8.13. + +The `action` keyword is to be used in conjunction with `on_stop` and is defined +in the job that is called to close the environment. + +Take for instance: + +```yaml +review_app: + stage: deploy + script: make deploy-app + environment: + name: review + on_stop: stop_review_app + +stop_review_app: + stage: deploy + script: make delete-app + when: manual + environment: + name: review + action: stop ``` -The `deploy to production` job will be marked as doing deployment to -`production` environment. +In the above example we set up the `review_app` job to deploy to the `review` +environment, and we also defined a new `stop_review_app` job under `on_stop`. +Once the `review_app` job is successfully finished, it will trigger the +`stop_review_app` job based on what is defined under `when`. In this case we +set it up to `manual` so it will need a [manual action](#manual-actions) via +GitLab's web interface in order to run. + +The `stop_review_app` job is **required** to have the following keywords defined: + +- `when` - [reference](#when) +- `environment:name` +- `environment:action` + +#### dynamic environments + +> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. + +`environment` can also represent a configuration hash with `name` and `url`. +These parameters can use any of the defined [CI variables](#variables) +(including predefined, secure variables and `.gitlab-ci.yml` variables). + +>**Note:** +Be aware than if the branch name contains special characters and you use the +`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might +be complications during deployment. Follow the +[issue 22849](https://gitlab.com/gitlab-org/gitlab-ce/issues/22849) for more +information. + +For example: + +``` +deploy as review app: + stage: deploy + script: make deploy + environment: + name: review-apps/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.review.example.com/ +``` + +The `deploy as review app` job will be marked as deployment to dynamically +create the `review-apps/$CI_BUILD_REF_NAME` environment, which `$CI_BUILD_REF_NAME` +is an [environment variable][variables] set by the Runner. If for example the +`deploy as review app` job was run in a branch named `pow`, this environment +should be accessible under `https://pow.review.example.com/`. + +This of course implies that the underlying server which hosts the application +is properly configured. + +The common use case is to create dynamic environments for branches and use them +as Review Apps. You can see a simple example using Review Apps at +https://gitlab.com/gitlab-examples/review-apps-nginx/. ### artifacts @@ -581,8 +730,8 @@ The `deploy to production` job will be marked as doing deployment to > - Build artifacts are only collected for successful builds by default. `artifacts` is used to specify a list of files and directories which should be -attached to the build after success. To pass artifacts between different builds, -see [dependencies](#dependencies). +attached to the build after success. You can only use paths that are within the +project workspace. To pass artifacts between different builds, see [dependencies](#dependencies). Below are some examples. @@ -611,6 +760,15 @@ artifacts: - binaries/ ``` +To disable artifact passing, define the job with empty [dependencies](#dependencies): + +```yaml +job: + stage: build + script: make build + dependencies: [] +``` + You may want to create artifacts only for tagged releases to avoid filling the build server storage with temporary build artifacts. @@ -638,8 +796,7 @@ be available for download in the GitLab UI. #### artifacts:name ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.0. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.0. The `name` directive allows you to define the name of the created artifacts archive. That way, you can have a unique name for every archive which could be @@ -702,8 +859,7 @@ job: #### artifacts:when ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:when` is used to upload artifacts on build failure or despite the failure. @@ -728,8 +884,7 @@ job: #### artifacts:expire_in ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:expire_in` is used to delete uploaded artifacts after the specified time. By default, artifacts are stored on GitLab forever. `expire_in` allows you @@ -764,8 +919,7 @@ job: ### dependencies ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. This feature should be used in conjunction with [`artifacts`](#artifacts) and allows you to define the artifacts to pass between different builds. @@ -839,32 +993,48 @@ job: ## Git Strategy ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future -releases or be removed completely. +> Introduced in GitLab 8.9 as an experimental feature. May change or be removed + completely in future releases. `GIT_STRATEGY=none` requires GitLab Runner + v1.7+. + +You can set the `GIT_STRATEGY` used for getting recent application code, either +in the global [`variables`](#variables) section or the [`variables`](#job-variables) +section for individual jobs. If left unspecified, the default from project +settings will be used. + +There are three possible values: `clone`, `fetch`, and `none`. -You can set the `GIT_STRATEGY` used for getting recent application code. `clone` -is slower, but makes sure you have a clean directory before every build. `fetch` -is faster. `GIT_STRATEGY` can be specified in the global `variables` section or -in the `variables` section for individual jobs. If it's not specified, then the -default from project settings will be used. +`clone` is the slowest option. It clones the repository from scratch for every +job, ensuring that the project workspace is always pristine. ``` variables: GIT_STRATEGY: clone ``` -or +`fetch` is faster as it re-uses the project workspace (falling back to `clone` +if it doesn't exist). `git clean` is used to undo any changes made by the last +job, and `git fetch` is used to retrieve commits made since the last job ran. ``` variables: GIT_STRATEGY: fetch ``` +`none` also re-uses the project workspace, but skips all Git operations +(including GitLab Runner's pre-clone script, if present). It is mostly useful +for jobs that operate exclusively on artifacts (e.g., `deploy`). Git repository +data may be present, but it is certain to be out of date, so you should only +rely on files brought into the project workspace from cache or artifacts. + +``` +variables: + GIT_STRATEGY: none +``` + ## Shallow cloning ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future +> Introduced in GitLab 8.9 as an experimental feature. May change in future releases or be removed completely. You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows @@ -894,8 +1064,7 @@ variables: ## Hidden keys ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. Keys that start with a dot (`.`) will be not processed by GitLab CI. You can use this feature to ignore jobs, or use the @@ -923,8 +1092,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya ### Anchors ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. YAML also has a handy feature called 'anchors', which let you easily duplicate content across your document. Anchors can be used to duplicate/inherit @@ -1066,4 +1234,9 @@ capitalization, the commit will be created but the builds will be skipped. Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. +[env-manual]: ../environments.md#manually-deploying-to-environments [examples]: ../examples/README.md +[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 +[environment]: ../environments.md +[ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 +[variables]: ../variables/README.md |
