From f2318dfc9e80555bae99f543d4e07ec089ab3935 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Sat, 21 Nov 2015 20:25:07 +0100 Subject: Fix CI yaml syntax documentation [ci skip] --- doc/ci/yaml/README.md | 90 +++++++++++++++++++++++---------------------------- 1 file changed, 41 insertions(+), 49 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3e6071a2ee7..3dbf1afc7a9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -278,26 +278,23 @@ The above script will: `artifacts` is used to specify list of files and directories which should be attached to build after success. 1. Send all files in `binaries` and `.config`: -``` -artifacts: - paths: - - binaries/ - - .config -``` + + artifacts: + paths: + - binaries/ + - .config 2. Send all git untracked files: -``` -artifacts: - untracked: true -``` + + artifacts: + untracked: true 3. Send all git untracked files and files in `binaries`: -``` -artifacts: - untracked: true - paths: - - binaries/ -``` + + artifacts: + untracked: true + paths: + - binaries/ The artifacts will be send after the build success to GitLab and will be accessible in GitLab interface to download. @@ -307,46 +304,41 @@ This feature requires GitLab Runner v0.7.0 or higher. `cache` is used to specify list of files and directories which should be cached between builds. 1. Cache all files in `binaries` and `.config`: -``` -rspec: - script: test - cache: - paths: - - binaries/ - - .config -``` + + rspec: + script: test + cache: + paths: + - binaries/ + - .config 2. Cache all git untracked files: -``` -rspec: - script: test - cache: - untracked: true -``` + rspec: + script: test + cache: + untracked: true + 3. Cache all git untracked files and files in `binaries`: -``` -rspec: - script: test - cache: - untracked: true - paths: - - binaries/ -``` -4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: + rspec: + script: test + cache: + untracked: true + paths: + - binaries/ -``` -cache: - paths: - - my/files +4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: -rspec: - script: test - cache: - paths: - - binaries/ -``` + cache: + paths: + - my/files + + rspec: + script: test + cache: + paths: + - binaries/ The cache is provided on best effort basis, so don't expect that cache will be present. For implementation details please check GitLab Runner. -- cgit v1.2.1 From 5f6117c0aa391a6e9c96493ca01a8a23eb40f0cd Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 5 Nov 2015 16:53:05 +0100 Subject: update ci docs with yml reason and better quickstart --- doc/ci/quick_start/README.md | 202 +++++++++++++++++++++++++++++-------------- doc/ci/yaml/README.md | 30 ++++--- 2 files changed, 156 insertions(+), 76 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index d69064a91fd..74626c8f8a4 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,44 +1,62 @@ # Quick Start -To start building projects with GitLab CI a few steps needs to be done. +GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You +only need to enable it in the Services settings of your project. -## 1. Install GitLab and CI +This guide assumes that you: -First you need to have a working GitLab and GitLab CI instance. +- have a working GitLab instance of version 8.0 or higher or are using + [GitLab.com](https://gitlab.com/users/sign_in) +- have a project in GitLab that you would like to use CI for -You can omit this step if you use [GitLab.com](https://GitLab.com/). +In brief, the steps needed to have a working CI can be summed up to: -## 2. Create repository on GitLab +1. Create a new project +1. Add `.gitlab-ci.yml` to the git repository and push to GitLab +1. Configure a Runner -Once you login on your GitLab add a new repository where you will store your source code. -Push your application to that repository. +From there on, on every push to your git repository the build will be +automagically started by the Runner and will appear under the project's +`/builds` page. -## 3. Add project to CI +Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -The next part is to login to GitLab CI. -Point your browser to the URL you have set GitLab or use [gitlab.com/ci](https://gitlab.com/ci/). +## 1. Creating a `.gitlab-ci.yml` file -On the first screen you will see a list of GitLab's projects that you have access to: + **GitLab CI** service is enabled automatically on the first push of a + `.gitlab-ci.yml` file in your repository and this is the recommended way. -![Projects](projects.png) +For other methods read [how to enable the GitLab CI service](../enable_ci.md). -Click **Add Project to CI**. -This will create project in CI and authorize GitLab CI to fetch sources from GitLab. +### What is `.gitlab-ci.yml` -> GitLab CI creates unique token that is used to configure GitLab CI service in GitLab. -> This token allows to access GitLab's repository and configures GitLab to trigger GitLab CI webhook on **Push events** and **Tag push events**. -> You can see that token by going to Project's Settings > Services > GitLab CI. -> You will see there token, the same token is assigned in GitLab CI settings of project. +The `.gitlab-ci.yml` file is where you configure what CI does with your project. +It lives in the root of your repository. -## 4. Create project's configuration - .gitlab-ci.yml +On any push to your repository, GitLab will look for the `.gitlab-ci.yml` +file and start builds on _Runners_ according to the contents of the file, +for that commit. -The next: You have to define how your project will be built. -GitLab CI uses [YAML](https://en.wikipedia.org/wiki/YAML) file to store build configuration. -You need to create `.gitlab-ci.yml` in root directory of your repository: +Because `.gitlab-ci.yml` is in the repository, it is version controlled, +old versions still build succesfully, forks can easily make use of CI, +branches can have separate builds and you have a single source of truth for CI. +You can read more about the reasons why we are using `.gitlab-ci.yml` +[in our blog about it][blog-ci]. + +`.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file. + +### Creating a simple `.gitlab-ci.yml` file + +You need to create a file named `.gitlab-ci.yml` in the root directory of your +repository. Below is an example for a Ruby on Rails project. ```yaml before_script: - - bundle install + - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs + - ruby -v + - which ruby + - gem install bundler --no-ri --no-rdoc + - bundle install --jobs $(nproc) "${FLAGS[@]}" rspec: script: @@ -49,71 +67,129 @@ rubocop: - bundle exec rubocop ``` -This is the simplest possible build configuration that will work for most Ruby applications: -1. Define two jobs `rspec` and `rubocop` with two different commands to be executed. -1. Before every job execute commands defined by `before_script`. +This is the simplest possible build configuration that will work for most Ruby +applications: + +1. Define two jobs `rspec` and `rubocop` with different commands to be executed. +1. Before every job, the commands defined by `before_script` are executed. + +The `.gitlab-ci.yml` file defines sets of jobs with constraints of how and when +they should be run. The jobs are defined as top-level elements with a name and +always have to contain the `script` name. Jobs are used to create builds, +which are then picked by [Runners](../runners/README.md) and executed within +the environment of the Runner. + +What is important is that each job is run independently from each other. -The `.gitlab-ci.yml` defines set of jobs with constrains how and when they should be run. -The jobs are defined as top-level elements with name and always have to contain the `script`. -Jobs are used to create builds, which are then picked by [runners](../runners/README.md) and executed within environment of the runner. -What is important that each job is run independently from each other. +If you want to check whether your `.gitlab-ci.yml` file is valid, there is a +Lint tool under the page `/ci/lint` of your GitLab instance. You can also find +the link under **Settings** -> **CI settings** in your project. -For more information and complete `.gitlab-ci.yml` syntax, please check the [Configuring project (.gitlab-ci.yml)](../yaml/README.md). +For more information and a complete `.gitlab-ci.yml` syntax, please check +[the documentation on .gitlab-ci.yml](../yaml/README.md). -## 5. Add file and push .gitlab-ci.yml to repository +### Push `.gitlab-ci.yml` to GitLab -Once you created `.gitlab-ci.yml` you should add it to git repository and push it to GitLab. +Once you've created `.gitlab-ci.yml`, you should add it to your git repository +and push it to GitLab. ```bash git add .gitlab-ci.yml -git commit +git commit -m "Add .gitlab-ci.yml" git push origin master ``` -If you refresh the project's page on GitLab CI you will notice a one new commit: +Now if you go to the **Builds** page you will see that the builds are pending. + +You can also go to the **Commits** page and notice the little clock icon next +to the commit SHA. + +![New commit pending](img/new_commit.png) + +Clicking on the clock icon you will be directed to the builds page for that +specific commit. + +![Single commit builds page](img/single_commit_status_pending.png) + +Notice that there are two jobs pending which are named after what we wrote in +`.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured +yet for these builds. + +The next step is to configure a Runner so that it picks the pending jobs. + +## 2. Configuring a Runner + +In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. +A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker +container or even a cluster of containers. GitLab and the Runners communicate +through an API, so the only needed requirement is that the machine on which the +Runner is configured to have Internet access. + +A Runner can be specific to a certain project or serve multiple projects in +GitLab. If it serves all projects it's called a _Shared Runner_. + +Find more information about different Runners in the +[Runners](../runners/README.md) documentation. + +You can find whether any Runners are assigned to your project by going to +**Settings** -> **Runners**. + +Setting up a Runner is easy and straightforward. The official Runner supported +by GitLab is written in Go and can be found at +. + +In order to have a functional Runner you need to: + +1. [Install it][runner-install] +2. [Configure it](../runners/README.md#registering-a-specific-runner) + +For other types of unofficial Runners written in other languages, see the +[instructions for the various GitLab Runners](https://about.gitlab.com/gitlab-ci/#gitlab-runner). + +Once the Runner has been set up, you should see it on the Runners page of your +project, following **Settings** -> **Runners**. -![](new_commit.png) +![Activated runners](img/runners_activated.png) -However the commit has status **pending** which means that commit was not yet picked by runner. +### Shared Runners -## 6. Configure runner +If you use [GitLab.com](https://gitlab.com/) you can use **Shared Runners** +provided by GitLab Inc. -In GitLab CI, Runners run your builds. -A runner is a machine (can be virtual, bare-metal or VPS) that picks up builds through the coordinator API of GitLab CI. +These are special virtual machines that are run on GitLab's infrastructure that +can build any project. -A runner can be specific to a certain project or serve any project in GitLab CI. -A runner that serves all projects is called a shared runner. -More information about different runner types can be found in [Configuring runner](../runners/README.md). +To enable **Shared Runners** you have to go to your project's +**Settings** -> **Runners** and click **Enable shared runners**. -To check if you have runners assigned to your project go to **Runners**. You will find there information how to setup project specific runner: +[Read more on Shared Runners](../runners/README.md). -1. Install GitLab Runner software. Checkout the [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner) section to install it. -1. Specify following URL during runner setup: https://gitlab.com/ci/ -1. Use the following registration token during setup: TOKEN +## 3. Seeing the status of your build -If you do it correctly your runner should be shown under **Runners activated for this project**: +After configuring the Runner succesfully, you should see the status of your +last commit change from _pending_ to either _running_, _success_ or _failed_. -![](runners_activated.png) +You can view all builds, by going to the **Builds** page in your project. -### Shared runners +![Commit status](img/builds_status.png) -If you use [gitlab.com/ci](https://gitlab.com/ci/) you can use **Shared runners** provided by GitLab Inc. -These are special virtual machines that are run on GitLab's infrastructure that can build any project. -To enable **Shared runners** you have to go to **Runners** and click **Enable shared runners** for this project. +By clicking on a Build ID, you will be able to see the log of that build. +This is important to diagnose why a build failed or acted differently than +you expected. -## 7. Check status of commit +![Build log](img/build_log.png) -If everything went OK and you go to commit, the status of the commit should change from **pending** to either **running**, **success** or **failed**. +You are also able to view the status of any commit in the various pages in +GitLab, such as **Commits** and **Merge Requests**. -![](commit_status.png) +## Next steps -You can click **Build ID** to view build log for specific job. +Awesome! You started using CI in GitLab! -## 8. Congratulations! +Next you can look into doing more with the CI. Many people are using GitLab +to package, containerize, test and deploy software. -You managed to build your first project using GitLab CI. -You may need to tune your `.gitlab-ci.yml` file to implement build plan for your project. -A few examples how it can be done you can find on [Examples](../examples/README.md) page. +We have a number of [examples](../examples/README.md) available. -GitLab CI also offers **the Lint** tool to verify validity of your `.gitlab-ci.yml` which can be useful to troubleshoot potential problems. -The Lint is available from project's settings or by adding `/lint` to GitLab CI url. +[runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation +[blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3dbf1afc7a9..9ee26c41e6d 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -20,6 +20,22 @@ Of course a command can execute code directly (`./configure;make;make install`) Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. What is important, is that each job is run independently from each other. +## Why `.gitlab-ci.yml` + +By placing a single configuration file in the root of your repository, +it is version controlled and you get all the advantages of git. + +In addition, builds for older versions of the repository will work just fine, +as GitLab look at the `.gitlab-ci.yml` of the pushed commit. +This means that forks also build without any problem. + +You can even set up different builds for different branches. This allows you +to only deploy the `production` branch, for instance. + +By having a single source of truth, everyone can view and contribute to the +stability of your CI builds, eventually improving the quality of your development +cycle. + ## .gitlab-ci.yml The YAML syntax allows for using more complex job specifications than in the above example: @@ -185,7 +201,7 @@ This are two parameters that allow for setting a refs policy to limit when jobs There are a few rules that apply to usage of refs policy: -1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. +1. `only` and `except` are exclusive. If both `only` and `except` are defined in job specification only `only` is taken into account. 1. `only` and `except` allow for using the regexp expressions. 1. `only` and `except` allow for using special keywords: `branches` and `tags`. These names can be used for example to exclude all tags and all branches. @@ -198,18 +214,6 @@ job: - branches # use special keyword ``` -1. `only` and `except` allow for specify repository path to filter jobs for forks. -The repository path can be used to have jobs executed only for parent repository. - -```yaml -job: - only: - - branches@gitlab-org/gitlab-ce - except: - - master@gitlab-org/gitlab-ce -``` -The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . - ### tags `tags` is used to select specific runners from the list of all runners that are allowed to run this project. -- cgit v1.2.1 From 2c30d11e80a4bb4112d01be7e1fbe51a321b3beb Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 5 Nov 2015 22:49:57 +0200 Subject: Clean up intro --- doc/ci/quick_start/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 74626c8f8a4..92dad2be3e8 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,8 +16,8 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the Runner and will appear under the project's -`/builds` page. +automagically started by the runner and will appear under the project's `/builds` +page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -- cgit v1.2.1 From 1058652a920d1106d29452aadfef953937a188e5 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 6 Nov 2015 10:43:28 +0200 Subject: Add section of enabling GitLab CI --- doc/ci/quick_start/README.md | 7 +++++++ doc/ci/quick_start/builds_tab.png | Bin 0 -> 3845 bytes doc/ci/quick_start/ci_service_disabled.png | Bin 0 -> 3598 bytes doc/ci/quick_start/ci_service_enabled.png | Bin 0 -> 3545 bytes doc/ci/quick_start/ci_service_mark_active.png | Bin 0 -> 17193 bytes doc/ci/quick_start/enable_ci.md | 24 ++++++++++++++++++++++++ 6 files changed, 31 insertions(+) create mode 100644 doc/ci/quick_start/builds_tab.png create mode 100644 doc/ci/quick_start/ci_service_disabled.png create mode 100644 doc/ci/quick_start/ci_service_enabled.png create mode 100644 doc/ci/quick_start/ci_service_mark_active.png create mode 100644 doc/ci/quick_start/enable_ci.md (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 92dad2be3e8..d45ad3e5e23 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -21,6 +21,13 @@ page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. +## 1. Enable GitLab CI + +After creating a new project, the first thing to do is enable the **GitLab CI** +service in your project's settings if it isn't already enabled. + +Read [how to enable the GitLab CI service](enable_ci.md). + ## 1. Creating a `.gitlab-ci.yml` file **GitLab CI** service is enabled automatically on the first push of a diff --git a/doc/ci/quick_start/builds_tab.png b/doc/ci/quick_start/builds_tab.png new file mode 100644 index 00000000000..d088b8b329d Binary files /dev/null and b/doc/ci/quick_start/builds_tab.png differ diff --git a/doc/ci/quick_start/ci_service_disabled.png b/doc/ci/quick_start/ci_service_disabled.png new file mode 100644 index 00000000000..351de4267a4 Binary files /dev/null and b/doc/ci/quick_start/ci_service_disabled.png differ diff --git a/doc/ci/quick_start/ci_service_enabled.png b/doc/ci/quick_start/ci_service_enabled.png new file mode 100644 index 00000000000..dfe7488c1ba Binary files /dev/null and b/doc/ci/quick_start/ci_service_enabled.png differ diff --git a/doc/ci/quick_start/ci_service_mark_active.png b/doc/ci/quick_start/ci_service_mark_active.png new file mode 100644 index 00000000000..8bc8694cec3 Binary files /dev/null and b/doc/ci/quick_start/ci_service_mark_active.png differ diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md new file mode 100644 index 00000000000..6f539c752d8 --- /dev/null +++ b/doc/ci/quick_start/enable_ci.md @@ -0,0 +1,24 @@ +# Enable GitLab CI + +GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You +only need to enable it in the **Services** settings of your project. + +First, head over your project's page that you would like to enable CI for. +If you can see the **Builds** tab in the sidebar, then CI is enabled. + +![Builds tab](builds_tab.png) + +If not, go to **Settings > Services** and search for **GitLab CI**. Its state +should be disabled. + +![CI service disabled](ci_service_disabled.png) + +Click on **GitLab CI** to enter its settings, mark it as active and hit +**Save**. + +![Mark CI service as active](ci_service_mark_active.png) + +Do you see that green dot? Then good, the service is now enabled! You can also +check its status under **Services**. + +![CI service enabled](ci_service_enabled.png) -- cgit v1.2.1 From 3b4806bad95e55a40d3efe02e329d271995d8a70 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Sat, 7 Nov 2015 09:11:37 +0200 Subject: Add option to enable GitLab CI via .gitlab-ci.yml --- doc/ci/quick_start/enable_ci.md | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md index 6f539c752d8..42635e35d5d 100644 --- a/doc/ci/quick_start/enable_ci.md +++ b/doc/ci/quick_start/enable_ci.md @@ -1,15 +1,30 @@ # Enable GitLab CI -GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You -only need to enable it in the **Services** settings of your project. +GitLab Continuous Integration (CI) is fully integrated into GitLab itself as +of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). -First, head over your project's page that you would like to enable CI for. -If you can see the **Builds** tab in the sidebar, then CI is enabled. +First, head over your project's page that you would like to enable GitLab CI +for. If you can see the **Builds** tab in the sidebar, then GitLab CI is +already enabled and you can skip reading the rest of this guide. ![Builds tab](builds_tab.png) -If not, go to **Settings > Services** and search for **GitLab CI**. Its state -should be disabled. +If not, there are two ways to enable it in your project. + +## Use .gitlab-ci.yml to enable GitLab CI + +GitLab CI will be automatically enabled if you just push a +[`.gitlab-ci.yml`](../yaml/README.md) in your git repository. GitLab will +pick up the change immediately and GitLab CI will be enabled for this project. +This is the recommended way. + +## Manually enable GitLab CI + +The second way is to manually enable it in the project's **Services** settings +and this is also the way to disable it if needed. + +Go to **Settings > Services** and search for **GitLab CI**. Its state should +be disabled. ![CI service disabled](ci_service_disabled.png) -- cgit v1.2.1 From 32a3c3e068e957716eac56a755f44268c07c5957 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 03:26:51 +0200 Subject: Place images in their own dir --- doc/ci/enable_ci.md | 39 ++++++++++++++++++++++++++ doc/ci/img/builds_tab.png | Bin 0 -> 3845 bytes doc/ci/img/ci_service_disabled.png | Bin 0 -> 3598 bytes doc/ci/img/ci_service_enabled.png | Bin 0 -> 3545 bytes doc/ci/img/ci_service_mark_active.png | Bin 0 -> 17193 bytes doc/ci/quick_start/README.md | 14 ++++----- doc/ci/quick_start/build_status.png | Bin 62140 -> 0 bytes doc/ci/quick_start/builds_tab.png | Bin 3845 -> 0 bytes doc/ci/quick_start/ci_service_disabled.png | Bin 3598 -> 0 bytes doc/ci/quick_start/ci_service_enabled.png | Bin 3545 -> 0 bytes doc/ci/quick_start/ci_service_mark_active.png | Bin 17193 -> 0 bytes doc/ci/quick_start/commit_status.png | Bin 33492 -> 0 bytes doc/ci/quick_start/enable_ci.md | 39 -------------------------- doc/ci/quick_start/img/build_status.png | Bin 0 -> 62140 bytes doc/ci/quick_start/img/commit_status.png | Bin 0 -> 33492 bytes doc/ci/quick_start/img/new_commit.png | Bin 0 -> 47527 bytes doc/ci/quick_start/img/projects.png | Bin 0 -> 37014 bytes doc/ci/quick_start/img/runners.png | Bin 0 -> 123048 bytes doc/ci/quick_start/img/runners_activated.png | Bin 0 -> 60769 bytes doc/ci/quick_start/new_commit.png | Bin 47527 -> 0 bytes doc/ci/quick_start/projects.png | Bin 37014 -> 0 bytes doc/ci/quick_start/runners.png | Bin 123048 -> 0 bytes doc/ci/quick_start/runners_activated.png | Bin 60769 -> 0 bytes 23 files changed, 45 insertions(+), 47 deletions(-) create mode 100644 doc/ci/enable_ci.md create mode 100644 doc/ci/img/builds_tab.png create mode 100644 doc/ci/img/ci_service_disabled.png create mode 100644 doc/ci/img/ci_service_enabled.png create mode 100644 doc/ci/img/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/build_status.png delete mode 100644 doc/ci/quick_start/builds_tab.png delete mode 100644 doc/ci/quick_start/ci_service_disabled.png delete mode 100644 doc/ci/quick_start/ci_service_enabled.png delete mode 100644 doc/ci/quick_start/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/commit_status.png delete mode 100644 doc/ci/quick_start/enable_ci.md create mode 100644 doc/ci/quick_start/img/build_status.png create mode 100644 doc/ci/quick_start/img/commit_status.png create mode 100644 doc/ci/quick_start/img/new_commit.png create mode 100644 doc/ci/quick_start/img/projects.png create mode 100644 doc/ci/quick_start/img/runners.png create mode 100644 doc/ci/quick_start/img/runners_activated.png delete mode 100644 doc/ci/quick_start/new_commit.png delete mode 100644 doc/ci/quick_start/projects.png delete mode 100644 doc/ci/quick_start/runners.png delete mode 100644 doc/ci/quick_start/runners_activated.png (limited to 'doc/ci') diff --git a/doc/ci/enable_ci.md b/doc/ci/enable_ci.md new file mode 100644 index 00000000000..01c684dabec --- /dev/null +++ b/doc/ci/enable_ci.md @@ -0,0 +1,39 @@ +# Enable GitLab CI + +GitLab Continuous Integration (CI) is fully integrated into GitLab itself as +of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). + +First, head over your project's page that you would like to enable GitLab CI +for. If you can see the **Builds** tab in the sidebar, then GitLab CI is +already enabled and you can skip reading the rest of this guide. + +![Builds tab](img/builds_tab.png) + +If not, there are two ways to enable it in your project. + +## Use .gitlab-ci.yml to enable GitLab CI + +GitLab CI will be automatically enabled if you just push a +[`.gitlab-ci.yml`](yaml/README.md) in your git repository. GitLab will +pick up the change immediately and GitLab CI will be enabled for this project. +This is the recommended way. + +## Manually enable GitLab CI + +The second way is to manually enable it in the project's **Services** settings +and this is also the way to disable it if needed. + +Go to **Settings > Services** and search for **GitLab CI**. Its state should +be disabled. + +![CI service disabled](img/ci_service_disabled.png) + +Click on **GitLab CI** to enter its settings, mark it as active and hit +**Save**. + +![Mark CI service as active](img/ci_service_mark_active.png) + +Do you see that green dot? Then good, the service is now enabled! You can also +check its status under **Services**. + +![CI service enabled](img/ci_service_enabled.png) diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png new file mode 100644 index 00000000000..d088b8b329d Binary files /dev/null and b/doc/ci/img/builds_tab.png differ diff --git a/doc/ci/img/ci_service_disabled.png b/doc/ci/img/ci_service_disabled.png new file mode 100644 index 00000000000..351de4267a4 Binary files /dev/null and b/doc/ci/img/ci_service_disabled.png differ diff --git a/doc/ci/img/ci_service_enabled.png b/doc/ci/img/ci_service_enabled.png new file mode 100644 index 00000000000..dfe7488c1ba Binary files /dev/null and b/doc/ci/img/ci_service_enabled.png differ diff --git a/doc/ci/img/ci_service_mark_active.png b/doc/ci/img/ci_service_mark_active.png new file mode 100644 index 00000000000..8bc8694cec3 Binary files /dev/null and b/doc/ci/img/ci_service_mark_active.png differ diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index d45ad3e5e23..ce05a6f417c 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,19 +16,17 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the runner and will appear under the project's `/builds` -page. +automagically started by the runner and will appear under the project's +`/builds` page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -## 1. Enable GitLab CI - -After creating a new project, the first thing to do is enable the **GitLab CI** -service in your project's settings if it isn't already enabled. +## 1. Creating a `.gitlab-ci.yml` file -Read [how to enable the GitLab CI service](enable_ci.md). + **GitLab CI** service is enabled automatically on the first push of a + `.gitlab-ci.yml` file in your repository and this is the recommended way. -## 1. Creating a `.gitlab-ci.yml` file +For other methods read [how to enable the GitLab CI service](../enable_ci.md). **GitLab CI** service is enabled automatically on the first push of a `.gitlab-ci.yml` file in your repository and this is the recommended way. diff --git a/doc/ci/quick_start/build_status.png b/doc/ci/quick_start/build_status.png deleted file mode 100644 index 333259e6acd..00000000000 Binary files a/doc/ci/quick_start/build_status.png and /dev/null differ diff --git a/doc/ci/quick_start/builds_tab.png b/doc/ci/quick_start/builds_tab.png deleted file mode 100644 index d088b8b329d..00000000000 Binary files a/doc/ci/quick_start/builds_tab.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_disabled.png b/doc/ci/quick_start/ci_service_disabled.png deleted file mode 100644 index 351de4267a4..00000000000 Binary files a/doc/ci/quick_start/ci_service_disabled.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_enabled.png b/doc/ci/quick_start/ci_service_enabled.png deleted file mode 100644 index dfe7488c1ba..00000000000 Binary files a/doc/ci/quick_start/ci_service_enabled.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_mark_active.png b/doc/ci/quick_start/ci_service_mark_active.png deleted file mode 100644 index 8bc8694cec3..00000000000 Binary files a/doc/ci/quick_start/ci_service_mark_active.png and /dev/null differ diff --git a/doc/ci/quick_start/commit_status.png b/doc/ci/quick_start/commit_status.png deleted file mode 100644 index 725b79e6f91..00000000000 Binary files a/doc/ci/quick_start/commit_status.png and /dev/null differ diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md deleted file mode 100644 index 42635e35d5d..00000000000 --- a/doc/ci/quick_start/enable_ci.md +++ /dev/null @@ -1,39 +0,0 @@ -# Enable GitLab CI - -GitLab Continuous Integration (CI) is fully integrated into GitLab itself as -of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). - -First, head over your project's page that you would like to enable GitLab CI -for. If you can see the **Builds** tab in the sidebar, then GitLab CI is -already enabled and you can skip reading the rest of this guide. - -![Builds tab](builds_tab.png) - -If not, there are two ways to enable it in your project. - -## Use .gitlab-ci.yml to enable GitLab CI - -GitLab CI will be automatically enabled if you just push a -[`.gitlab-ci.yml`](../yaml/README.md) in your git repository. GitLab will -pick up the change immediately and GitLab CI will be enabled for this project. -This is the recommended way. - -## Manually enable GitLab CI - -The second way is to manually enable it in the project's **Services** settings -and this is also the way to disable it if needed. - -Go to **Settings > Services** and search for **GitLab CI**. Its state should -be disabled. - -![CI service disabled](ci_service_disabled.png) - -Click on **GitLab CI** to enter its settings, mark it as active and hit -**Save**. - -![Mark CI service as active](ci_service_mark_active.png) - -Do you see that green dot? Then good, the service is now enabled! You can also -check its status under **Services**. - -![CI service enabled](ci_service_enabled.png) diff --git a/doc/ci/quick_start/img/build_status.png b/doc/ci/quick_start/img/build_status.png new file mode 100644 index 00000000000..333259e6acd Binary files /dev/null and b/doc/ci/quick_start/img/build_status.png differ diff --git a/doc/ci/quick_start/img/commit_status.png b/doc/ci/quick_start/img/commit_status.png new file mode 100644 index 00000000000..725b79e6f91 Binary files /dev/null and b/doc/ci/quick_start/img/commit_status.png differ diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png new file mode 100644 index 00000000000..3839e893c17 Binary files /dev/null and b/doc/ci/quick_start/img/new_commit.png differ diff --git a/doc/ci/quick_start/img/projects.png b/doc/ci/quick_start/img/projects.png new file mode 100644 index 00000000000..0b3430a69db Binary files /dev/null and b/doc/ci/quick_start/img/projects.png differ diff --git a/doc/ci/quick_start/img/runners.png b/doc/ci/quick_start/img/runners.png new file mode 100644 index 00000000000..25b4046bc00 Binary files /dev/null and b/doc/ci/quick_start/img/runners.png differ diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png new file mode 100644 index 00000000000..c934bd12f41 Binary files /dev/null and b/doc/ci/quick_start/img/runners_activated.png differ diff --git a/doc/ci/quick_start/new_commit.png b/doc/ci/quick_start/new_commit.png deleted file mode 100644 index 3839e893c17..00000000000 Binary files a/doc/ci/quick_start/new_commit.png and /dev/null differ diff --git a/doc/ci/quick_start/projects.png b/doc/ci/quick_start/projects.png deleted file mode 100644 index 0b3430a69db..00000000000 Binary files a/doc/ci/quick_start/projects.png and /dev/null differ diff --git a/doc/ci/quick_start/runners.png b/doc/ci/quick_start/runners.png deleted file mode 100644 index 25b4046bc00..00000000000 Binary files a/doc/ci/quick_start/runners.png and /dev/null differ diff --git a/doc/ci/quick_start/runners_activated.png b/doc/ci/quick_start/runners_activated.png deleted file mode 100644 index c934bd12f41..00000000000 Binary files a/doc/ci/quick_start/runners_activated.png and /dev/null differ -- cgit v1.2.1 From e70fb9cf75d08e8439d94637726dbee7e2324ab6 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 03:49:32 +0200 Subject: New images for the commit build status --- doc/ci/quick_start/README.md | 2 -- doc/ci/quick_start/img/new_commit.png | Bin 47527 -> 9033 bytes .../quick_start/img/single_commit_status_pending.png | Bin 0 -> 36431 bytes doc/ci/quick_start/img/status_pending.png | Bin 0 -> 19782 bytes 4 files changed, 2 deletions(-) create mode 100644 doc/ci/quick_start/img/single_commit_status_pending.png create mode 100644 doc/ci/quick_start/img/status_pending.png (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index ce05a6f417c..073146b9d15 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -120,8 +120,6 @@ Notice that there are two jobs pending which are named after what we wrote in `.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured yet for these builds. -The next step is to configure a Runner so that it picks the pending jobs. - ## 2. Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png index 3839e893c17..3d3c9d5c0bd 100644 Binary files a/doc/ci/quick_start/img/new_commit.png and b/doc/ci/quick_start/img/new_commit.png differ diff --git a/doc/ci/quick_start/img/single_commit_status_pending.png b/doc/ci/quick_start/img/single_commit_status_pending.png new file mode 100644 index 00000000000..23b3bb5acfc Binary files /dev/null and b/doc/ci/quick_start/img/single_commit_status_pending.png differ diff --git a/doc/ci/quick_start/img/status_pending.png b/doc/ci/quick_start/img/status_pending.png new file mode 100644 index 00000000000..a049ec2a5ba Binary files /dev/null and b/doc/ci/quick_start/img/status_pending.png differ -- cgit v1.2.1 From 11b204f1312c3d6aa579b7a04cb4bcf794ad5239 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 21:16:12 +0200 Subject: Final touches for the quick start quide --- doc/ci/quick_start/README.md | 6 ++++-- doc/ci/quick_start/img/build_log.png | Bin 0 -> 63272 bytes doc/ci/quick_start/img/builds_status.png | Bin 0 -> 49121 bytes doc/ci/quick_start/img/runners_activated.png | Bin 60769 -> 27597 bytes 4 files changed, 4 insertions(+), 2 deletions(-) create mode 100644 doc/ci/quick_start/img/build_log.png create mode 100644 doc/ci/quick_start/img/builds_status.png (limited to 'doc/ci') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 073146b9d15..7771d78d91f 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,7 +16,7 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the runner and will appear under the project's +automagically started by the Runner and will appear under the project's `/builds` page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. @@ -120,6 +120,8 @@ Notice that there are two jobs pending which are named after what we wrote in `.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured yet for these builds. +The next step is to configure a Runner so that it picks the pending jobs. + ## 2. Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. @@ -136,8 +138,8 @@ Find more information about different Runners in the You can find whether any Runners are assigned to your project by going to **Settings** -> **Runners**. - Setting up a Runner is easy and straightforward. The official Runner supported + by GitLab is written in Go and can be found at . diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png new file mode 100644 index 00000000000..89e6cd40cb6 Binary files /dev/null and b/doc/ci/quick_start/img/build_log.png differ diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png new file mode 100644 index 00000000000..b8e6c2a361a Binary files /dev/null and b/doc/ci/quick_start/img/builds_status.png differ diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png index c934bd12f41..eafcfd6ecd5 100644 Binary files a/doc/ci/quick_start/img/runners_activated.png and b/doc/ci/quick_start/img/runners_activated.png differ -- cgit v1.2.1 From aea5b72faafa6911377cb0c9fc0801269ac59e1e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 11 Nov 2015 09:57:21 +0200 Subject: Remove old images --- doc/ci/quick_start/img/build_status.png | Bin 62140 -> 0 bytes doc/ci/quick_start/img/commit_status.png | Bin 33492 -> 0 bytes doc/ci/quick_start/img/projects.png | Bin 37014 -> 0 bytes 3 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 doc/ci/quick_start/img/build_status.png delete mode 100644 doc/ci/quick_start/img/commit_status.png delete mode 100644 doc/ci/quick_start/img/projects.png (limited to 'doc/ci') diff --git a/doc/ci/quick_start/img/build_status.png b/doc/ci/quick_start/img/build_status.png deleted file mode 100644 index 333259e6acd..00000000000 Binary files a/doc/ci/quick_start/img/build_status.png and /dev/null differ diff --git a/doc/ci/quick_start/img/commit_status.png b/doc/ci/quick_start/img/commit_status.png deleted file mode 100644 index 725b79e6f91..00000000000 Binary files a/doc/ci/quick_start/img/commit_status.png and /dev/null differ diff --git a/doc/ci/quick_start/img/projects.png b/doc/ci/quick_start/img/projects.png deleted file mode 100644 index 0b3430a69db..00000000000 Binary files a/doc/ci/quick_start/img/projects.png and /dev/null differ -- cgit v1.2.1 From 6805f1bc47640886020e5febf015a73e5862114c Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 1 Dec 2015 10:05:24 +0200 Subject: Clean up quick start quide [ci skip] * Remove references to enabling CI since it it will be deprecated * Revert changes in yaml/README.md, refine it in a separate MR --- doc/ci/enable_ci.md | 39 ---------------------- doc/ci/img/ci_service_disabled.png | Bin 3598 -> 0 bytes doc/ci/img/ci_service_enabled.png | Bin 3545 -> 0 bytes doc/ci/img/ci_service_mark_active.png | Bin 17193 -> 0 bytes doc/ci/quick_start/README.md | 61 ++++++++++++++++------------------ doc/ci/quick_start/img/runners.png | Bin 123048 -> 0 bytes doc/ci/yaml/README.md | 30 ++++++++--------- 7 files changed, 42 insertions(+), 88 deletions(-) delete mode 100644 doc/ci/enable_ci.md delete mode 100644 doc/ci/img/ci_service_disabled.png delete mode 100644 doc/ci/img/ci_service_enabled.png delete mode 100644 doc/ci/img/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/img/runners.png (limited to 'doc/ci') diff --git a/doc/ci/enable_ci.md b/doc/ci/enable_ci.md deleted file mode 100644 index 01c684dabec..00000000000 --- a/doc/ci/enable_ci.md +++ /dev/null @@ -1,39 +0,0 @@ -# Enable GitLab CI - -GitLab Continuous Integration (CI) is fully integrated into GitLab itself as -of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). - -First, head over your project's page that you would like to enable GitLab CI -for. If you can see the **Builds** tab in the sidebar, then GitLab CI is -already enabled and you can skip reading the rest of this guide. - -![Builds tab](img/builds_tab.png) - -If not, there are two ways to enable it in your project. - -## Use .gitlab-ci.yml to enable GitLab CI - -GitLab CI will be automatically enabled if you just push a -[`.gitlab-ci.yml`](yaml/README.md) in your git repository. GitLab will -pick up the change immediately and GitLab CI will be enabled for this project. -This is the recommended way. - -## Manually enable GitLab CI - -The second way is to manually enable it in the project's **Services** settings -and this is also the way to disable it if needed. - -Go to **Settings > Services** and search for **GitLab CI**. Its state should -be disabled. - -![CI service disabled](img/ci_service_disabled.png) - -Click on **GitLab CI** to enter its settings, mark it as active and hit -**Save**. - -![Mark CI service as active](img/ci_service_mark_active.png) - -Do you see that green dot? Then good, the service is now enabled! You can also -check its status under **Services**. - -![CI service enabled](img/ci_service_enabled.png) diff --git a/doc/ci/img/ci_service_disabled.png b/doc/ci/img/ci_service_disabled.png deleted file mode 100644 index 351de4267a4..00000000000 Binary files a/doc/ci/img/ci_service_disabled.png and /dev/null differ diff --git a/doc/ci/img/ci_service_enabled.png b/doc/ci/img/ci_service_enabled.png deleted file mode 100644 index dfe7488c1ba..00000000000 Binary files a/doc/ci/img/ci_service_enabled.png and /dev/null differ diff --git a/doc/ci/img/ci_service_mark_active.png b/doc/ci/img/ci_service_mark_active.png deleted file mode 100644 index 8bc8694cec3..00000000000 Binary files a/doc/ci/img/ci_service_mark_active.png and /dev/null differ diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 7771d78d91f..a9b36139de9 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,7 +1,7 @@ # Quick Start -GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You -only need to enable it in the Services settings of your project. +Starting from version 8.0, GitLab Continuous Integration (CI) is fully +integrated into GitLab itself and is enabled by default on all projects. This guide assumes that you: @@ -21,17 +21,10 @@ automagically started by the Runner and will appear under the project's Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -## 1. Creating a `.gitlab-ci.yml` file +## Creating a `.gitlab-ci.yml` file - **GitLab CI** service is enabled automatically on the first push of a - `.gitlab-ci.yml` file in your repository and this is the recommended way. - -For other methods read [how to enable the GitLab CI service](../enable_ci.md). - - **GitLab CI** service is enabled automatically on the first push of a - `.gitlab-ci.yml` file in your repository and this is the recommended way. - -For other methods read [how to enable the GitLab CI service](../enable_ci.md). +Before you create `.gitlab-ci.yml` let's first explain in brief what this is +all about. ### What is `.gitlab-ci.yml` @@ -48,7 +41,9 @@ branches can have separate builds and you have a single source of truth for CI. You can read more about the reasons why we are using `.gitlab-ci.yml` [in our blog about it][blog-ci]. -`.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file. +**Note:** `.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file +so you have to pay extra attention to the identation. Always use spaces, not +tabs. ### Creating a simple `.gitlab-ci.yml` file @@ -75,20 +70,21 @@ rubocop: This is the simplest possible build configuration that will work for most Ruby applications: -1. Define two jobs `rspec` and `rubocop` with different commands to be executed. +1. Define two jobs `rspec` and `rubocop` (the names are arbitrary) with + different commands to be executed. 1. Before every job, the commands defined by `before_script` are executed. The `.gitlab-ci.yml` file defines sets of jobs with constraints of how and when -they should be run. The jobs are defined as top-level elements with a name and -always have to contain the `script` name. Jobs are used to create builds, -which are then picked by [Runners](../runners/README.md) and executed within -the environment of the Runner. +they should be run. The jobs are defined as top-level elements with a name (in +our case `rspec` and `rubocop`) and always have to contain the `script` keyword. +Jobs are used to create builds, which are then picked by +[Runners](../runners/README.md) and executed within the environment of the Runner. What is important is that each job is run independently from each other. If you want to check whether your `.gitlab-ci.yml` file is valid, there is a Lint tool under the page `/ci/lint` of your GitLab instance. You can also find -the link under **Settings** -> **CI settings** in your project. +the link under **Settings > CI settings** in your project. For more information and a complete `.gitlab-ci.yml` syntax, please check [the documentation on .gitlab-ci.yml](../yaml/README.md). @@ -122,13 +118,13 @@ yet for these builds. The next step is to configure a Runner so that it picks the pending jobs. -## 2. Configuring a Runner +## Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker container or even a cluster of containers. GitLab and the Runners communicate through an API, so the only needed requirement is that the machine on which the -Runner is configured to have Internet access. +Runner is configured to has Internet access. A Runner can be specific to a certain project or serve multiple projects in GitLab. If it serves all projects it's called a _Shared Runner_. @@ -137,22 +133,23 @@ Find more information about different Runners in the [Runners](../runners/README.md) documentation. You can find whether any Runners are assigned to your project by going to -**Settings** -> **Runners**. -Setting up a Runner is easy and straightforward. The official Runner supported - -by GitLab is written in Go and can be found at +**Settings > Runners**. Setting up a Runner is easy and straightforward. The +official Runner supported by GitLab is written in Go and can be found at . -In order to have a functional Runner you need to: +In order to have a functional Runner you need to follow two steps: 1. [Install it][runner-install] 2. [Configure it](../runners/README.md#registering-a-specific-runner) +Follow the links above to set up your own Runner or use a Shared Runner as +described in the next section. + For other types of unofficial Runners written in other languages, see the [instructions for the various GitLab Runners](https://about.gitlab.com/gitlab-ci/#gitlab-runner). Once the Runner has been set up, you should see it on the Runners page of your -project, following **Settings** -> **Runners**. +project, following **Settings > Runners**. ![Activated runners](img/runners_activated.png) @@ -161,15 +158,15 @@ project, following **Settings** -> **Runners**. If you use [GitLab.com](https://gitlab.com/) you can use **Shared Runners** provided by GitLab Inc. -These are special virtual machines that are run on GitLab's infrastructure that -can build any project. +These are special virtual machines that run on GitLab's infrastructure and can +build any project. To enable **Shared Runners** you have to go to your project's -**Settings** -> **Runners** and click **Enable shared runners**. +**Settings > Runners** and click **Enable shared runners**. [Read more on Shared Runners](../runners/README.md). -## 3. Seeing the status of your build +## Seeing the status of your build After configuring the Runner succesfully, you should see the status of your last commit change from _pending_ to either _running_, _success_ or _failed_. @@ -194,7 +191,7 @@ Awesome! You started using CI in GitLab! Next you can look into doing more with the CI. Many people are using GitLab to package, containerize, test and deploy software. -We have a number of [examples](../examples/README.md) available. +Visit our various languages examples at . [runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation [blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ diff --git a/doc/ci/quick_start/img/runners.png b/doc/ci/quick_start/img/runners.png deleted file mode 100644 index 25b4046bc00..00000000000 Binary files a/doc/ci/quick_start/img/runners.png and /dev/null differ diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 9ee26c41e6d..3dbf1afc7a9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -20,22 +20,6 @@ Of course a command can execute code directly (`./configure;make;make install`) Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. What is important, is that each job is run independently from each other. -## Why `.gitlab-ci.yml` - -By placing a single configuration file in the root of your repository, -it is version controlled and you get all the advantages of git. - -In addition, builds for older versions of the repository will work just fine, -as GitLab look at the `.gitlab-ci.yml` of the pushed commit. -This means that forks also build without any problem. - -You can even set up different builds for different branches. This allows you -to only deploy the `production` branch, for instance. - -By having a single source of truth, everyone can view and contribute to the -stability of your CI builds, eventually improving the quality of your development -cycle. - ## .gitlab-ci.yml The YAML syntax allows for using more complex job specifications than in the above example: @@ -201,7 +185,7 @@ This are two parameters that allow for setting a refs policy to limit when jobs There are a few rules that apply to usage of refs policy: -1. `only` and `except` are exclusive. If both `only` and `except` are defined in job specification only `only` is taken into account. +1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. 1. `only` and `except` allow for using the regexp expressions. 1. `only` and `except` allow for using special keywords: `branches` and `tags`. These names can be used for example to exclude all tags and all branches. @@ -214,6 +198,18 @@ job: - branches # use special keyword ``` +1. `only` and `except` allow for specify repository path to filter jobs for forks. +The repository path can be used to have jobs executed only for parent repository. + +```yaml +job: + only: + - branches@gitlab-org/gitlab-ce + except: + - master@gitlab-org/gitlab-ce +``` +The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . + ### tags `tags` is used to select specific runners from the list of all runners that are allowed to run this project. -- cgit v1.2.1 From 387e5656a1e158eaa1c010f22d332d758b336179 Mon Sep 17 00:00:00 2001 From: Kevin Pankonen Date: Thu, 3 Dec 2015 15:40:08 -0700 Subject: fixes #3263 slashes are replaced with two underscores --- doc/ci/docker/using_docker_images.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index ef8a7ec1e86..64e52eba3a2 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -60,11 +60,11 @@ This is image that have fully preconfigured `wordpress` and have `MySQL` server ``` Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum_wordpress`. +and you will have access to it from your build container under hostname: `tutum__wordpress`. Alias hostname for the service is made from the image name: 1. Everything after `:` is stripped, -2. '/' is replaced to `_`. +2. '/' is replaced with `__`. ### Configuring services Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. -- cgit v1.2.1 From 8a0507d523293289ae4b7d61bb3fce8a873e3dcc Mon Sep 17 00:00:00 2001 From: "Michael A. Smith" Date: Mon, 7 Dec 2015 10:29:42 -0500 Subject: Update Docker Syntax --- doc/ci/docker/using_docker_images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 64e52eba3a2..1feae62b1c7 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -190,7 +190,7 @@ This will create two service containers (MySQL and PostgreSQL). 1. Create a build container and execute script in its context: ``` -$ cat build_script | docker run -n build -i -l mysql:service-mysql -l postgres:service-postgres ruby:2.1 /bin/bash +$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` This will create build container that has two service containers linked. The build_script is piped using STDIN to bash interpreter which executes the build script in container. -- cgit v1.2.1 From 4f074aaa14faa8a866f18a80f58b66cd023a141f Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Wed, 25 Nov 2015 14:41:14 +0100 Subject: Introduce CI documentation for services and languages --- doc/ci/README.md | 12 +++ doc/ci/docker/using_docker_images.md | 115 +++++++++------------- doc/ci/languages/README.md | 3 + doc/ci/languages/php.md | 178 +++++++++++++++++++++++++++++++++++ doc/ci/services/README.md | 6 ++ doc/ci/services/docker-services.md | 5 + doc/ci/services/mysql.md | 72 ++++++++++++++ doc/ci/services/postgres.md | 70 ++++++++++++++ doc/ci/services/redis.md | 40 ++++++++ doc/ci/ssh_keys/README.md | 114 ++++++++++++++++++++++ 10 files changed, 547 insertions(+), 68 deletions(-) create mode 100644 doc/ci/languages/README.md create mode 100644 doc/ci/languages/php.md create mode 100644 doc/ci/services/README.md create mode 100644 doc/ci/services/docker-services.md create mode 100644 doc/ci/services/mysql.md create mode 100644 doc/ci/services/postgres.md create mode 100644 doc/ci/services/redis.md create mode 100644 doc/ci/ssh_keys/README.md (limited to 'doc/ci') diff --git a/doc/ci/README.md b/doc/ci/README.md index 97325069ceb..ae921b6f988 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -9,6 +9,18 @@ + [Using Docker Images](docker/using_docker_images.md) + [Using Docker Build](docker/using_docker_build.md) + [Using Variables](variables/README.md) ++ [Using SSH keys](ssh_keys/README.md) + +### Languages + ++ [Testing PHP](languages/php.md) + +### Services + ++ [Using MySQL](services/mysql.md) ++ [Using PostgreSQL](services/postgres.md) ++ [Using Redis](services/redis.md) ++ [Using Other Services](docker/using_docker_images.html#how-to-use-other-images-as-services) ### Examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 1feae62b1c7..2f0ca19cd0f 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -42,7 +42,44 @@ So, **to access your database service you have to connect to host: `mysql` inste ### How to use other images as services? You are not limited to have only database services. -You can hand modify `config.toml` to add any image as service found at [Docker Hub](https://registry.hub.docker.com/). +You can add the services to `.gitlab-ci.yml` or hand modify the `config.toml`. +You can use any image as service found at [Docker Hub](https://registry.hub.docker.com/). + +### Define image and services from `.gitlab-ci.yml` +You can simply define image or list services that you want to use for the build time. +``` +image: ruby:2.2 +services: + - postgres:9.3 +before_install: + - bundle install + +test: + script: + - bundle exec rake spec +``` + +It's possible to define image and service per-job: +``` +before_install: + - bundle install + +test:2.1: + image: ruby:2.1 + services: + - postgres:9.3 + script: + - bundle exec rake spec + +test:2.2: + image: ruby:2.2 + services: + - postgres:9.4 + script: + - bundle exec rake spec +``` + +### Define image and services in `config.toml` Look for `[runners.docker]` section: ``` [runners.docker] @@ -50,13 +87,16 @@ Look for `[runners.docker]` section: services = ["mysql:latest", "postgres:latest"] ``` +The image and services defined these way will be added to all builds run by that runner. + +### Accessing the services For example you need `wordpress` instance to test some API integration with `Wordpress`. -You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). -This is image that have fully preconfigured `wordpress` and have `MySQL` server built-in: +You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). + ``` -[runners.docker] - image = "ruby:2.1" - services = ["mysql:latest", "postgres:latest", "tutum/wordpress:latest"] +# .gitlab-ci.yml +services: +- tutum/wordpress:latest ``` Next time when you run your application the `tutum/wordpress` will be started @@ -64,7 +104,7 @@ and you will have access to it from your build container under hostname: `tutum_ Alias hostname for the service is made from the image name: 1. Everything after `:` is stripped, -2. '/' is replaced with `__`. +2. '/' is replaced to `__`. ### Configuring services Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. @@ -99,67 +139,6 @@ or README page for any other Docker image. **Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** -### Overwrite image and services -It's possible to overwrite `docker-image` and specify services from `.gitlab-ci.yml`. -If you add to your YAML the `image` and the `services` these parameters -be used instead of the ones that were specified during runner's registration. -``` -image: ruby:2.2 -services: - - postgres:9.3 -before_install: - - bundle install - -test: - script: - - bundle exec rake spec -``` - -It's possible to define image and service per-job: -``` -before_install: - - bundle install - -test:2.1: - image: ruby:2.1 - services: - - postgres:9.3 - script: - - bundle exec rake spec - -test:2.2: - image: ruby:2.2 - services: - - postgres:9.4 - script: - - bundle exec rake spec -``` - -#### How to enable overwriting? -To enable overwriting you have to **enable it first** (it's disabled by default for security reasons). -You can do that by hand modifying runner configuration: `config.toml`. -Please go to section where is `[runners.docker]` definition for your runner. -Add `allowed_images` and `allowed_services` to specify what images are allowed to be picked from `.gitlab-ci.yml`: -``` -[runners.docker] - image = "ruby:2.1" - allowed_images = ["ruby:*", "python:*"] - allowed_services = ["mysql:*", "redis:*"] -``` -This enables you to use in your `.gitlab-ci.yml` any image that matches above wildcards. -You will be able to pick only `ruby` and `python` images. -The same rule can be applied to limit services. - -If you are courageous enough, you can make it fully open and accept everything: -``` -[runners.docker] - image = "ruby:2.1" - allowed_images = ["*", "*/*"] - allowed_services = ["*", "*/*"] -``` - -**It the feature is not enabled, or image isn't allowed the error message will be put into the build log.** - ### How Docker integration works 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. 1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). diff --git a/doc/ci/languages/README.md b/doc/ci/languages/README.md new file mode 100644 index 00000000000..375adf58d18 --- /dev/null +++ b/doc/ci/languages/README.md @@ -0,0 +1,3 @@ +### Languages + ++ [Testing PHP](php.md) diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md new file mode 100644 index 00000000000..e0589182003 --- /dev/null +++ b/doc/ci/languages/php.md @@ -0,0 +1,178 @@ +## Testing PHP projects + +This guide covers basic of building PHP projects. + +Is it possible to test PHP apps on any system. +However, it will require manual configuration. +The simplest is to use Docker executor as described below. + +### PHP projects on Docker executor +It's possible to official [PHP](https://hub.docker.com/_/php/) repositories on Docker Hub. +They allow to test PHP projects against different versions of the runtime. +However, they require additional configuration. + +To build PHP project you need to create valid `.gitlab-ci.yml` describing the build environment: +1. First you need to specify PHP image as described here: http://doc.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-image. To your `.gitlab-ci.yml` add: + + image: php:5.6 + +2. The official images are great, but they are lacking a few useful tools for testing. We need to install them first in build environment. Create `ci/docker_install.sh` file with following content: + + #!/bin/bash + + # We need to install dependencies only for Docker + [[ ! -e /.dockerinit ]] && exit 0 + + set -xe + + # Install git (the php image doesn't have it) which is required by composer + apt-get update -yqq + apt-get install git -yqq + + # Install phpunit, the tool that we will use for testing + curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar + chmod +x /usr/local/bin/phpunit + + # Install mysql driver + # Here you can install any other extension that you need + docker-php-ext-install pdo_mysql + +3. From your `.gitlab-ci.yml` run the created script: + + before_script: + - bash ci/docker_install.sh > /dev/null + +4. Now you can run your tests. Usually it will be `phpunit` with arguments: + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +5. Commit your files, and push them to GitLab to see if it works. With GitLab Runner 1.0 you can also test the changes locally. From your terminal execute: + + # Check using docker executor + gitlab-runner exec docker test:app + + # Check using shell executor + gitlab-runner exec shell test:app + +The final `.gitlab-ci.yml` should look similar to this: + + # Select image from https://hub.docker.com/_/php/ + image: php:5.6 + + before_script: + # Install dependencies + - ci/docker_install.sh > /dev/null + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Test against different PHP versions in Docker builds + +You can also test against multiple version of PHP runtime: + + before_script: + # Install dependencies + - ci/docker_install.sh > /dev/null + + # We test PHP5.6 + test:5.6: + image: php:5.6 + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + + # We test PHP7.0 + test:7.0: + image: php:7.0 + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Custom PHP configuration in Docker builds + +You can customise your PHP environment by putting your .ini file into `/usr/local/etc/php/conf.d/`: + + before_script: + - cp my_php.ini /usr/local/etc/php/conf.d/test.ini + +### Test PHP projects using Shell + +Shell executor runs your builds in terminal session of your server. Thus in order to test your projects you need to have all dependencies installed as root. + +1. Install PHP dependencies: + + sudo apt-get update -qy + sudo apt-get install phpunit php5-mysql -y + + This will install the PHP version available for your distribution. + +2. Now you can run your tests. Usually it will be `phpunit` with arguments: + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Test against different PHP versions in Shell builds + +The [phpenv](https://github.com/phpenv/phpenv) allows you to easily manage different PHP with they own configs. +This is specially usefull when testing PHP project with Shell executor. + +Login as `gitlab-runner` user and follow [the installation guide](https://github.com/phpenv/phpenv#installation). + +Using phpenv also allows to easily configure PHP environment with: `phpenv config-add my_config.ini`. + +#### Install custom extensions + +Since we have pretty bare installation of our PHP environment you may need some extensions that are not present on your installation. + +To install additional extensions simply execute.: + + pecl install + + It's not advised to add this to the `.gitlab-ci.yml`. + You should execute this command once, only to setup the build environment. + +### Extend your tests + +#### Using atoum + +Instead of PHPUnit, you can use any other tool to run unit tests. For example [atoum](https://github.com/atoum/atoum): + + before_script: + - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar + + test:atoum: + script: + - php mageekguy.atoum.phar + +#### Using Composer + +Majority of the PHP projects use Composer for managing the packages. +It's very simple to execute the Composer before running your tests. +To your `.gitlab-ci.yml` add: + + # The composer stores all downloaded packages in vendor/ + # Remove it if you committed the vendor/ directory + cache: + paths: + - vendor/ + + before_script: + # Install composer dependencies + - curl -sS https://getcomposer.org/installer | php + - php composer.phar install + +### Access private packages / dependencies + +You need to configure [the SSH keys](../ssh_keys/README.md) in order to checkout the repositories. + +### Use databases or other services + +Please checkout the docs about configuring [the CI services](../services/README.md). + +### Example project + +You maybe interested in our [Example Project](https://gitlab.com/gitlab-examples/php) that runs on [GitLab.com](https://gitlab.com) using our publically available shared runners. + +Want to hack it? Simply fork it, commit and push changes. Within a few moments the changes will be picked and rebuilt by public runners. diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md new file mode 100644 index 00000000000..0550e9435a3 --- /dev/null +++ b/doc/ci/services/README.md @@ -0,0 +1,6 @@ +## GitLab CI Services + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) ++ [Using Other Services](../docker/using_docker_images.html#how-to-use-other-images-as-services) diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md new file mode 100644 index 00000000000..df36ebaf7d4 --- /dev/null +++ b/doc/ci/services/docker-services.md @@ -0,0 +1,5 @@ +## GitLab CI Services + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md new file mode 100644 index 00000000000..3155af6b3e1 --- /dev/null +++ b/doc/ci/services/mysql.md @@ -0,0 +1,72 @@ +## Using MySQL + +It's possible to use MySQL database test your apps during builds. + +### Use MySQL with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - mysql + + variables: + # Configure mysql service (https://hub.docker.com/_/mysql/) + MYSQL_DATABASE: hello_world_test + MYSQL_ROOT_PASSWORD: mysql + +2. Configure your application to use the database: + + Host: mysql + User: root + Password: mysql + Database: hello_world_test + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/mysql/). For example: `mysql:5.5`. + +Example: https://gitlab.com/gitlab-examples/mysql/blob/master/.gitlab-ci.yml + +### Use MySQL with Shell executor + +It's possible to use MySQL on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the MySQL server: + + sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev + + # Pick a MySQL root password (can be anything), type it and press enter + # Retype the MySQL root password and press enter + +2. Create an user: + + mysql -u root -p + + # Create a user which will be used by your apps + # do not type the 'mysql>', this is part of the prompt + # change $password in the command below to a real password you pick + mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; + + # Ensure you can use the InnoDB engine which is necessary to support long indexes + # If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting "innodb = off" + mysql> SET storage_engine=INNODB; + + # Create the database + mysql> CREATE DATABASE IF NOT EXISTS `hello_world_test` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; + + # Grant necessary permissions on the database + mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `hello_world_test`.* TO 'runner'@'localhost'; + + # Quit the database session + mysql> \q + +3. Try to connect to database: + + sudo -u gitlab-runner -H mysql -u runner -p -D hello_world_test + +4. Configure your application to use the database: + + Host: localhost + User: runner + Password: $password + Database: hello_world_test diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md new file mode 100644 index 00000000000..e57f8c5944a --- /dev/null +++ b/doc/ci/services/postgres.md @@ -0,0 +1,70 @@ +## Using PostgreSQL + +It's possible to use PostgreSQL database test your apps during builds. + +### Use PostgreSQL with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - postgres + + variables: + # Configure postgres service (https://hub.docker.com/_/postgres/) + POSTGRES_DB: hello_world_test + POSTGRES_USER: postgres + POSTGRES_PASSWORD: "" + +2. Configure your application to use the database: + + Host: postgres + User: postgres + Password: postgres + Database: hello_world_test + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/postgres/). For example: `postgres:9.3`. + +Example: https://gitlab.com/gitlab-examples/postgres/blob/master/.gitlab-ci.yml + +### Use PostgreSQL with Shell executor + +It's possible to use PostgreSQL on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the PostgreSQL server: + + sudo apt-get install -y postgresql postgresql-client libpq-dev + +2. Create an user: + + # Install the database packages + sudo apt-get install -y postgresql postgresql-client libpq-dev + + # Login to PostgreSQL + sudo -u postgres psql -d template1 + + # Create a user for runner + # Do not type the 'template1=#', this is part of the prompt + template1=# CREATE USER runner CREATEDB; + + # Create the database & grant all privileges on database + template1=# CREATE DATABASE hello_world_test OWNER runner; + + # Quit the database session + template1=# \q + +3. Try to connect to database: + + # Try connecting to the new database with the new user + sudo -u gitlab-runner -H psql -d hello_world_test + + # Quit the database session + hello_world_test> \q + +4. Configure your application to use the database: + + Host: localhost + User: runner + Password: + Database: hello_world_test diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md new file mode 100644 index 00000000000..523634a457e --- /dev/null +++ b/doc/ci/services/redis.md @@ -0,0 +1,40 @@ +## Using Redis + +It's possible to use Redis database test your apps during builds. + +### Use Redis with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - redis + +2. Configure your application to use the database: + + Host: redis + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/redis/). For example: `redis:2.6`. + +Example: https://gitlab.com/gitlab-examples/redis/blob/master/.gitlab-ci.yml + +### Use Redis with Shell executor + +It's possible to use Redis on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the Redis server: + + sudo apt-get install redis-server + +2. Try to connect to the server: + + # Try connecting the the Redis server + sudo -u gitlab-runner -H redis-cli + + # Quit the session + 127.0.0.1:6379> quit + +4. Configure your application to use the database: + + Host: localhost diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md new file mode 100644 index 00000000000..515194e5f5e --- /dev/null +++ b/doc/ci/ssh_keys/README.md @@ -0,0 +1,114 @@ +# Using SSH keys + +GitLab currently doesn't have built-in support for SSH keys in build environment. + +The SSH keys can be useful when: +1. You want to checkout internal submodules, +2. You want to download private packages using your package manager (ie. bundler), +3. You want to deploy your app (ex. to Heroku or own server), +4. You want to execute ssh commands from build environment on remote server, +5. You want to rsync files from your build to remote server. + +If anyone of the above holds true, then you most likely need SSH key. + +There are two possibilities to add SSH keys to build environment. + +## Inject keys in your build environment +The most widely supported is to inject SSH key into your build environment by extending your .gitlab-ci.yml. +This is the universal solution which works with any type of executor (docker, shell, etc.). + +### How it works? +1. We create a new SSH private key with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen). +2. We add the private key as the Secure Variable to project. +3. We run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during build to load the private key. + +The example [.gitlab-ci.yml](https://gitlab.com/gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml) looks like this. + +### Make it work? +1. First, go to terminal and generate a new SSH key: +```bash +$ ssh-keygen -t rsa -f my_key + +Generating public/private rsa key pair. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in my_key. +Your public key has been saved in my_key.pub. +The key fingerprint is: +SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint +The key's randomart image is: ++---[RSA 2048]----+ +|=*. .o++*= | +|..= +o..o. | +|.+++o + + . | +|+o*=.. + + | +|o+.=. . S | +|*.o .E . | +|o*o . . | +|.o.. | +| . | ++----[SHA256]-----+ +``` + +2. Create a new **Secure Variable** in your project settings on GitLab and name it: `SSH_PRIVATE_KEY`. + +3. Copy the content of `my_key` and paste it as a **Value** of **SSH_PRIVATE_KEY**. + +4. Next you need to modify your `.gitlab-ci.yml` and at the top of the file add: +``` +before_script: +# install ssh-agent (it is required for Docker, change apt-get to yum if you use CentOS-based image) +- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + +# run ssh-agent (in build environment) +- eval $(ssh-agent -s) + +# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store +- ssh-add <(echo "$SSH_PRIVATE_KEY") + +# for Docker builds disable host key checking, by adding that you are suspectible to man-in-the-middle attack +- mkdir -p ~/.ssh +- '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` +``` + +5. Add the public key from `my_key.pub` to services that you want to have an access from build. + +6. If your builds are run using `shell` executor, you may need to login to server and execute the `ssh ` to store the fingerprint of remote server. + +## SSH keys when using Shell executor +If use `shell`, not `docker` it can be easier to have the SSH key. + +We can generate the SSH key for the machine that holds `gitlab-runner` and use that key for all projects that are run on this machine. + +1. First, login to server that runs your builds. + +2. From terminal login as `gitlab-runner` user and generate the SSH private key: +```bash +$ ssh-keygen -t rsa +Generating public/private rsa key pair. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in ~/.ssh/id_rsa. +Your public key has been saved in ~/.ssh/id_rsa.pub. +The key fingerprint is: +SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint +The key's randomart image is: ++---[RSA 2048]----+ +|=*. .o++*= | +|..= +o..o. | +|.+++o + + . | +|+o*=.. + + | +|o+.=. . S | +|*.o .E . | +|o*o . . | +|.o.. | +| . | ++----[SHA256]-----+ +``` + +3. Add the public key from `~/.ssh/id_rsa.pub` to services that you want to have an access from build. + +4. Try to login for the first time and accept fingerprint: +```bash +ssh Date: Wed, 2 Dec 2015 19:24:15 +0200 Subject: Use .md instead of .html --- doc/ci/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/README.md b/doc/ci/README.md index ae921b6f988..5d9d7a81db3 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -20,7 +20,7 @@ + [Using MySQL](services/mysql.md) + [Using PostgreSQL](services/postgres.md) + [Using Redis](services/redis.md) -+ [Using Other Services](docker/using_docker_images.html#how-to-use-other-images-as-services) ++ [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services) ### Examples -- cgit v1.2.1 From 05267b64d224c19f1b6843a715f666aea2bd106a Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 00:36:45 +0200 Subject: Clean up using_docker_images.md --- doc/ci/docker/using_docker_images.md | 268 ++++++++++++++++++++++++----------- 1 file changed, 185 insertions(+), 83 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 2f0ca19cd0f..6551d47b697 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,16 +1,25 @@ # Using Docker Images -GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. -Docker is an open-source project that allows to use predefined images to run applications -in independent "containers" that are run within a single Linux instance. -[Docker Hub](https://registry.hub.docker.com/) have rich database of built images that can be used to build applications. +GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. -Docker when used with GitLab CI runs each build in separate and isolated container using predefined image and always from scratch. -It makes it easier to have simple and reproducible build environment that can also be run on your workstation. -This allows you to test all commands from your shell, rather than having to test them on a CI server. +Docker is an open-source project that allows to use predefined images to run +applications in independent "containers" that are run within a single Linux +instance. [Docker Hub][hub] has a rich database of prebuilt images that can be +used to test and build your applications. -### Register Docker runner -To use GitLab Runner with Docker you need to register new runner to use `docker` executor: +Docker, when used with GitLab CI, runs each build in a separate and isolated +container using the predefined image that is set up in `.gitlab-ci.yml`. The +container is always euphemeral which means you get only one container per build. + +This makes it easier to have a simple and reproducible build environment that +can also run on your workstation. The added benefit is that you can test all +the commands that we will explore later from your shell, rather than having to +test them on a CI server. + +### Register docker runner + +To use GitLab Runner with docker you need to register a new runner to use the +`docker` executor: ```bash gitlab-ci-multi-runner register \ @@ -23,45 +32,70 @@ gitlab-ci-multi-runner register \ --docker-mysql latest ``` -**The registered runner will use `ruby:2.1` image and will run two services (`postgres:latest` and `mysql:latest`) that will be accessible for time of the build.** +The registered runner will use the `ruby:2.1` docker image and will run two +services, `postgres:latest` and `mysql:latest`, both of which will be +accessible during the build process. + +### What is image + +The `image` keyword is the name of the docker image that is present in the +local Docker Engine (list all images with `docker images`) or any image that +can be found at [Docker Hub][hub]. For more information about images and Docker +Hub please read the [Docker Fundamentals][] documentation. + +In short, with `image` we refer to the docker image, which will be used to +create a container on which your build will run. + +### What is service -### What is image? -The image is the name of any repository that is present in local Docker Engine or any repository that can be found at [Docker Hub](https://registry.hub.docker.com/). -For more information about the image and Docker Hub please read the [Docker Fundamentals](https://docs.docker.com/introduction/understanding-docker/). +The `service` keyword defines just another docker image that is run during +your build and is linked to the docker image that the `image` keyword defines. +This allows you to access the service image during build time. -### What is service? -Service is just another image that is run for time of your build and is linked to your build. This allows you to access the service image during build time. -The service image can run any application, but most common use case is to run some database container, ie.: `mysql`. -It's easier and faster to use existing image, run it as additional container than install `mysql` every time project is built. +The service image can run any application, but the most common use case is to +run a database container, eg. `mysql`. It's easier and faster to use an +existing image and run it as an additional container than install `mysql` every +time the project is built. -#### How is service linked to the build? -There's good document that describes how Docker linking works: [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). -To summarize: if you add `mysql` as service to your application, the image will be used to create container that is linked to build container. -The service container for MySQL will be accessible under hostname `mysql`. -So, **to access your database service you have to connect to host: `mysql` instead of socket or `localhost`**. +#### How is service linked to the build -### How to use other images as services? -You are not limited to have only database services. -You can add the services to `.gitlab-ci.yml` or hand modify the `config.toml`. -You can use any image as service found at [Docker Hub](https://registry.hub.docker.com/). +To better undestand how the container linking works, read +[Linking containers together](https://docs.docker.com/userguide/dockerlinks/). + +To summarize, if you add `mysql` as service to your application, the image will +then be used to create a container that is linked to the build container. + +The service container for MySQL will be accessible under the hostname `mysql`. +So, in order to access your database service you have to connect to the host +named `mysql` instead of a socket or `localhost`. + +### How to use other images as services + +You are not limited to have only database services. You can add as many +services you need to `.gitlab-ci.yml` or manually modify `config.toml`. +Any image found at [Docker Hub][hub] can be used as a service. ### Define image and services from `.gitlab-ci.yml` -You can simply define image or list services that you want to use for the build time. -``` + +You can simply define an image that will be used for all jobs and a list of +services that you want to use during build time. + +```yaml image: ruby:2.2 services: - postgres:9.3 -before_install: +before_script: - bundle install - + test: script: - bundle exec rake spec ``` -It's possible to define image and service per-job: -``` -before_install: +It is also possible to define different images and services per job: + +```yaml +before_script: - bundle install test:2.1: @@ -80,68 +114,106 @@ test:2.2: ``` ### Define image and services in `config.toml` -Look for `[runners.docker]` section: + +Look for the `[runners.docker]` section: + ``` +... + [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] + +... ``` -The image and services defined these way will be added to all builds run by that runner. +The image and services defined this way will be added to all builds run by +that runner. ### Accessing the services -For example you need `wordpress` instance to test some API integration with `Wordpress`. -You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). -``` -# .gitlab-ci.yml +Let's say that you need a Wordpress instance to test some API integration with +your application. + +You can then use for example the [tutum/wordpress][] image in your +`.gitlab-ci.yml`: + +```yaml +... + services: - tutum/wordpress:latest + +... ``` -Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum__wordpress`. +When the build is run, `tutum/wordpress` will be started and you will have +access to it from your build container under the hostname `tutum_wordpress`. + +The alias hostname for the service is made from the image name following these +rules: -Alias hostname for the service is made from the image name: -1. Everything after `:` is stripped, -2. '/' is replaced to `__`. +1. Everything after `:` is stripped +2. Backslash (`/`) is replaced with double underscores (`__`) ### Configuring services -Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. -GitLab Runner 0.5.0 and up passes all YAML-defined variables to created service containers. +Many services accept environment variables which allow you to easily change +database names or set account names depending on the environment. -1. To configure database name for [postgres](https://registry.hub.docker.com/u/library/postgres/) service, -you need to set POSTGRES_DB. +GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created +service containers. - ```yaml - services: - - postgres - - variables: - POSTGRES_DB: gitlab - ``` +For all possible configuration variables check the documentation of each image +provided in their corresponding Docker hub page. -1. To use [mysql](https://registry.hub.docker.com/u/library/mysql/) service with empty password for time of build, -you need to set MYSQL_ALLOW_EMPTY_PASSWORD. +*Note: All variables will be passed to all service containers. It's not designed + to distinguish which variable should go where.* - ```yaml - services: - - mysql - - variables: - MYSQL_ALLOW_EMPTY_PASSWORD: "yes" - ``` +#### PostgreSQL service example -For other possible configuration variables check the -https://registry.hub.docker.com/u/library/mysql/ or https://registry.hub.docker.com/u/library/postgres/ -or README page for any other Docker image. +To configure the database name for [postgres][postgres-hub] service, you need +to set the `POSTGRES_DB` variable: -**Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** +```yaml +... + +services: +- postgres + +variables: + POSTGRES_DB: gitlab + +... +``` + +For a real example visit . + +#### MySQL service example + +To use the [mysql][mysql-hub] service with an empty password during the time of +build, you need to set the `MYSQL_ALLOW_EMPTY_PASSWORD` variable: + +```yaml +... + +services: +- mysql + +variables: + MYSQL_ALLOW_EMPTY_PASSWORD: "yes" + +... +``` ### How Docker integration works + +Below is a high level overview of the steps performed by docker during build +time. + 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. -1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). +1. Create cache container to store all volumes as defined in `config.toml` and + `Dockerfile` of build image (`ruby:2.1` as in above example). 1. Create build container and link any service container to build container. 1. Start build container and send build script to the container. 1. Run build script. @@ -151,32 +223,62 @@ or README page for any other Docker image. 1. Remove build container and all created service containers. ### How to debug a build locally -1. Create a file with build script: + +*Note: The following commands are run without root privileges. You should be +able to run docker with your regular user account.* + +First start with creating a file named `build script`: + ```bash -$ cat < build_script +cat < build_script git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner cd /builds/gitlab-org/gitlab-ci-multi-runner -make <- or any other build step +make EOF ``` -1. Create service containers: +Here we use as an example the GitLab Runner repository which contains a +Makefile, so running `make` will execute the commands defined in the Makefile. +Your mileage may vary, so instead of `make` you could run the command which +is specific to your project. + +Then create some service containers: + ``` -$ docker run -d -n service-mysql mysql:latest -$ docker run -d -n service-postgres postgres:latest +docker run -d -n service-mysql mysql:latest +docker run -d -n service-postgres postgres:latest ``` -This will create two service containers (MySQL and PostgreSQL). -1. Create a build container and execute script in its context: +This will create two service containers, named `service-mysql` and +`service-postgres` which use the latest MySQL and PostgreSQL images +respectively. They will both run in the background (`-d`). + +Finally, create a build container by executing the `build_script` file we +created earlier: + ``` -$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script +docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` -This will create build container that has two service containers linked. -The build_script is piped using STDIN to bash interpreter which executes the build script in container. -1. At the end remove all containers: +The above command will create a container named `build` that is spawned from +the `ruby:2.1` image and has two services linked to it. The `build_script` is +piped using STDIN to the bash interpreter which in turn executes the +`build_script` in the `build` container. + +When you finish testing and no longer need the containers, you can remove them +with: + ``` docker rm -f -v build service-mysql service-postgres ``` -This will forcefully (the `-f` switch) remove build container and service containers -and all volumes (the `-v` switch) that were created with the container creation. + +This will forcefully (`-f`) remove the `build` container, the two service +containers as well as all volumes (`-v`) that were created with the container +creation. + +[Docker Fundamentals]: https://docs.docker.com/engine/introduction/understanding-docker/ +[hub]: https://hub.docker.com/ +[linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/ +[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ +[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ +[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ -- cgit v1.2.1 From 223a02757909b406c1d5ebe42ca4bc3340318a2e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:43:06 +0200 Subject: Bring back removed heading and point to other section --- doc/ci/docker/using_docker_images.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 6551d47b697..63cfa436333 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -69,6 +69,10 @@ The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host named `mysql` instead of a socket or `localhost`. +### Overwrite image and services + +See the section below. + ### How to use other images as services You are not limited to have only database services. You can add as many -- cgit v1.2.1 From 71519d650d0317785f86ed935508d1dd5966cd6e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:43:29 +0200 Subject: Fix wrong example --- doc/ci/docker/using_docker_images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 63cfa436333..488479418b1 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -152,7 +152,7 @@ services: ``` When the build is run, `tutum/wordpress` will be started and you will have -access to it from your build container under the hostname `tutum_wordpress`. +access to it from your build container under the hostname `tutum__wordpress`. The alias hostname for the service is made from the image name following these rules: -- cgit v1.2.1 From bb75dfe38b481607952234212a15d29cd17b7cef Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:46:27 +0200 Subject: Add intro about languages --- doc/ci/languages/README.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/languages/README.md b/doc/ci/languages/README.md index 375adf58d18..54b2343e08b 100644 --- a/doc/ci/languages/README.md +++ b/doc/ci/languages/README.md @@ -1,3 +1,7 @@ ### Languages +This is a list of languages you can test with GitLab CI. Each section has +comprehensive documentation and comes with a test repository hosted on +GitLab.com + + [Testing PHP](php.md) -- cgit v1.2.1 From a08cc70232b1c5071f199106a582c77ba9541a83 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 18:13:19 +0200 Subject: Clean up PHP CI example [ci skip] --- doc/ci/languages/php.md | 325 +++++++++++++++++++++++++++++++----------------- 1 file changed, 211 insertions(+), 114 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index e0589182003..c71ae40786e 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -1,178 +1,275 @@ -## Testing PHP projects +# Testing PHP projects -This guide covers basic of building PHP projects. +This guide covers basic building instructions for PHP projects. -Is it possible to test PHP apps on any system. -However, it will require manual configuration. -The simplest is to use Docker executor as described below. +There are covered two cases: testing using the Docker executor and testing +using the Shell executor. -### PHP projects on Docker executor -It's possible to official [PHP](https://hub.docker.com/_/php/) repositories on Docker Hub. -They allow to test PHP projects against different versions of the runtime. -However, they require additional configuration. +## Test PHP projects using the Docker executor -To build PHP project you need to create valid `.gitlab-ci.yml` describing the build environment: -1. First you need to specify PHP image as described here: http://doc.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-image. To your `.gitlab-ci.yml` add: +While it is possible to test PHP apps on any system, this would require manual +configuration from the developer. To overcome this we will be using the +official [PHP docker image][php-hub] that can be found in Docker Hub. - image: php:5.6 +This will allow us to test PHP projects against different versions of PHP. +However, not everything is plug 'n' play, you still need to onfigure some +things manually. -2. The official images are great, but they are lacking a few useful tools for testing. We need to install them first in build environment. Create `ci/docker_install.sh` file with following content: +As with every build, you need to create a valid `.gitlab-ci.yml` describing the +build environment. - #!/bin/bash +Let's first specify the PHP image that will be used for the build process +(you can read more about what an image means in the Runner's lingo reading +about [Using Docker images](../docker/using_docker_images.md#what-is-image)). - # We need to install dependencies only for Docker - [[ ! -e /.dockerinit ]] && exit 0 +Start by adding the image to your `.gitlab-ci.yml`: - set -xe +```yaml +image: php:5.6 +``` - # Install git (the php image doesn't have it) which is required by composer - apt-get update -yqq - apt-get install git -yqq +The official images are great, but they lack a few useful tools for testing. +We need to first prepare the build environment. A way to overcome this is to +create a script which installs all prerequisites prior the actual testing is +done. - # Install phpunit, the tool that we will use for testing - curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar - chmod +x /usr/local/bin/phpunit +Let's create a `ci/docker_install.sh` file in the root directory of our +repository with the following content: - # Install mysql driver - # Here you can install any other extension that you need - docker-php-ext-install pdo_mysql +```bash +#!/bin/bash -3. From your `.gitlab-ci.yml` run the created script: +# We need to install dependencies only for Docker +[[ ! -e /.dockerinit ]] && exit 0 - before_script: - - bash ci/docker_install.sh > /dev/null +set -xe -4. Now you can run your tests. Usually it will be `phpunit` with arguments: +# Install git (the php image doesn't have it) which is required by composer +apt-get update -yqq +apt-get install git -yqq - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +# Install phpunit, the tool that we will use for testing +curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar +chmod +x /usr/local/bin/phpunit -5. Commit your files, and push them to GitLab to see if it works. With GitLab Runner 1.0 you can also test the changes locally. From your terminal execute: +# Install mysql driver +# Here you can install any other extension that you need +docker-php-ext-install pdo_mysql +``` - # Check using docker executor - gitlab-runner exec docker test:app +You might wonder what `docker-php-ext-install` is. In short, it is a script +provided by the official php docker image that you can use to easilly install +extensions. For more information read the the documentation at +. - # Check using shell executor - gitlab-runner exec shell test:app +Now that we created the script that contains all prerequisites for our build +environment, let's add it in `.gitlab-ci.yml`: + +```yaml +... + +before_script: +- bash ci/docker_install.sh > /dev/null + +... +``` + +Last step, run the actual tests using `phpunit`: + +```yaml +... + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml + +... +``` + +Finally, commit your files and push them to GitLab to see your build succeeding +(or failing). The final `.gitlab-ci.yml` should look similar to this: - # Select image from https://hub.docker.com/_/php/ - image: php:5.6 +```yaml +# Select image from https://hub.docker.com/_/php/ +image: php:5.6 + +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Test against different PHP versions in Docker builds + +Testing against multiple versions of PHP is super easy. Just add another job +with a different docker image version and the runner will do the rest: + +```yaml +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +# We test PHP5.6 +test:5.6: + image: php:5.6 + script: + - phpunit --configuration phpunit_myapp.xml + +# We test PHP7.0 (good luck with that) +test:7.0: + image: php:7.0 + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Custom PHP configuration in Docker builds + +There are times where you will need to customise your PHP environment by +putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose +add a `before_script` action: + +```yaml +before_script: +- cp my_php.ini /usr/local/etc/php/conf.d/test.ini +``` - before_script: - # Install dependencies - - ci/docker_install.sh > /dev/null +Of course, `my_php.ini` must be present in the root directory of your repository. - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +## Test PHP projects using the Shell executor -#### Test against different PHP versions in Docker builds +The shell executor runs your builds in a terminal session on your server. +Thus, in order to test your projects you first need to make sure that all +dependencies are installed. -You can also test against multiple version of PHP runtime: +For example, in a VM running Debian 8 we first update the cache, then we +install `phpunit` and `php5-mysql`: - before_script: - # Install dependencies - - ci/docker_install.sh > /dev/null +```bash +sudo apt-get update -y +sudo apt-get install -y phpunit php5-mysql +``` - # We test PHP5.6 - test:5.6: - image: php:5.6 - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +Next, add the following snippet to your `.gitlab-ci.yml`: - # We test PHP7.0 - test:7.0: - image: php:7.0 - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +```yaml +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` -#### Custom PHP configuration in Docker builds +Finally, push to GitLab and let the tests begin! -You can customise your PHP environment by putting your .ini file into `/usr/local/etc/php/conf.d/`: +### Test against different PHP versions in Shell builds - before_script: - - cp my_php.ini /usr/local/etc/php/conf.d/test.ini +The [phpenv][] project allows you to easily manage different versions of PHP +each with its own config. This is specially usefull when testing PHP projects +with the Shell executor. -### Test PHP projects using Shell +You will have to install it on your build machine under the `gitlab-runner` +user following [the upstream installation guide][phpenv-installation]. -Shell executor runs your builds in terminal session of your server. Thus in order to test your projects you need to have all dependencies installed as root. +Using phpenv also allows to easily configure the PHP environment with: -1. Install PHP dependencies: +``` +phpenv config-add my_config.ini +``` - sudo apt-get update -qy - sudo apt-get install phpunit php5-mysql -y +### Install custom extensions - This will install the PHP version available for your distribution. +Since this is a pretty bare installation of the PHP environment, you may need +some extensions that are not currently present on the build machine. -2. Now you can run your tests. Usually it will be `phpunit` with arguments: +To install additional extensions simply execute: - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +```bash +pecl install +``` -#### Test against different PHP versions in Shell builds +It's not advised to add this to `.gitlab-ci.yml`. You should execute this +command once, only to setup the build environment. -The [phpenv](https://github.com/phpenv/phpenv) allows you to easily manage different PHP with they own configs. -This is specially usefull when testing PHP project with Shell executor. +## Extend your tests -Login as `gitlab-runner` user and follow [the installation guide](https://github.com/phpenv/phpenv#installation). +### Using atoum -Using phpenv also allows to easily configure PHP environment with: `phpenv config-add my_config.ini`. +Instead of PHPUnit, you can use any other tool to run unit tests. For example +you can use [atoum](https://github.com/atoum/atoum): -#### Install custom extensions +```yaml +before_script: +- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar -Since we have pretty bare installation of our PHP environment you may need some extensions that are not present on your installation. +test:atoum: + script: + - php mageekguy.atoum.phar +``` -To install additional extensions simply execute.: +### Using Composer - pecl install +The majority of the PHP projects use Composer for managing their PHP packages. +In order to execute Composer before running your tests, simply add the +following in your `.gitlab-ci.yml`: - It's not advised to add this to the `.gitlab-ci.yml`. - You should execute this command once, only to setup the build environment. +```yaml +... -### Extend your tests +# Composer stores all downloaded packages in the vendor/ directory. +# Do not use the following if the vendor/ directory is commited to +# your git repository. +cache: + paths: + - vendor/ -#### Using atoum +before_script: +# Install composer dependencies +- curl -sS https://getcomposer.org/installer | php +- php composer.phar install -Instead of PHPUnit, you can use any other tool to run unit tests. For example [atoum](https://github.com/atoum/atoum): +... +``` - before_script: - - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar +## Access private packages / dependencies - test:atoum: - script: - - php mageekguy.atoum.phar +If your test suite needs to access a private repository, you need to configure +[the SSH keys](../ssh_keys/README.md) in order to be able to clone it. -#### Using Composer +## Use databases or other services -Majority of the PHP projects use Composer for managing the packages. -It's very simple to execute the Composer before running your tests. -To your `.gitlab-ci.yml` add: +Most of the time you will need a running database in order for your tests to +run. If you are using the Docker executor you can leverage Docker's ability to +connect to other containers. In GitLab Runner lingo, this can be achieved by +defining a `service`. - # The composer stores all downloaded packages in vendor/ - # Remove it if you committed the vendor/ directory - cache: - paths: - - vendor/ +This functionality is covered in [the CI services](../services/README.md) +documentation. - before_script: - # Install composer dependencies - - curl -sS https://getcomposer.org/installer | php - - php composer.phar install +## Testing things locally -### Access private packages / dependencies +With GitLab Runner 1.0 you can also test any changes locally. From your +terminal execute: -You need to configure [the SSH keys](../ssh_keys/README.md) in order to checkout the repositories. +```bash +# Check using docker executor +gitlab-runner exec docker test:app -### Use databases or other services +# Check using shell executor +gitlab-runner exec shell test:app +``` -Please checkout the docs about configuring [the CI services](../services/README.md). +## Example project -### Example project +We have set up an [Example PHP Project](https://gitlab.com/gitlab-examples/php) +for your convenience that runs on [GitLab.com](https://gitlab.com) using our +publicly available [shared runners](../runners/README.md). -You maybe interested in our [Example Project](https://gitlab.com/gitlab-examples/php) that runs on [GitLab.com](https://gitlab.com) using our publically available shared runners. +Want to hack it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. -Want to hack it? Simply fork it, commit and push changes. Within a few moments the changes will be picked and rebuilt by public runners. +[php-hub]: https://hub.docker.com/_/php/ +[phpenv]: https://github.com/phpenv/phpenv +[phpenv-installation]: https://github.com/phpenv/phpenv#installation -- cgit v1.2.1 From 9b8babb601ac6bdb3b7301b017a8ce20c5dc4814 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 21:18:34 +0200 Subject: Use link instead of connect to be more Docker friendly --- doc/ci/languages/php.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index c71ae40786e..589253cfc4c 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -242,7 +242,7 @@ If your test suite needs to access a private repository, you need to configure Most of the time you will need a running database in order for your tests to run. If you are using the Docker executor you can leverage Docker's ability to -connect to other containers. In GitLab Runner lingo, this can be achieved by +link to other containers. In GitLab Runner lingo, this can be achieved by defining a `service`. This functionality is covered in [the CI services](../services/README.md) -- cgit v1.2.1 From d13d43aca9c486365fbe6eab8d30cb1e005c3f61 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 4 Dec 2015 11:46:08 +0200 Subject: Clean up CI ssh keys docs [ci skip] --- doc/ci/ssh_keys/README.md | 167 ++++++++++++++++++++++------------------------ 1 file changed, 81 insertions(+), 86 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 515194e5f5e..210f9c3e849 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,114 +1,109 @@ # Using SSH keys -GitLab currently doesn't have built-in support for SSH keys in build environment. +GitLab currently doesn't have built-in support for managing SSH keys in a build +environment. The SSH keys can be useful when: -1. You want to checkout internal submodules, -2. You want to download private packages using your package manager (ie. bundler), -3. You want to deploy your app (ex. to Heroku or own server), -4. You want to execute ssh commands from build environment on remote server, -5. You want to rsync files from your build to remote server. -If anyone of the above holds true, then you most likely need SSH key. +1. You want to checkout internal submodules +2. You want to download private packages using your package manager (eg. bundler) +3. You want to deploy your application to eg. Heroku or your own server +4. You want to execute SSH commands from the build server to the remote server +5. You want to rsync files from your build server to the remote server -There are two possibilities to add SSH keys to build environment. +If anything of the above rings a bell, then you most likely need an SSH key. -## Inject keys in your build environment -The most widely supported is to inject SSH key into your build environment by extending your .gitlab-ci.yml. -This is the universal solution which works with any type of executor (docker, shell, etc.). +## Inject keys in your build server -### How it works? -1. We create a new SSH private key with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen). -2. We add the private key as the Secure Variable to project. -3. We run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during build to load the private key. +The most widely supported method is to inject an SSH key into your build +environment by extending your `.gitlab-ci.yml`. -The example [.gitlab-ci.yml](https://gitlab.com/gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml) looks like this. +This is the universal solution which works with any type of executor +(docker, shell, etc.). -### Make it work? -1. First, go to terminal and generate a new SSH key: -```bash -$ ssh-keygen -t rsa -f my_key - -Generating public/private rsa key pair. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in my_key. -Your public key has been saved in my_key.pub. -The key fingerprint is: -SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint -The key's randomart image is: -+---[RSA 2048]----+ -|=*. .o++*= | -|..= +o..o. | -|.+++o + + . | -|+o*=.. + + | -|o+.=. . S | -|*.o .E . | -|o*o . . | -|.o.. | -| . | -+----[SHA256]-----+ -``` +### How it works + +1. Create a new SSH key pair with [ssh-keygen][] +2. Add the private key as a **Secret Variable** to the project +3. Run the [ssh-agent][] during build to load the private key. + +## SSH keys when using the Docker executor -2. Create a new **Secure Variable** in your project settings on GitLab and name it: `SSH_PRIVATE_KEY`. +You will first need to create an SSH key pair. For more information, follow the +instructions to [generate an SSH key](../ssh/README.md). -3. Copy the content of `my_key` and paste it as a **Value** of **SSH_PRIVATE_KEY**. +Then, create a new **Secret Variable** in your project settings on GitLab +following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` +and in the **Value** field paste the content of your _private_ key that you +created earlier. + +Next you need to modify your `.gitlab-ci.yml` with a `before_script` action. +Add it to the top: -4. Next you need to modify your `.gitlab-ci.yml` and at the top of the file add: ``` before_script: -# install ssh-agent (it is required for Docker, change apt-get to yum if you use CentOS-based image) -- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + # Install ssh-agent if not already installed, it is required by Docker. + # (change apt-get to yum if you use a CentOS-based image) + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + + # Run ssh-agent (inside the build environment) + - eval $(ssh-agent -s) + + # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + - ssh-add <(echo "$SSH_PRIVATE_KEY") + + # For Docker builds disable host key checking. Be aware that by adding that + # you are suspectible to man-in-the-middle attacks. + # WARNING: Use this only with the Docker executor, if you use it with shell + # you will overwrite your user's SSH config. + - mkdir -p ~/.ssh + - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` +``` -# run ssh-agent (in build environment) -- eval $(ssh-agent -s) +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). -# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store -- ssh-add <(echo "$SSH_PRIVATE_KEY") +That's it! You can now have access to private servers or repositories in your +build environment. -# for Docker builds disable host key checking, by adding that you are suspectible to man-in-the-middle attack -- mkdir -p ~/.ssh -- '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` -``` +## SSH keys when using the Shell executor -5. Add the public key from `my_key.pub` to services that you want to have an access from build. +If you are using the Shell executor and not Docker, it is easier to set up an +SSH key. -6. If your builds are run using `shell` executor, you may need to login to server and execute the `ssh ` to store the fingerprint of remote server. +You can generate the SSH key from the machine that GitLab Runner is installed +on, and use that key for all projects that are run on this machine. -## SSH keys when using Shell executor -If use `shell`, not `docker` it can be easier to have the SSH key. +First, you need to login to the server that runs your builds. -We can generate the SSH key for the machine that holds `gitlab-runner` and use that key for all projects that are run on this machine. +Then from the terminal login as the `gitlab-runner` user and generate the SSH +key pair as described in the [SSH keys documentation](../ssh/README.md). -1. First, login to server that runs your builds. +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). + +Once done, try to login to the remote server in order to accept the fingerprint: -2. From terminal login as `gitlab-runner` user and generate the SSH private key: ```bash -$ ssh-keygen -t rsa -Generating public/private rsa key pair. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in ~/.ssh/id_rsa. -Your public key has been saved in ~/.ssh/id_rsa.pub. -The key fingerprint is: -SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint -The key's randomart image is: -+---[RSA 2048]----+ -|=*. .o++*= | -|..= +o..o. | -|.+++o + + . | -|+o*=.. + + | -|o+.=. . S | -|*.o .E . | -|o*o . . | -|.o.. | -| . | -+----[SHA256]-----+ +ssh ``` -3. Add the public key from `~/.ssh/id_rsa.pub` to services that you want to have an access from build. +For accessing repositories on GitLab.com, the `` would be +`git@gitlab.com`. -4. Try to login for the first time and accept fingerprint: -```bash -ssh Date: Fri, 4 Dec 2015 11:47:02 +0200 Subject: Move markdown link to the bottom [ci skip] --- doc/ci/languages/php.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index 589253cfc4c..60079c090d3 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -263,13 +263,14 @@ gitlab-runner exec shell test:app ## Example project -We have set up an [Example PHP Project](https://gitlab.com/gitlab-examples/php) -for your convenience that runs on [GitLab.com](https://gitlab.com) using our -publicly available [shared runners](../runners/README.md). +We have set up an [Example PHP Project][php-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). -Want to hack it? Simply fork it, commit and push your changes. Within a few +Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. [php-hub]: https://hub.docker.com/_/php/ [phpenv]: https://github.com/phpenv/phpenv [phpenv-installation]: https://github.com/phpenv/phpenv#installation +[php-example-repo]: https://gitlab.com/gitlab-examples/php -- cgit v1.2.1 From 4fc9e6944e363e3167fede07c0d6898ff6c7e19f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 4 Dec 2015 11:47:50 +0200 Subject: Add an intro to CI services documentation [ci skip] --- doc/ci/services/README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 0550e9435a3..0856b7679c2 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,6 +1,9 @@ ## GitLab CI Services +GitLab CI uses the `service` keyword to define what docker containers should be +linked with your base image. Below is a list of examples you may use. + + [Using MySQL](mysql.md) + [Using PostgreSQL](postgres.md) + [Using Redis](redis.md) -+ [Using Other Services](../docker/using_docker_images.html#how-to-use-other-images-as-services) ++ [Using Other Services](../docker/using_docker_images.md#how-to-use-other-images-as-services) -- cgit v1.2.1 From 4236861df03ff86fc1b2bb8a6a0835b0f3a03244 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 7 Dec 2015 09:39:14 +0200 Subject: Clean up Redis CI service example [ci skip] --- doc/ci/services/redis.md | 72 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 50 insertions(+), 22 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 523634a457e..b0d04f95408 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -1,40 +1,68 @@ -## Using Redis +# Using Redis -It's possible to use Redis database test your apps during builds. +As many applications depend on Redis as their key-value store, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. -### Use Redis with Docker executor +## Use Redis with Docker executor -If you are using our Docker integration you basically have everything already. +If you are using our Docker integration you basically have everything set up +already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - redis +```yaml +services: + - redis:latest +``` -2. Configure your application to use the database: +Then you need to configure your application to use the Redis database, for +example: - Host: redis +```bash +Host: redis +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/redis/). For example: `redis:2.6`. +And that's it. Redis will now be available to be used within your testing +framework. -Example: https://gitlab.com/gitlab-examples/redis/blob/master/.gitlab-ci.yml +If you want to use any other version of Redis, check the available versions +on [Docker Hub](https://hub.docker.com/_/redis/). -### Use Redis with Shell executor +## Use Redis with Shell executor -It's possible to use Redis on manually configured servers that are using GitLab Runner with Shell executor. +Redis can also be used on manually configured servers that are using GitLab +Runner with the Shell executor. -1. First install the Redis server: +In your build machine install the Redis server: - sudo apt-get install redis-server +```bash +sudo apt-get install redis-server +``` -2. Try to connect to the server: +Verify that you can connect to the server with the `gitlab-runner` user: - # Try connecting the the Redis server - sudo -u gitlab-runner -H redis-cli +```bash +# Try connecting the the Redis server +sudo -u gitlab-runner -H redis-cli - # Quit the session - 127.0.0.1:6379> quit +# Quit the session +127.0.0.1:6379> quit +``` -4. Configure your application to use the database: +Finally, configure your application to use the database, for example: - Host: localhost +```bash +Host: localhost +``` + +## Example project + +We have set up an [Example Redis Project][redis-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[redis-example-repo]: https://gitlab.com/gitlab-examples/redis -- cgit v1.2.1 From 149c934a1d38662358b48cdf844430de7845d1c4 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 15:59:41 +0200 Subject: More cleanup on Redis example --- doc/ci/services/redis.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index b0d04f95408..e6add57ca76 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -4,10 +4,10 @@ As many applications depend on Redis as their key-value store, you will eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. -## Use Redis with Docker executor +## Use Redis with the Docker executor -If you are using our Docker integration you basically have everything set up -already. +If you are using GitLab's Runner Docker integration you basically have +everything set up already. First, in your `.gitlab-ci.yml` add: @@ -29,7 +29,7 @@ framework. If you want to use any other version of Redis, check the available versions on [Docker Hub](https://hub.docker.com/_/redis/). -## Use Redis with Shell executor +## Use Redis with the Shell executor Redis can also be used on manually configured servers that are using GitLab Runner with the Shell executor. -- cgit v1.2.1 From 06b86de996232b956129f2bb5dde4a7647f94f69 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 18:22:45 +0200 Subject: Clean up postgres CI example [ci skip] --- doc/ci/services/postgres.md | 116 +++++++++++++++++++++++++++----------------- 1 file changed, 71 insertions(+), 45 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index e57f8c5944a..f82a828756c 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -1,70 +1,96 @@ -## Using PostgreSQL +# Using PostgreSQL -It's possible to use PostgreSQL database test your apps during builds. +As many applications depend on PostgreSQL as their database, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. -### Use PostgreSQL with Docker executor +## Use PostgreSQL with the Docker executor -If you are using our Docker integration you basically have everything already. +If you are using GitLab's Runner with the Docker executor you basically have +everything set up already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - postgres +```yaml +services: + - postgres - variables: - # Configure postgres service (https://hub.docker.com/_/postgres/) - POSTGRES_DB: hello_world_test - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "" +variables: + POSTGRES_DB: nice_marmot + POSTGRES_USER: gitlab_runner + POSTGRES_PASSWORD: "" +``` -2. Configure your application to use the database: +And then configure your application to use PostgreSQL, for example: - Host: postgres - User: postgres - Password: postgres - Database: hello_world_test +```yaml +Host: localhost +User: gitlab_runner +Password: +Database: nice_marmot +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/postgres/). For example: `postgres:9.3`. +You can also use any other docker image available on [Docker Hub][hub-pg]. +For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`. -Example: https://gitlab.com/gitlab-examples/postgres/blob/master/.gitlab-ci.yml +The `postgres` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-pg]. -### Use PostgreSQL with Shell executor +## Use PostgreSQL with the Shell executor -It's possible to use PostgreSQL on manually configured servers that are using GitLab Runner with Shell executor. +You can also use PostgreSQL on manually configured servers that are using +GitLab Runner with the Shell executor. -1. First install the PostgreSQL server: +First install the PostgreSQL server: - sudo apt-get install -y postgresql postgresql-client libpq-dev +```bash +sudo apt-get install -y postgresql postgresql-client libpq-dev +``` -2. Create an user: +Then create a user: - # Install the database packages - sudo apt-get install -y postgresql postgresql-client libpq-dev +```bash +# Login to PostgreSQL +sudo -u postgres psql -d template1 - # Login to PostgreSQL - sudo -u postgres psql -d template1 +# Create a user for GitLab Runner that can create databases +# Do not type the 'template1=#', this is part of the prompt +template1=# CREATE USER gitlab_runner CREATEDB; - # Create a user for runner - # Do not type the 'template1=#', this is part of the prompt - template1=# CREATE USER runner CREATEDB; +# Create the database & grant all privileges on database +template1=# CREATE DATABASE nice_marmot OWNER gitlab_runner; - # Create the database & grant all privileges on database - template1=# CREATE DATABASE hello_world_test OWNER runner; +# Quit the database session +template1=# \q +``` - # Quit the database session - template1=# \q +Try to connect to database: -3. Try to connect to database: +```bash +# Try connecting to the new database with the new user +sudo -u gitlab_runner -H psql -d nice_marmot - # Try connecting to the new database with the new user - sudo -u gitlab-runner -H psql -d hello_world_test +# Quit the database session +nice_marmot> \q +``` - # Quit the database session - hello_world_test> \q +Finally, configure your application to use the database: -4. Configure your application to use the database: +```bash +Host: localhost +User: gitlab_runner +Password: +Database: nice_marmot +``` - Host: localhost - User: runner - Password: - Database: hello_world_test +## Example project + +We have set up an [Example PostgreSQL Project][postgres-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-pg]: https://hub.docker.com/_/postgres/ +[postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres -- cgit v1.2.1 From 47e81da8c2746196ad0506d30720c775a538ebdb Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:11:11 +0200 Subject: Clean up mysql CI example --- doc/ci/services/mysql.md | 137 +++++++++++++++++++++++++++++++---------------- 1 file changed, 90 insertions(+), 47 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 3155af6b3e1..7f50b4d76c8 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -1,72 +1,115 @@ -## Using MySQL +# Using MySQL -It's possible to use MySQL database test your apps during builds. +As many applications depend on MySQL as their database, you will eventually +need it in order for your tests to run. Below you are guided how to do this +with the Docker and Shell executors of GitLab Runner. -### Use MySQL with Docker executor +## Use MySQL with the Docker executor -If you are using our Docker integration you basically have everything already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - mysql +```yaml +services: + - mysql - variables: - # Configure mysql service (https://hub.docker.com/_/mysql/) - MYSQL_DATABASE: hello_world_test - MYSQL_ROOT_PASSWORD: mysql +variables: + # Configure mysql environment variables (https://hub.docker.com/_/mysql/) + MYSQL_DATABASE: el_duderino + MYSQL_ROOT_PASSWORD: mysql_strong_password +``` -2. Configure your application to use the database: +And then configure your application to use the database, for example: - Host: mysql - User: root - Password: mysql - Database: hello_world_test +```yaml +Host: localhost +User: root +Password: mysql_strong_password +Database: el_duderino +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/mysql/). For example: `mysql:5.5`. +You can also use any other docker image available on [Docker Hub][hub-mysql]. +For example, to use MySQL 5.5 the service becomes `mysql:5.5`. -Example: https://gitlab.com/gitlab-examples/mysql/blob/master/.gitlab-ci.yml +The `mysql` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-mysql]. -### Use MySQL with Shell executor +## Use MySQL with the Shell executor -It's possible to use MySQL on manually configured servers that are using GitLab Runner with Shell executor. +You can also use MySQL on manually configured servers that are using +GitLab Runner with the Shell executor. -1. First install the MySQL server: - - sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev +First install the MySQL server: - # Pick a MySQL root password (can be anything), type it and press enter - # Retype the MySQL root password and press enter +```bash +sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev +``` -2. Create an user: +Pick a MySQL root password (can be anything), and type it twice when asked. - mysql -u root -p +*Note: As a security measure you can run `mysql_secure_installation` to +remove anonymous users, drop the test database and disable remote logins with +the root user.* - # Create a user which will be used by your apps - # do not type the 'mysql>', this is part of the prompt - # change $password in the command below to a real password you pick - mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; +The next step is to create a user, so login to MySQL as root: - # Ensure you can use the InnoDB engine which is necessary to support long indexes - # If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting "innodb = off" - mysql> SET storage_engine=INNODB; +```bash +mysql -u root -p +``` - # Create the database - mysql> CREATE DATABASE IF NOT EXISTS `hello_world_test` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. - # Grant necessary permissions on the database - mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `hello_world_test`.* TO 'runner'@'localhost'; +*Note: Do not type `mysql>`, this is part of the MySQL prompt.* - # Quit the database session - mysql> \q +```bash +mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; +``` -3. Try to connect to database: +Create the database: - sudo -u gitlab-runner -H mysql -u runner -p -D hello_world_test +```bash +mysql> CREATE DATABASE IF NOT EXISTS `el_duderino` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; +``` -4. Configure your application to use the database: +Grant the necessary permissions on the database: - Host: localhost - User: runner - Password: $password - Database: hello_world_test +```bash +mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `el_duderino`.* TO 'runner'@'localhost'; +``` + +If all went well you can now quit the database session: + +```bash +mysql> \q +``` + +Now, try to connect to the newly created database to check that everything is +in place: + +```bash +mysql -u runner -p -D el_duderino +``` + +As a final step, configure your application to use the database, for example: + +```bash +Host: localhost +User: runner +Password: $password +Database: el_duderino +``` + +## Example project + +We have set up an [Example MySQL Project][mysql-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-mysql]: https://hub.docker.com/_/mysql/ +[mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql -- cgit v1.2.1 From 9dc91f46df377d6220928d6292dac73bc6bae295 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:11:27 +0200 Subject: Add link to runners doc [ci skip] --- doc/ci/services/postgres.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index f82a828756c..b8436f4f7ca 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -6,8 +6,8 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use PostgreSQL with the Docker executor -If you are using GitLab's Runner with the Docker executor you basically have -everything set up already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -21,7 +21,7 @@ variables: POSTGRES_PASSWORD: "" ``` -And then configure your application to use PostgreSQL, for example: +And then configure your application to use the database, for example: ```yaml Host: localhost -- cgit v1.2.1 From cf44dc510786775a1303384d858eb0a7bf33c34f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:35:08 +0200 Subject: Add note about the various phpenv tools --- doc/ci/languages/php.md | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index 60079c090d3..dacb67fa3ff 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -179,6 +179,14 @@ Using phpenv also allows to easily configure the PHP environment with: phpenv config-add my_config.ini ``` +*__Important note:__ It seems `phpenv/phpenv` + [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork + at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring + the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also + seems like a good alternative. Picking any of the mentioned tools will work + with the basic phpenv commands. Guiding you to choose the right phpenv is out + of the scope of this tutorial.* + ### Install custom extensions Since this is a pretty bare installation of the PHP environment, you may need -- cgit v1.2.1 From 4ea0f06470620064c5d23e28c778f83296aeb551 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:35:25 +0200 Subject: Fix wrong naming of services keyword [ci skip] --- doc/ci/services/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 0856b7679c2..1ebb0a4a250 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,6 +1,6 @@ ## GitLab CI Services -GitLab CI uses the `service` keyword to define what docker containers should be +GitLab CI uses the `services` keyword to define what docker containers should be linked with your base image. Below is a list of examples you may use. + [Using MySQL](mysql.md) -- cgit v1.2.1 From 0c22635aa2036ab3394c7e7bd99fd61ddd99c43c Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:41:56 +0200 Subject: More redis CI example clean up --- doc/ci/services/redis.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index e6add57ca76..b281e8f9f60 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -6,8 +6,8 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use Redis with the Docker executor -If you are using GitLab's Runner Docker integration you basically have -everything set up already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -19,15 +19,15 @@ services: Then you need to configure your application to use the Redis database, for example: -```bash +```yaml Host: redis ``` And that's it. Redis will now be available to be used within your testing framework. -If you want to use any other version of Redis, check the available versions -on [Docker Hub](https://hub.docker.com/_/redis/). +You can also use any other docker image available on [Docker Hub][hub-redis]. +For example, to use Redis 2.8 the service becomes `redis:2.8`. ## Use Redis with the Shell executor @@ -52,7 +52,7 @@ sudo -u gitlab-runner -H redis-cli Finally, configure your application to use the database, for example: -```bash +```yaml Host: localhost ``` @@ -65,4 +65,5 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. +[hub-redis]: https://hub.docker.com/_/redis/ [redis-example-repo]: https://gitlab.com/gitlab-examples/redis -- cgit v1.2.1 From bbe652cdaf11dae7fa917bac1ffa4a02f2099026 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 22:26:41 +0200 Subject: More postgres CI service example cleanup --- doc/ci/services/postgres.md | 55 ++++++++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 20 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index b8436f4f7ca..db5be070ee7 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -13,11 +13,11 @@ First, in your `.gitlab-ci.yml` add: ```yaml services: - - postgres + - postgres:latest variables: POSTGRES_DB: nice_marmot - POSTGRES_USER: gitlab_runner + POSTGRES_USER: runner POSTGRES_PASSWORD: "" ``` @@ -25,7 +25,7 @@ And then configure your application to use the database, for example: ```yaml Host: localhost -User: gitlab_runner +User: runner Password: Database: nice_marmot ``` @@ -47,39 +47,54 @@ First install the PostgreSQL server: sudo apt-get install -y postgresql postgresql-client libpq-dev ``` -Then create a user: +The next step is to create a user, so login to PostgreSQL: ```bash -# Login to PostgreSQL sudo -u postgres psql -d template1 +``` -# Create a user for GitLab Runner that can create databases -# Do not type the 'template1=#', this is part of the prompt -template1=# CREATE USER gitlab_runner CREATEDB; +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. -# Create the database & grant all privileges on database -template1=# CREATE DATABASE nice_marmot OWNER gitlab_runner; +*__Note:__ Do not type `template1=#`, this is part of the PostgreSQL prompt.* -# Quit the database session -template1=# \q +```bash +template1=# CREATE USER runner WITH PASSWORD '$password' CREATEDB; ``` -Try to connect to database: +*__Note:__ Notice that we created the user with the privilege to be able to +create databases (`CREATEDB`). In the following steps we will create a database +explicitly for that user but having that privilege can be useful if in your +testing framework you have tools that drop and create databases.* + +Create the database and grant all privileges on it for the user `runner`: ```bash -# Try connecting to the new database with the new user -sudo -u gitlab_runner -H psql -d nice_marmot +template1=# CREATE DATABASE nice_marmot OWNER runner; +``` + +If all went well you can now quit the database session: -# Quit the database session -nice_marmot> \q +```bash +template1=# \q ``` -Finally, configure your application to use the database: +Now, try to connect to the newly created database with the user `runner` to +check that everything is in place. ```bash +psql -U runner -h localhost -d nice_marmot -W +``` + +*__Note:__ We are explicitly telling `psql` to connect to localhost in order +to use the md5 authentication. If you omit this step you will be denied access.* + +Finally, configure your application to use the database, for example: + +```yaml Host: localhost -User: gitlab_runner -Password: +User: runner +Password: $password Database: nice_marmot ``` -- cgit v1.2.1 From d55dcd11d59ebfadf49290746c5f30c2e4780c6b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 22:27:23 +0200 Subject: Use the latest docker image in mysql CI example [ci skip] --- doc/ci/services/mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 7f50b4d76c8..44bc613e6f7 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -13,7 +13,7 @@ First, in your `.gitlab-ci.yml` add: ```yaml services: - - mysql + - mysql:latest variables: # Configure mysql environment variables (https://hub.docker.com/_/mysql/) -- cgit v1.2.1 From b312edaf440b729dad02749259577f6e0589c061 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 23:29:49 +0200 Subject: More clean up to the CI example on how to use docker images [ci skip] --- doc/ci/docker/using_docker_images.md | 100 +++++++++++++---------------------- 1 file changed, 36 insertions(+), 64 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 488479418b1..8d4bd44053e 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,28 +1,29 @@ # Using Docker Images -GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. +GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use +[Docker Engine](https://www.docker.com/) to test and build any application. -Docker is an open-source project that allows to use predefined images to run -applications in independent "containers" that are run within a single Linux +Docker is an open-source project that allows you to use predefined images to +run applications in independent "containers" that are run within a single Linux instance. [Docker Hub][hub] has a rich database of prebuilt images that can be used to test and build your applications. Docker, when used with GitLab CI, runs each build in a separate and isolated -container using the predefined image that is set up in `.gitlab-ci.yml`. The -container is always euphemeral which means you get only one container per build. +container using the predefined image that is set up in +[`.gitlab-ci.yml`](../yaml/README.md). This makes it easier to have a simple and reproducible build environment that can also run on your workstation. The added benefit is that you can test all the commands that we will explore later from your shell, rather than having to -test them on a CI server. +test them on a dedicated CI server. -### Register docker runner +## Register docker runner To use GitLab Runner with docker you need to register a new runner to use the `docker` executor: ```bash -gitlab-ci-multi-runner register \ +gitlab-runner register \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ @@ -36,7 +37,7 @@ The registered runner will use the `ruby:2.1` docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. -### What is image +## What is image The `image` keyword is the name of the docker image that is present in the local Docker Engine (list all images with `docker images`) or any image that @@ -46,9 +47,9 @@ Hub please read the [Docker Fundamentals][] documentation. In short, with `image` we refer to the docker image, which will be used to create a container on which your build will run. -### What is service +## What is service -The `service` keyword defines just another docker image that is run during +The `services` keyword defines just another docker image that is run during your build and is linked to the docker image that the `image` keyword defines. This allows you to access the service image during build time. @@ -57,9 +58,12 @@ run a database container, eg. `mysql`. It's easier and faster to use an existing image and run it as an additional container than install `mysql` every time the project is built. -#### How is service linked to the build +You can see some widely used services examples in the relevant documentation of +[CI services examples](../services/README.md). -To better undestand how the container linking works, read +### How is service linked to the build + +To better understand how the container linking works, read [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). To summarize, if you add `mysql` as service to your application, the image will @@ -69,25 +73,27 @@ The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host named `mysql` instead of a socket or `localhost`. -### Overwrite image and services +## Overwrite image and services -See the section below. +See [How to use other images as services](#how-to-use-other-images-as-services). -### How to use other images as services +## How to use other images as services You are not limited to have only database services. You can add as many services you need to `.gitlab-ci.yml` or manually modify `config.toml`. Any image found at [Docker Hub][hub] can be used as a service. -### Define image and services from `.gitlab-ci.yml` +## Define image and services from `.gitlab-ci.yml` You can simply define an image that will be used for all jobs and a list of services that you want to use during build time. ```yaml image: ruby:2.2 + services: - postgres:9.3 + before_script: - bundle install @@ -117,24 +123,20 @@ test:2.2: - bundle exec rake spec ``` -### Define image and services in `config.toml` +## Define image and services in `config.toml` Look for the `[runners.docker]` section: ``` -... - [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] - -... ``` The image and services defined this way will be added to all builds run by that runner. -### Accessing the services +## Accessing the services Let's say that you need a Wordpress instance to test some API integration with your application. @@ -143,12 +145,8 @@ You can then use for example the [tutum/wordpress][] image in your `.gitlab-ci.yml`: ```yaml -... - services: - tutum/wordpress:latest - -... ``` When the build is run, `tutum/wordpress` will be started and you will have @@ -160,7 +158,7 @@ rules: 1. Everything after `:` is stripped 2. Backslash (`/`) is replaced with double underscores (`__`) -### Configuring services +## Configuring services Many services accept environment variables which allow you to easily change database names or set account names depending on the environment. @@ -171,46 +169,20 @@ service containers. For all possible configuration variables check the documentation of each image provided in their corresponding Docker hub page. -*Note: All variables will be passed to all service containers. It's not designed - to distinguish which variable should go where.* - -#### PostgreSQL service example - -To configure the database name for [postgres][postgres-hub] service, you need -to set the `POSTGRES_DB` variable: - -```yaml -... - -services: -- postgres - -variables: - POSTGRES_DB: gitlab +*Note: All variables will be passed to all services containers. It's not +designed to distinguish which variable should go where.* -... -``` - -For a real example visit . +### PostgreSQL service example -#### MySQL service example +See the specific documentation for +[using PostgreSQL as a service](../services/postgres.md). -To use the [mysql][mysql-hub] service with an empty password during the time of -build, you need to set the `MYSQL_ALLOW_EMPTY_PASSWORD` variable: +### MySQL service example -```yaml -... - -services: -- mysql - -variables: - MYSQL_ALLOW_EMPTY_PASSWORD: "yes" - -... -``` +See the specific documentation for +[using MySQL as a service](../services/mysql.md). -### How Docker integration works +## How Docker integration works Below is a high level overview of the steps performed by docker during build time. @@ -226,7 +198,7 @@ time. 1. Check exit status of build script. 1. Remove build container and all created service containers. -### How to debug a build locally +## How to debug a build locally *Note: The following commands are run without root privileges. You should be able to run docker with your regular user account.* -- cgit v1.2.1 From 1e7156ed701945349bef484d35c7f53f2ee4474b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 23:49:05 +0200 Subject: Use the name of the linked containers instead of localhost https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1917#note_2833978 --- doc/ci/services/mysql.md | 5 ++++- doc/ci/services/postgres.md | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 44bc613e6f7..c66d77122b2 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -24,12 +24,15 @@ variables: And then configure your application to use the database, for example: ```yaml -Host: localhost +Host: mysql User: root Password: mysql_strong_password Database: el_duderino ``` +If you are wondering why we used `mysql` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + You can also use any other docker image available on [Docker Hub][hub-mysql]. For example, to use MySQL 5.5 the service becomes `mysql:5.5`. diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index db5be070ee7..17d21dbda1c 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -24,12 +24,15 @@ variables: And then configure your application to use the database, for example: ```yaml -Host: localhost +Host: postgres User: runner Password: Database: nice_marmot ``` +If you are wondering why we used `postgres` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + You can also use any other docker image available on [Docker Hub][hub-pg]. For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`. -- cgit v1.2.1 From baa97175abd25eb566d7273bc188a6459639abf3 Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Fri, 11 Dec 2015 11:09:34 -0600 Subject: Clarify cache behavior --- doc/ci/yaml/README.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3dbf1afc7a9..7e2edb945da 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -113,6 +113,9 @@ The YAML-defined variables are also set to all created service containers, thus ### cache `cache` is used to specify list of files and directories which should be cached between builds. +Caches are stored according to the branch/ref and the job name. Caches are not +currently shared between different job names or between branches/refs. This means +caching will benefit you if you push subsequent commits to an existing feature branch. **The global setting allows to specify default cached files for all jobs.** -- cgit v1.2.1 From fc9fbdce44570042c79ff9a66289890bcb479524 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 10 Dec 2015 15:56:20 +0200 Subject: Clean up ci yaml doc [ci skip] * Fix headings so that they are picked up by doc.gitlab.com * Stick to 80 characters * Clean up some examples --- doc/ci/yaml/README.md | 345 +++++++++++++++++++++++++++++++------------------- 1 file changed, 212 insertions(+), 133 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7e2edb945da..c5bdce2f778 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,9 +1,12 @@ # Configuration of your builds with .gitlab-ci.yml -From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) file (**.gitlab-ci.yml**) for the project configuration. -It is placed in the root of your repository and contains definitions of how your project should be built. -The YAML file defines a set of jobs with constraints stating when they should be run. -The jobs are defined as top-level elements with a name and always have to contain the `script` clause: +From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) +file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root +of your repository and contains definitions of how your project should be built. + +The YAML file defines a set of jobs with constraints stating when they should +be run. The jobs are defined as top-level elements with a name and always have +to contain the `script` clause: ```yaml job1: @@ -13,15 +16,21 @@ job2: script: "execute-script-for-job2" ``` -The above example is the simplest possible CI configuration with two separate jobs, -where each of the jobs executes a different command. -Of course a command can execute code directly (`./configure;make;make install`) or run a script (`test.sh`) in the repository. +The above example is the simplest possible CI configuration with two separate +jobs, where each of the jobs executes a different command. + +Of course a command can execute code directly (`./configure;make;make install`) +or run a script (`test.sh`) in the repository. -Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. -What is important, is that each job is run independently from each other. +Jobs are used to create builds, which are then picked up by +[runners](../runners/README.md) and executed within the environment of the +runner. What is important, is that each job is run independently from each +other. ## .gitlab-ci.yml -The YAML syntax allows for using more complex job specifications than in the above example: + +The YAML syntax allows for using more complex job specifications than in the +above example: ```yaml image: ruby:2.1 @@ -46,26 +55,31 @@ job1: - docker ``` -There are a few `keywords` that can't be used as job names: +There are a few reserved `keywords` that **cannot** be used as job names: -| keyword | required | description | +| Keyword | Required | Description | |---------------|----------|-------------| -| image | optional | Use docker image, covered in [Use Docker](../docker/README.md) | -| services | optional | Use docker services, covered in [Use Docker](../docker/README.md) | -| stages | optional | Define build stages | -| types | optional | Alias for `stages` | -| before_script | optional | Define commands prepended for each job's script | -| variables | optional | Define build variables | -| cache | optional | Define list of files that should be cached between subsequent runs | +| image | no | Use docker image, covered in [Use Docker](../docker/README.md) | +| services | no | Use docker services, covered in [Use Docker](../docker/README.md) | +| stages | no | Define build stages | +| types | no | Alias for `stages` | +| before_script | no | Define commands that run before each job's script | +| variables | no | Define build variables | +| cache | no | Define list of files that should be cached between subsequent runs | ### image and services -This allows to specify a custom Docker image and a list of services that can be used for time of the build. -The configuration of this feature is covered in separate document: [Use Docker](../docker/README.md). + +This allows to specify a custom Docker image and a list of services that can be +used for time of the build. The configuration of this feature is covered in +separate document: [Use Docker](../docker/README.md). ### before_script -`before_script` is used to define the command that should be run before all builds, including deploy builds. This can be an array or a multiline string. + +`before_script` is used to define the command that should be run before all +builds, including deploy builds. This can be an array or a multi-line string. ### stages + `stages` is used to define build stages that can be used by jobs. The specification of `stages` allows for having flexible multi stage pipelines. @@ -75,7 +89,8 @@ The ordering of elements in `stages` defines the ordering of builds' execution: 1. Builds of next stage are run after success. Let's consider the following example, which defines 3 stages: -``` + +```yaml stages: - build - test @@ -86,21 +101,26 @@ stages: 1. If all jobs of `build` succeeds, the `test` jobs are executed in parallel. 1. If all jobs of `test` succeeds, the `deploy` jobs are executed in parallel. 1. If all jobs of `deploy` succeeds, the commit is marked as `success`. -1. If any of the previous jobs fails, the commit is marked as `failed` and no jobs of further stage are executed. +1. If any of the previous jobs fails, the commit is marked as `failed` and no + jobs of further stage are executed. There are also two edge cases worth mentioning: -1. If no `stages` is defined in `.gitlab-ci.yml`, then by default the `build`, `test` and `deploy` are allowed to be used as job's stage by default. +1. If no `stages` is defined in `.gitlab-ci.yml`, then by default the `build`, + `test` and `deploy` are allowed to be used as job's stage by default. 2. If a job doesn't specify `stage`, the job is assigned the `test` stage. ### types + Alias for [stages](#stages). ### variables -**This feature requires `gitlab-runner` with version equal or greater than 0.5.0.** -GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build environment. -The variables are stored in repository and are meant to store non-sensitive project configuration, ie. RAILS_ENV or DATABASE_URL. +_**Note:** Introduced in GitLab Runner v0.5.0._ + +GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build +environment. The variables are stored in the git repository and are meant to +store non-sensitive project configuration, for example: ```yaml variables: @@ -109,18 +129,23 @@ variables: These variables can be later used in all executed commands and scripts. -The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +The YAML-defined variables are also set to all created service containers, +thus allowing to fine tune them. ### cache -`cache` is used to specify list of files and directories which should be cached between builds. -Caches are stored according to the branch/ref and the job name. Caches are not -currently shared between different job names or between branches/refs. This means -caching will benefit you if you push subsequent commits to an existing feature branch. -**The global setting allows to specify default cached files for all jobs.** +`cache` is used to specify a list of files and directories which should be +cached between builds. Caches are stored according to the branch/ref and the +job name. They are not currently shared between different job names or between +branches/refs, which means that caching will benefit you if you push subsequent +commits to an existing feature branch. + +If `cache` is defined outside the scope of the jobs, it means it is set +globally and all jobs will use its definition. To cache all git untracked files and files in `binaries`: -``` + +```yaml cache: untracked: true paths: @@ -128,9 +153,10 @@ cache: ``` ## Jobs -`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. -Each job has to have a unique `job_name`, which is not one of the keywords mentioned above. -A job is defined by a list of parameters that define the build behaviour. + +`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job +must have a unique name, which is not one of the Keywords mentioned above. +A job is defined by a list of parameters that define the build behavior. ```yaml job_name: @@ -148,21 +174,22 @@ job_name: allow_failure: true ``` -| keyword | required | description | +| Keyword | Required | Description | |---------------|----------|-------------| -| script | required | Defines a shell script which is executed by runner | -| stage | optional (default: test) | Defines a build stage | -| type | optional | Alias for `stage` | -| only | optional | Defines a list of git refs for which build is created | -| except | optional | Defines a list of git refs for which build is not created | -| tags | optional | Defines a list of tags which are used to select runner | -| allow_failure | optional | Allow build to fail. Failed build doesn't contribute to commit status | -| when | optional | Define when to run build. Can be `on_success`, `on_failure` or `always` | -| artifacts | optional | Define list build artifacts | -| cache | optional | Define list of files that should be cached between subsequent runs | +| script | yes | Defines a shell script which is executed by runner | +| stage | no (default: `test`) | Defines a build stage | +| type | no | Alias for `stage` | +| only | no | Defines a list of git refs for which build is created | +| except | no | Defines a list of git refs for which build is not created | +| tags | no | Defines a list of tags which are used to select runner | +| allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status | +| when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` | +| artifacts | no | Define list build artifacts | +| cache | no | Define list of files that should be cached between subsequent runs | ### script -`script` is a shell script which is executed by runner. The shell script is prepended with `before_script`. + +`script` is a shell script which is executed by the runner. For example: ```yaml job: @@ -170,6 +197,7 @@ job: ``` This parameter can also contain several commands using an array: + ```yaml job: script: @@ -178,31 +206,45 @@ job: ``` ### stage -`stage` allows to group build into different stages. Builds of the same `stage` are executed in `parallel`. -For more info about the use of `stage` please check the [stages](#stages). + +`stage` allows to group build into different stages. Builds of the same `stage` +are executed in `parallel`. For more info about the use of `stage` please check +[stages](#stages). ### only and except -This are two parameters that allow for setting a refs policy to limit when jobs are built: -1. `only` defines the names of branches and tags for which job will be built. -2. `except` defines the names of branches and tags for which the job wil **not** be built. -There are a few rules that apply to usage of refs policy: +`only` and `except` are two parameters that set a refs policy to limit when +jobs are built: -1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. -1. `only` and `except` allow for using the regexp expressions. -1. `only` and `except` allow for using special keywords: `branches` and `tags`. -These names can be used for example to exclude all tags and all branches. +1. `only` defines the names of branches and tags for which the job will be + built. +2. `except` defines the names of branches and tags for which the job will + **not** be built. + +There are a few rules that apply to the usage of refs policy: + +* `only` and `except` are inclusive. If both `only` and `except` are defined + in a job specification, the ref is filtered by `only` and `except`. +* `only` and `except` allow the use of regular expressions. +* `only` and `except` allow the use of special keywords: `branches` and `tags`. +* `only` and `except` allow to specify a repository path to filter jobs for + forks. + +In the example below, `job` will run only for refs that start with `issue-`, +whereas all branches will be skipped. ```yaml job: + # use regexp only: - - /^issue-.*$/ # use regexp + - /^issue-.*$/ + # use special keyword except: - - branches # use special keyword + - branches ``` -1. `only` and `except` allow for specify repository path to filter jobs for forks. -The repository path can be used to have jobs executed only for parent repository. +The repository path can be used to have jobs executed only for the parent +repository and not forks: ```yaml job: @@ -211,33 +253,47 @@ job: except: - master@gitlab-org/gitlab-ce ``` -The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . + +The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, +except master. ### tags -`tags` is used to select specific runners from the list of all runners that are allowed to run this project. -During registration of a runner, you can specify the runner's tags, ie.: `ruby`, `postgres`, `development`. -`tags` allow you to run builds with runners that have the specified tags assigned: +`tags` is used to select specific runners from the list of all runners that are +allowed to run this project. -``` +During the registration of a runner, you can specify the runner's tags, for +example `ruby`, `postgres`, `development`. + +`tags` allow you to run builds with runners that have the specified tags +assigned to them: + +```yaml job: tags: - ruby - postgres ``` -The above specification will make sure that `job` is built by a runner that have `ruby` AND `postgres` tags defined. +The specification above, will make sure that `job` is built by a runner that +has both `ruby` AND `postgres` tags defined. ### when -`when` is used to implement jobs that are run in case of failure or despite the failure. + +`when` is used to implement jobs that are run in case of failure or despite the +failure. `when` can be set to one of the following values: -1. `on_success` - execute build only when all builds from prior stages succeeded. This is the default. -1. `on_failure` - execute build only when at least one build from prior stages failed. +1. `on_success` - execute build only when all builds from prior stages + succeeded. This is the default. +1. `on_failure` - execute build only when at least one build from prior stages + failed. 1. `always` - execute build despite the status of builds from prior stages. -``` +For example: + +```yaml stages: - build - cleanup_build @@ -245,28 +301,28 @@ stages: - deploy - cleanup -build: +build_job: stage: build script: - make build -cleanup_build: +cleanup_build_job: stage: cleanup_build script: - cleanup build when failed when: on_failure -test: +test_job: stage: test script: - make test -deploy: +deploy_job: stage: deploy script: - make deploy -cleanup: +cleanup_job: stage: cleanup script: - cleanup after builds @@ -274,84 +330,107 @@ cleanup: ``` The above script will: -1. Execute `cleanup_build` only when the `build` failed, -2. Always execute `cleanup` as the last step in pipeline. + +1. Execute `cleanup_build_job` only when `build_job` fails +2. Always execute `cleanup_job` as the last step in pipeline. ### artifacts -`artifacts` is used to specify list of files and directories which should be attached to build after success. -1. Send all files in `binaries` and `.config`: +_**Note:** Introduced in GitLab Runner v0.7.0._ - artifacts: - paths: - - binaries/ - - .config +`artifacts` is used to specify list of files and directories which should be +attached to build after success. Below are some examples. -2. Send all git untracked files: +Send all files in `binaries` and `.config`: - artifacts: - untracked: true +```yaml +artifacts: + paths: + - binaries/ + - .config +``` -3. Send all git untracked files and files in `binaries`: +Send all git untracked files: - artifacts: - untracked: true - paths: - - binaries/ +```yaml +artifacts: + untracked: true +``` + +Send all git untracked files and files in `binaries`: -The artifacts will be send after the build success to GitLab and will be accessible in GitLab interface to download. +```yaml +artifacts: + untracked: true + paths: + - binaries/ +``` -This feature requires GitLab Runner v0.7.0 or higher. +The artifacts will be send after a successful build success to GitLab, and will +be accessible in the GitLab UI to download. ### cache -`cache` is used to specify list of files and directories which should be cached between builds. -1. Cache all files in `binaries` and `.config`: +_**Note:** Introduced in GitLab Runner v0.7.0._ - rspec: - script: test - cache: - paths: - - binaries/ - - .config +`cache` is used to specify list of files and directories which should be cached +between builds. Below are some examples: -2. Cache all git untracked files: +Cache all files in `binaries` and `.config`: - rspec: - script: test - cache: - untracked: true - -3. Cache all git untracked files and files in `binaries`: +```yaml +rspec: + script: test + cache: + paths: + - binaries/ + - .config +``` - rspec: - script: test - cache: - untracked: true - paths: - - binaries/ +Cache all git untracked files: -4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: +```yaml +rspec: + script: test + cache: + untracked: true +``` + +Cache all git untracked files and files in `binaries`: + +```yaml +rspec: + script: test + cache: + untracked: true + paths: + - binaries/ +``` - cache: - paths: - - my/files - - rspec: - script: test - cache: - paths: - - binaries/ +Locally defined cache overwrites globally defined options. This will cache only +`binaries/`: -The cache is provided on best effort basis, so don't expect that cache will be present. -For implementation details please check GitLab Runner. +```yaml +cache: + paths: + - my/files -This feature requires GitLab Runner v0.7.0 or higher. +rspec: + script: test + cache: + paths: + - binaries/ +``` +The cache is provided on best effort basis, so don't expect that cache will be +always present. For implementation details please check GitLab Runner. ## Validate the .gitlab-ci.yml + Each instance of GitLab CI has an embedded debug tool called Lint. -You can find the link to the Lint in the project's settings page or use short url `/lint`. +You can find the link under `/ci/lint` of your gitlab instance. ## Skipping builds -There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped + +If your commit message contains `[ci skip]`, the commit will be created but the +builds will be skipped. -- cgit v1.2.1 From d2b5ef1c1e1011b5ccc8a3142b0147ce29632963 Mon Sep 17 00:00:00 2001 From: Cauan Cabral Date: Wed, 16 Dec 2015 14:23:02 -0300 Subject: Remove CI_BUILD_BEFORE_SHA from CI documentation As pointed in #3210, the environment variable isn't usable any more. --- doc/ci/variables/README.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 022afb70042..b99ea25a3fe 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -27,7 +27,6 @@ The API_TOKEN will take the Secure Variable value: `SECURE`. | **CI_BUILD_TAG** | 0.5 | The commit tag name. Present only when building tags. | | **CI_BUILD_NAME** | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | | **CI_BUILD_STAGE** | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | -| **CI_BUILD_BEFORE_SHA** | all | The first commit that were included in push request | | **CI_BUILD_REF_NAME** | all | The branch or tag name for which project is built | | **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally | | **CI_BUILD_REPO** | all | The URL to clone the Git repository | @@ -40,7 +39,6 @@ The API_TOKEN will take the Secure Variable value: `SECURE`. Example values: ```bash -export CI_BUILD_BEFORE_SHA="9df57456fa9de2a6d335ca5edf9750ed812b9df0" export CI_BUILD_ID="50" export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_BUILD_REF_NAME="master" -- cgit v1.2.1 From 8579fea2cfe4c822dae12efaf50f9e01adbc752e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 17 Dec 2015 08:53:00 +0200 Subject: Add info on using private Docker registries in CI [ci skip] --- doc/ci/docker/using_docker_images.md | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) (limited to 'doc/ci') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 8d4bd44053e..31458d61674 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,11 +1,11 @@ # Using Docker Images -GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use +GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use [Docker Engine](https://www.docker.com/) to test and build any application. Docker is an open-source project that allows you to use predefined images to run applications in independent "containers" that are run within a single Linux -instance. [Docker Hub][hub] has a rich database of prebuilt images that can be +instance. [Docker Hub][hub] has a rich database of pre-built images that can be used to test and build your applications. Docker, when used with GitLab CI, runs each build in a separate and isolated @@ -136,6 +136,24 @@ Look for the `[runners.docker]` section: The image and services defined this way will be added to all builds run by that runner. +## Define an image from a private Docker registry + +Starting with GitLab Runner 0.6.0, you are able to define images located to +private registries that could also require authentication. + +All you have to do is be explicit on the image definition in `.gitlab-ci.yml`. + +```yaml +image: my.registry.tld:5000/namepace/image:tag +``` + +In the example above, GitLab Runner will look at `my.registry.tld:5000` for the +image `namespace/image:tag`. + +If the repository is private you need to authenticate your GitLab Runner in the +registry. Learn how to do that on +[GitLab Runner's documentation][runner-priv-reg]. + ## Accessing the services Let's say that you need a Wordpress instance to test some API integration with @@ -258,3 +276,4 @@ creation. [tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ [postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ [mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ +[runner-priv-reg]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry -- cgit v1.2.1 From c019154a911e751efba707f0323797b240775b08 Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Thu, 17 Dec 2015 13:01:57 -0600 Subject: Clarify Windows shell executor artifact upload support --- doc/ci/yaml/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc/ci') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index c5bdce2f778..fd0d49de4e4 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -336,7 +336,8 @@ The above script will: ### artifacts -_**Note:** Introduced in GitLab Runner v0.7.0._ +_**Note:** Introduced in GitLab Runner v0.7.0. Also, the Windows shell executor + does not currently support artifact uploads._ `artifacts` is used to specify list of files and directories which should be attached to build after success. Below are some examples. -- cgit v1.2.1