From 7a8a892efdf59925a95cdf6504f7c74c31b87eeb Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Fri, 25 Sep 2015 17:12:41 +0200 Subject: Add "rake gitlab:list_repos" task --- doc/raketasks/list_repos.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 doc/raketasks/list_repos.md (limited to 'doc') diff --git a/doc/raketasks/list_repos.md b/doc/raketasks/list_repos.md new file mode 100644 index 00000000000..476428eb4f5 --- /dev/null +++ b/doc/raketasks/list_repos.md @@ -0,0 +1,30 @@ +# Listing repository directories + +You can print a list of all Git repositories on disk managed by +GitLab with the following command: + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production +``` + +If you only want to list projects with recent activity you can pass +a date with the 'SINCE' environment variable. The time you specify +is parsed by the Rails [TimeZone#parse +function](http://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html#method-i-parse). + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos SINCE='Sep 1 2015' + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production SINCE='Sep 1 2015' +``` + +Note that the projects listed are NOT sorted by activity; they use +the default ordering of the GitLab Rails application. -- cgit v1.2.1 From 5bcd0efe3e0b1fef06147d87f843adac717d7c42 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Fri, 25 Sep 2015 18:31:54 +0200 Subject: Add parallel-rsync-repos script and start docs --- doc/operations/rsyncing_repositories.md | 87 +++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 doc/operations/rsyncing_repositories.md (limited to 'doc') diff --git a/doc/operations/rsyncing_repositories.md b/doc/operations/rsyncing_repositories.md new file mode 100644 index 00000000000..231e09f0462 --- /dev/null +++ b/doc/operations/rsyncing_repositories.md @@ -0,0 +1,87 @@ +# Moving repositories managed by GitLab + +Sometimes you need to move all repositories managed by GitLab to +another filesystem or another server. In this document we will look +at some of the ways you can copy all your repositories from +`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`. + +We will look at three scenarios: the target directory is empty, the +target directory contains an outdated copy of the repositories, and +how to deal with thousands of repositories. + +**Each of the approaches we list can/will overwrite data in the +target directory `/mnt/gitlab/repositories`. Do not mix up the +source and the target.** + +## Target directory is empty: use a tar pipe + +If the target directory `/mnt/gitlab/repositories` is empty the +simplest thing to do is to use a tar pipe. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to see progress, replace `-xf` with `-xvf`. + +### Tar pipe to another server + +You can also use a tar pipe to copy data to another server. If your +'git' user has SSH access to the newserver as 'git@newserver', you +can pipe the data through SSH. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + ssh git@newserver tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to compress the data before it goes over the network +(which will cost you CPU cycles) you can replace `ssh` with `ssh +-C`. + +## The target directory contains an outdated copy of the repositories: use rsync + +In this scenario it is better to use rsync. This utility is either +already installed on your system or easily installable via apt, yum +etc. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + /mnt/gitlab/repositories +``` + +The `/.` in the command above is very important, without it you can +easily get the wrong directory structure in the target directory. +If you want to see progress, replace `-a` with `-av`. + +### Single rsync to another server + +If the 'git' user on your source system has SSH access to the target +server you can send the repositories over the network with rsync. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + git@newserver:/mnt/gitlab/repositories +``` + +## Thousands of Git repositories: use one rsync per repository + +Every time you start an rsync job it has to inspect all files in +the source directory, all files in the target directory, and then +decide what files to copy or not. If the source or target directory +has many contents this startup phase of rsync can become a burden +for your GitLab server. In cases like this you can make rsync's +life easier by dividing its work in smaller pieces, and sync one +repository at a time. + +In addition to rsync we will use [GNU +Parallel](http://www.gnu.org/software/parallel/). This utility is +not included in GitLab so you need to install it yourself with apt +or yum. Also note that the GitLab scripts we used below were added +in GitLab 8.???. + -- cgit v1.2.1 From 9f3984b5e8fb261eb24be76ec548d83c43d58b96 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Fri, 25 Sep 2015 18:32:51 +0200 Subject: Rename doc file --- doc/operations/moving_repositories.md | 87 +++++++++++++++++++++++++++++++++ doc/operations/rsyncing_repositories.md | 87 --------------------------------- 2 files changed, 87 insertions(+), 87 deletions(-) create mode 100644 doc/operations/moving_repositories.md delete mode 100644 doc/operations/rsyncing_repositories.md (limited to 'doc') diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md new file mode 100644 index 00000000000..231e09f0462 --- /dev/null +++ b/doc/operations/moving_repositories.md @@ -0,0 +1,87 @@ +# Moving repositories managed by GitLab + +Sometimes you need to move all repositories managed by GitLab to +another filesystem or another server. In this document we will look +at some of the ways you can copy all your repositories from +`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`. + +We will look at three scenarios: the target directory is empty, the +target directory contains an outdated copy of the repositories, and +how to deal with thousands of repositories. + +**Each of the approaches we list can/will overwrite data in the +target directory `/mnt/gitlab/repositories`. Do not mix up the +source and the target.** + +## Target directory is empty: use a tar pipe + +If the target directory `/mnt/gitlab/repositories` is empty the +simplest thing to do is to use a tar pipe. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to see progress, replace `-xf` with `-xvf`. + +### Tar pipe to another server + +You can also use a tar pipe to copy data to another server. If your +'git' user has SSH access to the newserver as 'git@newserver', you +can pipe the data through SSH. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + ssh git@newserver tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to compress the data before it goes over the network +(which will cost you CPU cycles) you can replace `ssh` with `ssh +-C`. + +## The target directory contains an outdated copy of the repositories: use rsync + +In this scenario it is better to use rsync. This utility is either +already installed on your system or easily installable via apt, yum +etc. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + /mnt/gitlab/repositories +``` + +The `/.` in the command above is very important, without it you can +easily get the wrong directory structure in the target directory. +If you want to see progress, replace `-a` with `-av`. + +### Single rsync to another server + +If the 'git' user on your source system has SSH access to the target +server you can send the repositories over the network with rsync. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + git@newserver:/mnt/gitlab/repositories +``` + +## Thousands of Git repositories: use one rsync per repository + +Every time you start an rsync job it has to inspect all files in +the source directory, all files in the target directory, and then +decide what files to copy or not. If the source or target directory +has many contents this startup phase of rsync can become a burden +for your GitLab server. In cases like this you can make rsync's +life easier by dividing its work in smaller pieces, and sync one +repository at a time. + +In addition to rsync we will use [GNU +Parallel](http://www.gnu.org/software/parallel/). This utility is +not included in GitLab so you need to install it yourself with apt +or yum. Also note that the GitLab scripts we used below were added +in GitLab 8.???. + diff --git a/doc/operations/rsyncing_repositories.md b/doc/operations/rsyncing_repositories.md deleted file mode 100644 index 231e09f0462..00000000000 --- a/doc/operations/rsyncing_repositories.md +++ /dev/null @@ -1,87 +0,0 @@ -# Moving repositories managed by GitLab - -Sometimes you need to move all repositories managed by GitLab to -another filesystem or another server. In this document we will look -at some of the ways you can copy all your repositories from -`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`. - -We will look at three scenarios: the target directory is empty, the -target directory contains an outdated copy of the repositories, and -how to deal with thousands of repositories. - -**Each of the approaches we list can/will overwrite data in the -target directory `/mnt/gitlab/repositories`. Do not mix up the -source and the target.** - -## Target directory is empty: use a tar pipe - -If the target directory `/mnt/gitlab/repositories` is empty the -simplest thing to do is to use a tar pipe. - -``` -# As the git user -tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ - tar -C /mnt/gitlab/repositories -xf - -``` - -If you want to see progress, replace `-xf` with `-xvf`. - -### Tar pipe to another server - -You can also use a tar pipe to copy data to another server. If your -'git' user has SSH access to the newserver as 'git@newserver', you -can pipe the data through SSH. - -``` -# As the git user -tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ - ssh git@newserver tar -C /mnt/gitlab/repositories -xf - -``` - -If you want to compress the data before it goes over the network -(which will cost you CPU cycles) you can replace `ssh` with `ssh --C`. - -## The target directory contains an outdated copy of the repositories: use rsync - -In this scenario it is better to use rsync. This utility is either -already installed on your system or easily installable via apt, yum -etc. - -``` -# As the 'git' user -rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ - /mnt/gitlab/repositories -``` - -The `/.` in the command above is very important, without it you can -easily get the wrong directory structure in the target directory. -If you want to see progress, replace `-a` with `-av`. - -### Single rsync to another server - -If the 'git' user on your source system has SSH access to the target -server you can send the repositories over the network with rsync. - -``` -# As the 'git' user -rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ - git@newserver:/mnt/gitlab/repositories -``` - -## Thousands of Git repositories: use one rsync per repository - -Every time you start an rsync job it has to inspect all files in -the source directory, all files in the target directory, and then -decide what files to copy or not. If the source or target directory -has many contents this startup phase of rsync can become a burden -for your GitLab server. In cases like this you can make rsync's -life easier by dividing its work in smaller pieces, and sync one -repository at a time. - -In addition to rsync we will use [GNU -Parallel](http://www.gnu.org/software/parallel/). This utility is -not included in GitLab so you need to install it yourself with apt -or yum. Also note that the GitLab scripts we used below were added -in GitLab 8.???. - -- cgit v1.2.1 From 4dd7c2f1e0174f8de6be9c57f7296e64e1534af5 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Fri, 25 Sep 2015 18:35:41 +0200 Subject: Remove unwanted linebreak --- doc/operations/moving_repositories.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md index 231e09f0462..88b90e91316 100644 --- a/doc/operations/moving_repositories.md +++ b/doc/operations/moving_repositories.md @@ -39,8 +39,7 @@ tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ ``` If you want to compress the data before it goes over the network -(which will cost you CPU cycles) you can replace `ssh` with `ssh --C`. +(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`. ## The target directory contains an outdated copy of the repositories: use rsync -- cgit v1.2.1 From e0ef09d9a35bf001acbb89e4177d942f6db93e50 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Mon, 5 Oct 2015 18:02:32 +0200 Subject: Some more text in the doc --- doc/operations/moving_repositories.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md index 88b90e91316..d156f3ad777 100644 --- a/doc/operations/moving_repositories.md +++ b/doc/operations/moving_repositories.md @@ -16,7 +16,10 @@ source and the target.** ## Target directory is empty: use a tar pipe If the target directory `/mnt/gitlab/repositories` is empty the -simplest thing to do is to use a tar pipe. +simplest thing to do is to use a tar pipe. This method has low +overhead and tar is almost always already installed on your system. +However, it is not possible to resume an interrupted tar pipe: if +that happens then all data must be copied again. ``` # As the git user @@ -43,9 +46,11 @@ If you want to compress the data before it goes over the network ## The target directory contains an outdated copy of the repositories: use rsync -In this scenario it is better to use rsync. This utility is either -already installed on your system or easily installable via apt, yum -etc. +If the target directory already contains a partial / outdated copy +of the repositories it may be wasteful to copy all the data again +with tar. In this scenario it is better to use rsync. This utility +is either already installed on your system or easily installable +via apt, yum etc. ``` # As the 'git' user -- cgit v1.2.1 From ad37f58ebd97fee3c06a531e4067c8adb4c9ecc7 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Tue, 6 Oct 2015 12:42:05 +0200 Subject: Add explanation about parallel-rsync-repos --- doc/operations/moving_repositories.md | 55 ++++++++++++++++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md index d156f3ad777..a89602b367f 100644 --- a/doc/operations/moving_repositories.md +++ b/doc/operations/moving_repositories.md @@ -87,5 +87,58 @@ In addition to rsync we will use [GNU Parallel](http://www.gnu.org/software/parallel/). This utility is not included in GitLab so you need to install it yourself with apt or yum. Also note that the GitLab scripts we used below were added -in GitLab 8.???. +in GitLab 8.1. +** This process does not clean up repositories at the target location that no +longer exist at the source. ** If you start using your GitLab instance with +`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos` +after switching to the new repository storage directory. + +### Parallel rsync for all repositories known to GitLab + +This will sync repositories with 10 rsync processes at a time. + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos |\ + sudo -u git \ + /usr/bin/env JOBS=10 \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repoos \ + /var/opt/gitlab/git-data/repositories \ + /mnt/gitlab/repositories + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos |\ + sudo -u git -H \ + /usr/bin/env JOBS=10 \ + bin/parallel-rsync-repos \ + /home/git/repositories \ + /mnt/gitlab/repositories +``` + +### Parallel rsync only for repositories with recent activity + +Suppose you have already done one sync that started after 2015-10-1 12:00 UTC. +Then you might only want to sync repositories that were changed via GitLab +_after_ that time. You can use the 'SINCE' variable to tell 'rake +gitlab:list_repos' to only print repositories with recent activity. + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ + sudo -u git \ + /usr/bin/env JOBS=10 \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repoos \ + /var/opt/gitlab/git-data/repositories \ + /mnt/gitlab/repositories + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ + sudo -u git -H \ + /usr/bin/env JOBS=10 \ + bin/parallel-rsync-repos \ + /home/git/repositories \ + /mnt/gitlab/repositories +``` -- cgit v1.2.1 From 2f048df4a4a83ff009d2ef2d14ee04e5a2798618 Mon Sep 17 00:00:00 2001 From: Zeger-Jan van de Weg Date: Wed, 18 Nov 2015 11:17:41 +0100 Subject: API support, incorporated feedback --- doc/api/merge_requests.md | 60 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index ffa7f2cdf14..1b2ad1caf49 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -2,8 +2,8 @@ ## List merge requests -Get all merge requests for this project. -The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). +Get all merge requests for this project. +The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. ``` @@ -292,9 +292,59 @@ PUT /projects/:id/merge_request/:merge_request_id/merge Parameters: -- `id` (required) - The ID of a project -- `merge_request_id` (required) - ID of MR -- `merge_commit_message` (optional) - Custom merge commit message +- `id` (required) - The ID of a project +- `merge_request_id` (required) - ID of MR +- `merge_commit_message` (optional) - Custom merge commit message +- `should_remove_source_branch` (optional) - if `true` removes the source branch +- `merge_when_build_succeeds` (optional) - if `true` the MR is merge when the build succeeds + +```json +{ + "id": 1, + "target_branch": "master", + "source_branch": "test1", + "project_id": 3, + "title": "test1", + "state": "merged", + "upvotes": 0, + "downvotes": 0, + "author": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + }, + "assignee": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + } +} +``` + +## Cancel Merge When Build Succeeds + +Cancels the merge when build succeeds and reset the merge parameters + +If successfull you'll get `200 OK`. + +If you don't have permissions to accept this merge request - you'll get a 401 + +If the merge request is already merged or closed - you get 405 and error message 'Method Not Allowed' + +In case the merge request is not set to be merged when the build succeeds, you'll also get a 405 with the error message 'Method Not Allowed' +``` +PUT /projects/:id/merge_request/:merge_request_id/cancel_merge_when_build_succeeds +``` +Parameters: + +- `id` (required) - The ID of a project +- `merge_request_id` (required) - ID of MR ```json { -- cgit v1.2.1 From 3cc2b48a82aae8b535ddf11e9057a29f2242524c Mon Sep 17 00:00:00 2001 From: Marin Jankovski Date: Wed, 18 Nov 2015 11:39:26 +0100 Subject: Backup LFS objects same as any upload. --- doc/raketasks/backup_restore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 1a5442cdac7..4e645b21a85 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -29,7 +29,7 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ``` Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, -uploads (attachments), repositories, builds(CI build output logs), artifacts (CI build artifacts). +uploads (attachments), repositories, builds(CI build output logs), artifacts (CI build artifacts), lfs (LFS objects). Use a comma to specify several options at the same time. ``` -- cgit v1.2.1 From cf61b8e22c9c78165ce55c7c2f78ae0b3c7af7aa Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Thu, 19 Nov 2015 16:01:55 +0100 Subject: Update gitlab-workhorse docs to 0.4.2 --- doc/install/installation.md | 4 ++-- doc/update/8.1-to-8.2.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 7ef46b04065..71b0ef3ebb0 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -327,12 +327,12 @@ GitLab Shell is an SSH access and repository management software developed speci cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse - sudo -u git -H git checkout 0.4.1 + sudo -u git -H git checkout 0.4.2 sudo -u git -H make ### Initialize Database and Activate Advanced Features - # Go to Gitlab installation folder + # Go to GitLab installation folder cd /home/git/gitlab diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index 73d899f9c2e..81e13f714b7 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -81,7 +81,7 @@ from GitLab 8.1. cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse -sudo -u git -H git checkout 0.4.1 +sudo -u git -H git checkout 0.4.2 sudo -u git -H make ``` -- cgit v1.2.1 From 52bcfd037fdc17bd73a674c248045dc750cc55c6 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 19 Nov 2015 17:18:07 +0200 Subject: Update public access documentation [ci skip] --- doc/public_access/public_access.md | 51 ++++++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 18 deletions(-) (limited to 'doc') diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md index bd439f7c6f3..6e22ea7b72a 100644 --- a/doc/public_access/public_access.md +++ b/doc/public_access/public_access.md @@ -1,44 +1,59 @@ # Public access -GitLab allows you to open selected projects to be accessed **publicly** or **internally**. +GitLab allows you to change your projects' visibility in order be accessed +**publicly** or **internally**. -Projects with either of these visibility levels will be listed in the [public access directory](/public). +Projects with either of these visibility levels will be listed in the +public access directory (`/public` under your GitLab instance). +Here is the [GitLab.com example](https://gitlab.com/public). Internal projects will only be available to authenticated users. -## Public projects +## Visibility of projects + +### Public projects Public projects can be cloned **without any** authentication. -It will also be listed on the [public access directory](/public). +They will also be listed on the public access directory (`/public`). -**Any logged in user** will have [Guest](../permissions/permissions) permissions on the repository. +**Any logged in user** will have [Guest](../permissions/permissions) +permissions on the repository. -## Internal projects +### Internal projects Internal projects can be cloned by any logged in user. -It will also be listed on the [public access directory](/public) for logged in users. +They will also be listed on the public access directory (`/public`) for logged +in users. -Any logged in user will have [Guest](../permissions/permissions) permissions on the repository. +Any logged in user will have [Guest](../permissions/permissions) permissions on +the repository. -## How to change project visibility +### How to change project visibility -1. Go to your project dashboard -1. Click on the "Edit" tab -1. Change "Visibility Level" +1. Go to your project's **Settings** +1. Change "Visibility Level" to either Public, Internal or Private ## Visibility of users -The public page of users, located at `/u/username` is visible if either: +The public page of a user, located at `/u/username`, is always visible whether +you are logged in or not. + +When visiting the public page of a user, you can only see the projects which +you are privileged to. -- You are logged in. -- You are logged out, and the target user is authorized to (is Guest, Reporter, etc.) at least one public project. +## Visibility of groups -Otherwise, you will be redirected to the sign in page. +The public page of a group, located at `/groups/groupname`, is always visible +to everyone. -When visiting the public page of an user, you will only see listed projects which you can view yourself. +Logged out users will be able to see the description and the avatar of the +group as well as all public projects belonging to that group. ## Restricting the use of public or internal projects -In the Admin area under Settings you can disable public projects or public and internal projects for the entire GitLab installation to prevent people making code public by accident. The restricted visibility settings do not apply to admin users. +In the Admin area under **Settings** (`/admin/application_settings`), you can +restrict the use of visibility levels for users when they create a project or a +snippet. This is useful to prevent people exposing their repositories to public +by accident. The restricted visibility settings do not apply to admin users. -- cgit v1.2.1 From de440ee5e3fd03d0e3de33c474b1889e03aaf04a Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Thu, 19 Nov 2015 16:25:17 +0100 Subject: More GitLab spellings. No dedicated CI in omnibus [ci skip] --- doc/install/installation.md | 2 +- doc/release/monthly.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 52ae30af805..a710407d4f6 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -320,7 +320,7 @@ GitLab Shell is an SSH access and repository management software developed speci **Note:** If you want to use HTTPS, see [Using HTTPS](#using-https) for the additional steps. -**Note:** Make sure your hostname can be resolved on the machine itself by either a proper DNS record or an additional line in /etc/hosts ("127.0.0.1 hostname"). This might be necessary for example if you set up gitlab behind a reverse proxy. If the hostname cannot be resolved, the final installation check will fail with "Check GitLab API access: FAILED. code: 401" and pushing commits will be rejected with "[remote rejected] master -> master (hook declined)". +**Note:** Make sure your hostname can be resolved on the machine itself by either a proper DNS record or an additional line in /etc/hosts ("127.0.0.1 hostname"). This might be necessary for example if you set up GitLab behind a reverse proxy. If the hostname cannot be resolved, the final installation check will fail with "Check GitLab API access: FAILED. code: 401" and pushing commits will be rejected with "[remote rejected] master -> master (hook declined)". ### Install gitlab-workhorse diff --git a/doc/release/monthly.md b/doc/release/monthly.md index c9ab87671d2..aff3f066b24 100644 --- a/doc/release/monthly.md +++ b/doc/release/monthly.md @@ -159,7 +159,7 @@ Please do not raise issues directly in this issue but link to issues that might The decision to create a patch release or not is with the release manager who is assigned to this issue. The release manager will comment here about the plans for patch releases. -Assign the issue to the release manager and at mention all members of gitlab core team. If there are any known bugs in the release add them immediately. +Assign the issue to the release manager and at mention all members of GitLab core team. If there are any known bugs in the release add them immediately. ## Tweet about RC1 -- cgit v1.2.1 From 91a76957e3d18e3cb89bc8320f8513e1002e551e Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Thu, 19 Nov 2015 17:05:30 +0100 Subject: Update LFS docs for cloning [ci skip] --- doc/workflow/git_lfs.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/workflow/git_lfs.md b/doc/workflow/git_lfs.md index e1064051fe8..616a71522ae 100644 --- a/doc/workflow/git_lfs.md +++ b/doc/workflow/git_lfs.md @@ -55,7 +55,7 @@ In `config/gitlab.yml`: * When SSH is set as a remote, Git LFS objects still go through HTTPS * Any Git LFS request will ask for HTTPS credentials to be provided so good Git credentials store is recommended * Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported -* Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the url to Git config manually (see #troubleshooting-tips) +* Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting-tips) ## Using Git LFS @@ -77,11 +77,10 @@ git commit -am "Added Debian iso" # commit the file meta data git push origin master # sync the git repo and large file to the GitLab server ``` -Downloading a single large file is also very simple: +Cloning the repository works the same as before. Git automatically detects the LFS-tracked files and clones them via HTTP. If you performed the git clone command with a SSH URL, you have to enter your GitLab credentials for HTTP authentication. ```bash git clone git@gitlab.example.com:group/project.git -git lfs fetch debian.iso # download the large file ``` -- cgit v1.2.1 From 899807f604f7923cc40a64cdd0f61c4248b8870d Mon Sep 17 00:00:00 2001 From: Marin Jankovski Date: Fri, 20 Nov 2015 16:10:08 +0100 Subject: Update required version of lfs client and separate the docs for users and admins. --- doc/README.md | 1 + doc/workflow/README.md | 1 + doc/workflow/git_lfs.md | 135 --------------------- doc/workflow/lfs/lfs_administration.md | 41 +++++++ .../lfs/manage_large_binaries_with_git_lfs.md | 125 +++++++++++++++++++ 5 files changed, 168 insertions(+), 135 deletions(-) delete mode 100644 doc/workflow/git_lfs.md create mode 100644 doc/workflow/lfs/lfs_administration.md create mode 100644 doc/workflow/lfs/manage_large_binaries_with_git_lfs.md (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index 0f6866475f7..58ab5dd08e0 100644 --- a/doc/README.md +++ b/doc/README.md @@ -50,6 +50,7 @@ - [Welcome message](customization/welcome_message.md) Add a custom welcome message to the sign-in page. - [Reply by email](incoming_email/README.md) Allow users to comment on issues and merge requests by replying to notification emails. - [Migrate GitLab CI to CE/EE](migrate_ci_to_ce/README.md) Follow this guide to migrate your existing GitLab CI data to GitLab CE/EE. +- [Git LFS configuration](workflow/lfs/lfs_administration.md) ## Contributor documentation diff --git a/doc/workflow/README.md b/doc/workflow/README.md index c1a4f64981f..a6b4d951188 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -17,3 +17,4 @@ - [Milestones](milestones.md) - [Merge Requests](merge_requests.md) - ["Work In Progress" Merge Requests](wip_merge_requests.md) +- [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) diff --git a/doc/workflow/git_lfs.md b/doc/workflow/git_lfs.md deleted file mode 100644 index 616a71522ae..00000000000 --- a/doc/workflow/git_lfs.md +++ /dev/null @@ -1,135 +0,0 @@ -# Git LFS - -Managing large files such as audio, video and graphics files has always been one of the shortcomings of Git. -The general recommendation is to not have Git repositories larger than 1GB to preserve performance. - -GitLab already supports [managing large files with git annex](http://doc.gitlab.com/ee/workflow/git_annex.html) (EE only), however in certain -environments it is not always convenient to use different commands to differentiate between the large files and regular ones. - -Git LFS makes this simpler for the end user by removing the requirement to learn new commands. - - -## How it works - -Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication to authorize client requests. -Once the request is authorized, Git LFS client receives instructions from where to fetch or where to push the large file. - -## Requirements - -* Git LFS is supported in GitLab starting with version 8.2 -* Git LFS [client](https://git-lfs.github.com) version 0.6.0 and up - -## GitLab and Git LFS - -### Configuration - -Git LFS objects can be large in size. By default, they are stored on the server GitLab is installed on. - -There are two configuration options to help GitLab server administrators: - -* Enabling/disabling Git LFS support -* Changing the location of LFS object storage - -#### Omnibus packages - -In `/etc/gitlab/gitlab.rb`: - -```ruby -gitlab_rails['lfs_enabled'] = false -gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects" -``` - -#### Installations from source - -In `config/gitlab.yml`: - -```yaml - lfs: - enabled: false - storage_path: /mnt/storage/lfs-objects -``` - -## Known limitations - -* Git LFS v1 original API is not supported since it was deprecated early in LFS development, starting with Git LFS version 0.6.0 -* When SSH is set as a remote, Git LFS objects still go through HTTPS -* Any Git LFS request will ask for HTTPS credentials to be provided so good Git credentials store is recommended -* Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported -* Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting-tips) - -## Using Git LFS - -Lets take a look at the workflow when you need to check large files into your Git repository with Git LFS: -For example, if you want to upload a very large file and check it into your Git repository: - -```bash -git clone git@gitlab.example.com:group/project.git -git lfs init # initialize the Git LFS project project -git lfs track "*.iso" # select the file extensions that you want to treat as large files -``` - -Once a certain file extension is marked for tracking as a LFS object you can use Git as usual without having to redo the command to track a file with the same extension: - -```bash -cp ~/tmp/debian.iso ./ # copy a large file into the current directory -git add . # add the large file to the project -git commit -am "Added Debian iso" # commit the file meta data -git push origin master # sync the git repo and large file to the GitLab server -``` - -Cloning the repository works the same as before. Git automatically detects the LFS-tracked files and clones them via HTTP. If you performed the git clone command with a SSH URL, you have to enter your GitLab credentials for HTTP authentication. - -```bash -git clone git@gitlab.example.com:group/project.git -``` - - -## Troubleshooting - -### error: Repository or object not found - -There are a couple of reasons why this error can occur: - -* Wrong version of LFS client used: - -Check the version of Git LFS on the client machine with `git lfs version`. Only version 0.6.0 and newer are supported. - -* Project is using deprecated LFS API - -Check the Git config of the project for traces of deprecated API with `git lfs -l`. If `batch = false` is set in the config, remove the line and try using Git LFS client newer than 0.6.0. - -### Invalid status for : 501 - -When attempting to push a LFS object to a GitLab server that doesn't have Git LFS support enabled, server will return status `error 501`. Check with your GitLab administrator why Git LFS is not enabled on the server. See [Configuration section](#configuration) for instructions on how to enable LFS support. - -### getsockopt: connection refused - -If you push a LFS object to a project and you receive an error similar to: `Post /info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`, -the LFS client is trying to reach GitLab through HTTPS. However, your GitLab instance is being served on HTTP. - -This behaviour is caused by Git LFS using HTTPS connections by default when a `lfsurl` is not set in the Git config. - -To prevent this from happening, set the lfs url in project Git config: - -```bash - -git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs/objects/batch" -``` - -### Credentials are always required when pushing an object - -Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing the LFS object on every push for every object, user HTTPS credentials are required. - -By default, Git has support for remembering the credentials for each repository you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials). - -For example, you can tell Git to remember the password for a period of time in which you expect to push the objects: - -```bash -git config --global credential.helper 'cache --timeout=3600' -``` - -This will remember the credentials for an hour after which Git operations will require re-authentication. - -If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. For Windows, `wincred` is available. - -More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage) \ No newline at end of file diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md new file mode 100644 index 00000000000..9fa307a9d5b --- /dev/null +++ b/doc/workflow/lfs/lfs_administration.md @@ -0,0 +1,41 @@ +## GitLab Git LFS Administration + +Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md). + +## Requirements + +* Git LFS is supported in GitLab starting with version 8.2. +* Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up + +## Configuration + +Git LFS objects can be large in size. By default, they are stored on the server GitLab is installed on. + +There are two configuration options to help GitLab server administrators: + +* Enabling/disabling Git LFS support +* Changing the location of LFS object storage + +### Omnibus packages + +In `/etc/gitlab/gitlab.rb`: + +```ruby +gitlab_rails['lfs_enabled'] = false +gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects" +``` + +### Installations from source + +In `config/gitlab.yml`: + +```yaml + lfs: + enabled: false + storage_path: /mnt/storage/lfs-objects +``` + +## Known limitations + +* Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported +* Currently, removing LFS objects from GitLab Git LFS storage is not supported diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md new file mode 100644 index 00000000000..a93fd3dfb13 --- /dev/null +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -0,0 +1,125 @@ +# Git LFS + +Managing large files such as audio, video and graphics files has always been one of the shortcomings of Git. +The general recommendation is to not have Git repositories larger than 1GB to preserve performance. + +GitLab already supports [managing large files with git annex](http://doc.gitlab.com/ee/workflow/git_annex.html) (EE only), however in certain +environments it is not always convenient to use different commands to differentiate between the large files and regular ones. + +Git LFS makes this simpler for the end user by removing the requirement to learn new commands. + +## How it works + +Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication to authorize client requests. +Once the request is authorized, Git LFS client receives instructions from where to fetch or where to push the large file. + +## GitLab server configuration + +Documentation for GitLab instance administrators is under [LFS administration doc](lfs_administration.md). + +## Requirements + +* Git LFS is supported in GitLab starting with version 8.2 +* [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up + +## Known limitations + +* Git LFS v1 original API is not supported since it was deprecated early in LFS development +* When SSH is set as a remote, Git LFS objects still go through HTTPS +* Any Git LFS request will ask for HTTPS credentials to be provided so good Git credentials store is recommended +* Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting) + +## Using Git LFS + +Lets take a look at the workflow when you need to check large files into your Git repository with Git LFS: +For example, if you want to upload a very large file and check it into your Git repository: + +```bash +git clone git@gitlab.example.com:group/project.git +git lfs init # initialize the Git LFS project project +git lfs track "*.iso" # select the file extensions that you want to treat as large files +``` + +Once a certain file extension is marked for tracking as a LFS object you can use Git as usual without having to redo the command to track a file with the same extension: + +```bash +cp ~/tmp/debian.iso ./ # copy a large file into the current directory +git add . # add the large file to the project +git commit -am "Added Debian iso" # commit the file meta data +git push origin master # sync the git repo and large file to the GitLab server +``` + +Cloning the repository works the same as before. Git automatically detects the LFS-tracked files and clones them via HTTP. If you performed the git clone command with a SSH URL, you have to enter your GitLab credentials for HTTP authentication. + +```bash +git clone git@gitlab.example.com:group/project.git +``` + +If you already cloned the repository and you want to get the latest LFS object that are on the remote repository, eg. from branch `master`: + +```bash +git lfs fetch master +``` + +## Troubleshooting + +### error: Repository or object not found + +There are a couple of reasons why this error can occur: + +* You don't have permissions to access certain LFS object + +Check if you have permissions to push to the project or fetch from the project. + +* Project is not allowed to access the LFS object + +Check if the LFS object you are trying to push to the project or fetch from the project is available to the project. + +* Project is using deprecated LFS API + +### Invalid status for : 501 + +Git LFS will log the failures into a log file. +To view this log file, while in project directory: + +```bash +git lfs logs last +``` + +If the status `error 501` is shown, it is because: + +* Git LFS support is not enabled on the GitLab server. Check with your GitLab administrator why Git LFS is not enabled on the server. See [LFS administration documentation](lfs_administration.md) for instructions on how to enable LFS support. + +* Git LFS client version is not supported by GitLab server. Check your Git LFS version with `git lfs version`. Check the Git config of the project for traces of deprecated API with `git lfs -l`. If `batch = false` is set in the config, remove the line and try to update your Git LFS client. Only version 1.0.1 and newer are supported. + +### getsockopt: connection refused + +If you push a LFS object to a project and you receive an error similar to: `Post /info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`, +the LFS client is trying to reach GitLab through HTTPS. However, your GitLab instance is being served on HTTP. + +This behaviour is caused by Git LFS using HTTPS connections by default when a `lfsurl` is not set in the Git config. + +To prevent this from happening, set the lfs url in project Git config: + +```bash + +git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs/objects/batch" +``` + +### Credentials are always required when pushing an object + +Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing the LFS object on every push for every object, user HTTPS credentials are required. + +By default, Git has support for remembering the credentials for each repository you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials). + +For example, you can tell Git to remember the password for a period of time in which you expect to push the objects: + +```bash +git config --global credential.helper 'cache --timeout=3600' +``` + +This will remember the credentials for an hour after which Git operations will require re-authentication. + +If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. For Windows, `wincred` is available. + +More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). -- cgit v1.2.1 From ca735446efe46e9309fa59673ecea97fb11ddcd5 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 12:13:30 +0100 Subject: Add tags page to readme [ci skip] --- doc/api/README.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/api/README.md b/doc/api/README.md index 6b8528de50c..25a31b235cc 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -10,6 +10,7 @@ - [Repositories](repositories.md) - [Repository Files](repository_files.md) - [Commits](commits.md) +- [Tags](tags.md) - [Branches](branches.md) - [Merge Requests](merge_requests.md) - [Issues](issues.md) -- cgit v1.2.1 From c113e23489fe4ee0a0d0be837090597595800553 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 16:50:37 +0100 Subject: Use award emoticons in GitLab workflow [ci skip] --- doc/workflow/gitlab_flow.md | 9 ++++----- doc/workflow/voting_slider.png | Bin 5329 -> 0 bytes 2 files changed, 4 insertions(+), 5 deletions(-) delete mode 100644 doc/workflow/voting_slider.png (limited to 'doc') diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 31495bce76e..8965e5b3654 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -244,13 +244,12 @@ Developing software happen in small messy steps and it is OK to have your histor You can use tools to view the network graphs of commits and understand the messy history that created your code. If you rebase code the history is incorrect, and there is no way for tools to remedy this because they can't deal with changing commit identifiers. -## Voting on merge requests +## Award emojis on issues and merge requests -![Voting slider in GitLab](voting_slider.png) +![Emoji bar in GitLab](award_emoji.png) -It is common to voice approval or disapproval by using +1 or -1 emoticons. -In GitLab the +1 and -1 are aggregated and shown at the top of the merge request. -As a rule of thumb anything that doesn't have two times more +1's than -1's is suspect and should not be merged yet. +It is common to voice approval or disapproval by using +1 or -1. In GitLab you +can use emojis to give a virtual high five on issues and merge requests. ## Pushing and removing branches diff --git a/doc/workflow/voting_slider.png b/doc/workflow/voting_slider.png deleted file mode 100644 index 4c660ef9593..00000000000 Binary files a/doc/workflow/voting_slider.png and /dev/null differ -- cgit v1.2.1 From 2cba93a0d2d12ee36bf98569e5c6c14ac7ea40e0 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 17:29:26 +0100 Subject: Make tag API consistent for release feature --- doc/api/tags.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/api/tags.md b/doc/api/tags.md index b5b90cf6b82..cf95f87dc3e 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -29,7 +29,7 @@ Parameters: ] }, "release": { - "tag": "1.0.0", + "tag_name": "1.0.0", "description": "Amazing release. Wow" }, "name": "v1.0.0", @@ -70,7 +70,7 @@ Parameters: ] }, "release": { - "tag": "1.0.0", + "tag_name": "1.0.0", "description": "Amazing release. Wow" }, "name": "v1.0.0", @@ -89,18 +89,18 @@ It returns 200 if the operation succeed. In case of an error, Add release notes to the existing git tag ``` -PUT /projects/:id/repository/:tag/release +PUT /projects/:id/repository/tags/:tag_name/release ``` Parameters: - `id` (required) - The ID of a project -- `tag` (required) - The name of a tag +- `tag_name` (required) - The name of a tag - `description` (required) - Release notes with markdown support ```json { - "tag": "1.0.0", + "tag_name": "1.0.0", "description": "Amazing release. Wow" } ``` -- cgit v1.2.1 From faef95af1a07bdcfd02eead36d144f332b428f1f Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 18:08:45 +0100 Subject: API: Return 404 if the tag for a release does not exist --- doc/api/tags.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/tags.md b/doc/api/tags.md index cf95f87dc3e..ca9e2cef651 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -86,7 +86,8 @@ It returns 200 if the operation succeed. In case of an error, ## New release -Add release notes to the existing git tag +Add release notes to the existing git tag. It returns 200 if the release is +created successfully. If the tag does not exist, 404 is returned. ``` PUT /projects/:id/repository/tags/:tag_name/release -- cgit v1.2.1 From 6f7e90f6dba300591281aba08ffbe30ce3cc5c90 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 18:51:41 +0100 Subject: Use POST to create a new release instead of PUT --- doc/api/tags.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/api/tags.md b/doc/api/tags.md index ca9e2cef651..ab135117250 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -84,13 +84,13 @@ It returns 200 if the operation succeed. In case of an error, 405 with an explaining error message is returned. -## New release +## Create a new release -Add release notes to the existing git tag. It returns 200 if the release is +Add release notes to the existing git tag. It returns 201 if the release is created successfully. If the tag does not exist, 404 is returned. ``` -PUT /projects/:id/repository/tags/:tag_name/release +POST /projects/:id/repository/tags/:tag_name/release ``` Parameters: -- cgit v1.2.1 From c502695fdb9b01b4ff8721845e1d39f3d3369008 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sat, 21 Nov 2015 13:15:51 -0500 Subject: Add award_emoji.png doc image [ci skip] --- doc/workflow/award_emoji.png | Bin 0 -> 6620 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 doc/workflow/award_emoji.png (limited to 'doc') diff --git a/doc/workflow/award_emoji.png b/doc/workflow/award_emoji.png new file mode 100644 index 00000000000..fb26ee04393 Binary files /dev/null and b/doc/workflow/award_emoji.png differ -- cgit v1.2.1 From f2318dfc9e80555bae99f543d4e07ec089ab3935 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Sat, 21 Nov 2015 20:25:07 +0100 Subject: Fix CI yaml syntax documentation [ci skip] --- doc/ci/yaml/README.md | 90 +++++++++++++++++++++++---------------------------- 1 file changed, 41 insertions(+), 49 deletions(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3e6071a2ee7..3dbf1afc7a9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -278,26 +278,23 @@ The above script will: `artifacts` is used to specify list of files and directories which should be attached to build after success. 1. Send all files in `binaries` and `.config`: -``` -artifacts: - paths: - - binaries/ - - .config -``` + + artifacts: + paths: + - binaries/ + - .config 2. Send all git untracked files: -``` -artifacts: - untracked: true -``` + + artifacts: + untracked: true 3. Send all git untracked files and files in `binaries`: -``` -artifacts: - untracked: true - paths: - - binaries/ -``` + + artifacts: + untracked: true + paths: + - binaries/ The artifacts will be send after the build success to GitLab and will be accessible in GitLab interface to download. @@ -307,46 +304,41 @@ This feature requires GitLab Runner v0.7.0 or higher. `cache` is used to specify list of files and directories which should be cached between builds. 1. Cache all files in `binaries` and `.config`: -``` -rspec: - script: test - cache: - paths: - - binaries/ - - .config -``` + + rspec: + script: test + cache: + paths: + - binaries/ + - .config 2. Cache all git untracked files: -``` -rspec: - script: test - cache: - untracked: true -``` + rspec: + script: test + cache: + untracked: true + 3. Cache all git untracked files and files in `binaries`: -``` -rspec: - script: test - cache: - untracked: true - paths: - - binaries/ -``` -4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: + rspec: + script: test + cache: + untracked: true + paths: + - binaries/ -``` -cache: - paths: - - my/files +4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: -rspec: - script: test - cache: - paths: - - binaries/ -``` + cache: + paths: + - my/files + + rspec: + script: test + cache: + paths: + - binaries/ The cache is provided on best effort basis, so don't expect that cache will be present. For implementation details please check GitLab Runner. -- cgit v1.2.1 From 26b12e2c374c8f07abda06a8b19bd116448325f4 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 21:36:31 +0100 Subject: Add upvote/downvote fields to merge request and note API to preserve compatibility --- doc/api/merge_requests.md | 32 +++++++++++++++++++++++++------- doc/api/notes.md | 11 ++++++++--- 2 files changed, 33 insertions(+), 10 deletions(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 2f17d4ae06b..0cef09d5b27 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -3,8 +3,9 @@ ## List merge requests Get all merge requests for this project. -The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). -The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. +The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). +The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. With GitLab 8.2 the return fields `upvotes` and +`downvotes` are deprecated and always return `0`. ``` GET /projects/:id/merge_requests @@ -31,6 +32,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "opened", + "upvotes": 0, + "downvotes": 0, "author": { "id": 1, "username": "admin", @@ -55,7 +58,7 @@ Parameters: ## Get single MR -Shows information about a single merge request. +Shows information about a single merge request. With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and always return `0`. ``` GET /projects/:id/merge_request/:merge_request_id @@ -75,6 +78,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "merged", + "upvotes": 0, + "downvotes": 0, "author": { "id": 1, "username": "admin", @@ -98,7 +103,9 @@ Parameters: ## Get single MR changes -Shows information about the merge request including its files and changes +Shows information about the merge request including its files and changes. +With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and +always return `0`. ``` GET /projects/:id/merge_request/:merge_request_id/changes @@ -122,6 +129,8 @@ Parameters: "updated_at": "2015-02-02T20:08:49.959Z", "target_branch": "secret_token", "source_branch": "version-1-9", + "upvotes": 0, + "downvotes": 0, "author": { "name": "Chad Hamill", "username": "jarrett", @@ -167,7 +176,8 @@ Parameters: ## Create MR -Creates a new merge request. +Creates a new merge request. With GitLab 8.2 the return fields `upvotes` and ` +downvotes` are deprecated and always return `0`. ``` POST /projects/:id/merge_requests @@ -192,6 +202,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "opened", + "upvotes": 0, + "downvotes": 0, "author": { "id": 1, "username": "admin", @@ -217,7 +229,8 @@ If an error occurs, an error number and a message explaining the reason is retur ## Update MR -Updates an existing merge request. You can change the target branch, title, or even close the MR. +Updates an existing merge request. You can change the target branch, title, or even close the MR. With GitLab 8.2 the return fields `upvotes` and `downvotes` +are deprecated and always return `0`. ``` PUT /projects/:id/merge_request/:merge_request_id @@ -242,6 +255,8 @@ Parameters: "title": "test1", "description": "description1", "state": "opened", + "upvotes": 0, + "downvotes": 0, "author": { "id": 1, "username": "admin", @@ -266,7 +281,8 @@ If an error occurs, an error number and a message explaining the reason is retur ## Accept MR -Merge changes submitted with MR using this API. +Merge changes submitted with MR using this API. With GitLab 8.2 the return +fields `upvotes` and `downvotes` are deprecated and always return `0`. If merge success you get `200 OK`. @@ -294,6 +310,8 @@ Parameters: "project_id": 3, "title": "test1", "state": "merged", + "upvotes": 0, + "downvotes": 0, "author": { "id": 1, "username": "admin", diff --git a/doc/api/notes.md b/doc/api/notes.md index bcba1f62151..e7f299c0994 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -6,7 +6,8 @@ Notes are comments on snippets, issues or merge requests. ### List project issue notes -Gets a list of all notes for a single issue. +Gets a list of all notes for a single issue. With GitLab 8.2 the return fields +`upvote` and `downvote` are deprecated and always return `false`. ``` GET /projects/:id/issues/:issue_id/notes @@ -32,7 +33,9 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T09:22:45Z", - "system": true + "system": true, + "upvote": false, + "downvote": false }, { "id": 305, @@ -47,7 +50,9 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T09:56:03Z", - "system": false + "system": true, + "upvote": false, + "downvote": false } ] ``` -- cgit v1.2.1 From 3ea05c5b5b253de33d8bf8d615c66e2935b940ef Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 22:24:34 +0100 Subject: Only allow to create a release if it does not exist yet --- doc/api/tags.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/tags.md b/doc/api/tags.md index ab135117250..bc61aa1118f 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -87,7 +87,8 @@ It returns 200 if the operation succeed. In case of an error, ## Create a new release Add release notes to the existing git tag. It returns 201 if the release is -created successfully. If the tag does not exist, 404 is returned. +created successfully. If the tag does not exist, 404 is returned. If there +already exists a release for the given tag, 409 is returned. ``` POST /projects/:id/repository/tags/:tag_name/release -- cgit v1.2.1 From 04a3d27eaba0312d99e8d88a3a9ee4b5c83ecce1 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 21 Nov 2015 22:34:53 +0100 Subject: Allow editing a release in API via PUT method --- doc/api/tags.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) (limited to 'doc') diff --git a/doc/api/tags.md b/doc/api/tags.md index bc61aa1118f..085d387e824 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -106,3 +106,26 @@ Parameters: "description": "Amazing release. Wow" } ``` + +## Update a release + +Updates the release notes of a given release. It returns 200 if the release is +successfully updated. If the tag or the release does not exist, it returns 404 +with a proper error message. + +``` +PUT /projects/:id/repository/tags/:tag_name/release +``` + +Parameters: + +- `id` (required) - The ID of a project +- `tag_name` (required) - The name of a tag +- `description` (required) - Release notes with markdown support + +```json +{ + "tag_name": "1.0.0", + "description": "Amazing release. Wow" +} +``` \ No newline at end of file -- cgit v1.2.1 From 732c2d15d235ca112db4b1d15c888da52660a1d5 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Sun, 22 Nov 2015 18:24:53 -0500 Subject: Update doc/release/patch.md with more current patch procedures [ci skip] --- doc/release/patch.md | 40 ++++++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) (limited to 'doc') diff --git a/doc/release/patch.md b/doc/release/patch.md index 6aa11b283df..3022e375aca 100644 --- a/doc/release/patch.md +++ b/doc/release/patch.md @@ -1,21 +1,46 @@ # Things to do when doing a patch release -NOTE: This is a guide for GitLab developers. If you are trying to install GitLab see the latest stable [installation guide](install/installation.md) and if you are trying to upgrade, see the [upgrade guides](update). +NOTE: This is a guide for GitLab developers. If you are trying to install GitLab +see the latest stable [installation guide](install/installation.md) and if you +are trying to upgrade, see the [upgrade guides](update). ## When to do a patch release -Do a patch release when there is a critical regression that needs to be addresses before the next monthly release. - -Otherwise include it in the monthly release and note there was a regression fix in the release announcement. +Patch releases are done as-needed in order to fix regressions in the current +major release that cannot or should not wait until the next major release. +What's included and when to release is at the discretion of the release manager. ## Release Procedure +### Create a patch issue + +Create an issue in the GitLab CE project. Name it "Release x.y.z", tag it with +the `release` label, and assign it to the milestone of the corresponding major +release. + +Use the following template: + +``` +- Picked into respective `stable` branches: +- [ ] Merge `x-y-stable` into `x-y-stable-ee` +- [ ] release-tools: `x.y.z` +- gitlab-omnibus + - [ ] `x.y.z+ee.0` + - [ ] `x.y.z+ce.0` +- [ ] Deploy +- [ ] Add patch notice to [x.y regressions]() +- [ ] [Blog post]() +- [ ] [Tweet]() +- [ ] Add entry to version.gitlab.com +``` + +Update the issue with links to merge requests that need to be/have been picked +into the `stable` branches. + ### Preparation 1. Verify that the issue can be reproduced 1. Note in the 'GitLab X.X regressions' that you will create a patch -1. Create an issue on private GitLab development server -1. Name the issue "Release X.X.X CE and X.X.X EE", this will make searching easier 1. Fix the issue on a feature branch, do this on the private GitLab development server 1. If it is a security issue, then assign it to the release manager and apply a 'security' label 1. Consider creating and testing workarounds @@ -25,7 +50,6 @@ Otherwise include it in the monthly release and note there was a regression fix 1. For EE, update the CHANGELOG-EE if it is EE specific fix. Otherwise, merge the stable CE branch and add to CHANGELOG-EE "Merge community edition changes for version X.X.X" 1. Merge CE stable branch into EE stable branch - ### Bump version Get release tools @@ -54,4 +78,4 @@ bundle exec rake release["x.x.x"] 1. Note in the 'GitLab X.X regressions' issue that the patch was published (CE only) 1. Create the 'x.y.0' version on version.gitlab.com 1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) -1. Create a new patch release issue for the next potential release \ No newline at end of file +1. Create a new patch release issue for the next potential release -- cgit v1.2.1 From 8608c6325e19f529f7b43ff881c562d3a0114e1c Mon Sep 17 00:00:00 2001 From: Zeger-Jan van de Weg Date: Mon, 23 Nov 2015 09:42:20 +0100 Subject: Refactor MergeWhenBuildSucceedsService and incorporate feedback --- doc/api/merge_requests.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 1b2ad1caf49..19e6fbb7887 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -296,7 +296,7 @@ Parameters: - `merge_request_id` (required) - ID of MR - `merge_commit_message` (optional) - Custom merge commit message - `should_remove_source_branch` (optional) - if `true` removes the source branch -- `merge_when_build_succeeds` (optional) - if `true` the MR is merge when the build succeeds +- `merged_when_build_succeeds` (optional) - if `true` the MR is merge when the build succeeds ```json { @@ -329,15 +329,13 @@ Parameters: ## Cancel Merge When Build Succeeds -Cancels the merge when build succeeds and reset the merge parameters - -If successfull you'll get `200 OK`. +If successful you'll get `200 OK`. If you don't have permissions to accept this merge request - you'll get a 401 If the merge request is already merged or closed - you get 405 and error message 'Method Not Allowed' -In case the merge request is not set to be merged when the build succeeds, you'll also get a 405 with the error message 'Method Not Allowed' +In case the merge request is not set to be merged when the build succeeds, you'll also get a 406 error. ``` PUT /projects/:id/merge_request/:merge_request_id/cancel_merge_when_build_succeeds ``` -- cgit v1.2.1 From 7ac112d8e814a8e3bb08e24f5af0d8828e38d4e9 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Mon, 23 Nov 2015 15:18:59 +0100 Subject: Add few fixes to documentation based on comments from review Signed-off-by: Dmitriy Zaporozhets --- doc/workflow/lfs/lfs_administration.md | 4 ++-- doc/workflow/lfs/manage_large_binaries_with_git_lfs.md | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md index 9fa307a9d5b..5076b2697a3 100644 --- a/doc/workflow/lfs/lfs_administration.md +++ b/doc/workflow/lfs/lfs_administration.md @@ -1,11 +1,11 @@ -## GitLab Git LFS Administration +# GitLab Git LFS Administration Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md). ## Requirements * Git LFS is supported in GitLab starting with version 8.2. -* Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up +* Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up. ## Configuration diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index a93fd3dfb13..210a8f71c3b 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -73,9 +73,10 @@ Check if you have permissions to push to the project or fetch from the project. * Project is not allowed to access the LFS object -Check if the LFS object you are trying to push to the project or fetch from the project is available to the project. +LFS object you are trying to push to the project or fetch from the project is not available to the project anymore. +Probably the object was removed from the server. -* Project is using deprecated LFS API +* Local git repository is using deprecated LFS API ### Invalid status for : 501 -- cgit v1.2.1 From d6e0a1883b9b0eb0faabc77f6c439d981e64a84a Mon Sep 17 00:00:00 2001 From: Anton Davydov Date: Tue, 24 Nov 2015 02:19:17 +0300 Subject: Fix typos in all docs [skip ci] --- doc/api/settings.md | 2 +- doc/development/db_dump.md | 2 +- doc/raketasks/backup_restore.md | 4 ++-- doc/release/monthly.md | 2 +- doc/web_hooks/web_hooks.md | 4 ++-- 5 files changed, 7 insertions(+), 7 deletions(-) (limited to 'doc') diff --git a/doc/api/settings.md b/doc/api/settings.md index d1b93a09c02..96867c67915 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -57,7 +57,7 @@ Parameters: - `default_project_visibility` - what visibility level new project receives - `default_snippet_visibility` - what visibility level new snippet receives - `restricted_signup_domains` - force people to use only corporate emails for signup -- `user_oauth_applications` - allow users to create oauth applicaitons +- `user_oauth_applications` - allow users to create oauth applications - `after_sign_out_path` - where redirect user after logout All parameters are optional. You can send only one that you want to change. diff --git a/doc/development/db_dump.md b/doc/development/db_dump.md index 21f1b3edecd..e4ff72aa349 100644 --- a/doc/development/db_dump.md +++ b/doc/development/db_dump.md @@ -1,4 +1,4 @@ -# Importing a database dump into a staging enviroment +# Importing a database dump into a staging environment Sometimes it is useful to import the database from a production environment into a staging environment for testing. The procedure below assumes you have diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 4e645b21a85..014f6a12dfc 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -360,8 +360,8 @@ If you are using backup restore procedures you might encounter the following war ``` psql:/var/opt/gitlab/backups/db/database.sql:22: ERROR: must be owner of extension plpgsql -psql:/var/opt/gitlab/backups/db/database.sql:2931: WARNING: no privileges could be revoked for "public" (two occurences) -psql:/var/opt/gitlab/backups/db/database.sql:2933: WARNING: no privileges were granted for "public" (two occurences) +psql:/var/opt/gitlab/backups/db/database.sql:2931: WARNING: no privileges could be revoked for "public" (two occurrences) +psql:/var/opt/gitlab/backups/db/database.sql:2933: WARNING: no privileges were granted for "public" (two occurrences) ``` diff --git a/doc/release/monthly.md b/doc/release/monthly.md index aff3f066b24..907c19e65a0 100644 --- a/doc/release/monthly.md +++ b/doc/release/monthly.md @@ -176,7 +176,7 @@ Tweet about the RC release: 1. Also check the CI changelog 1. Add a proposed tweet text to the blog post WIP MR description. 1. Create a WIP MR for the blog post -1. Make sure merge request title starts with `WIP` so it can not be accidently merged until ready. +1. Make sure merge request title starts with `WIP` so it can not be accidentally merged until ready. 1. Ask Dmitriy (or a team member with OS X) to add screenshots to the WIP MR. 1. Decide with core team who will be the MVP user. 1. Create WIP MR for adding MVP to MVP page on website diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index 7d838187a26..d24aa0293cf 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -184,7 +184,7 @@ X-Gitlab-Event: Note Hook { "object_kind": "note", "user": { - "name": "Adminstrator", + "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, @@ -337,7 +337,7 @@ X-Gitlab-Event: Note Hook { "object_kind": "note", "user": { - "name": "Adminstrator", + "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, -- cgit v1.2.1 From b9828278127724ff5eac60eb71e36f675b9985cd Mon Sep 17 00:00:00 2001 From: Eirik Lygre Date: Tue, 24 Nov 2015 20:55:18 +0000 Subject: Add reference to Microsoft's Git Credential Manager for Windows. --- doc/workflow/lfs/manage_large_binaries_with_git_lfs.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 210a8f71c3b..b59e92cb317 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -121,6 +121,6 @@ git config --global credential.helper 'cache --timeout=3600' This will remember the credentials for an hour after which Git operations will require re-authentication. -If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. For Windows, `wincred` is available. +If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases). -More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). +More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). \ No newline at end of file -- cgit v1.2.1 From 82ff9e65939d2da279b684cf378ca33e60eed66b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurens=20St=C3=B6tzel?= Date: Wed, 25 Nov 2015 12:27:07 +0000 Subject: Correction of markdown in SSH docs --- doc/ssh/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 9753504ac8b..fe5b45dd432 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -15,7 +15,8 @@ Note: It is a best practice to use a password for an SSH key, but it is not required and you can skip creating a password by pressing enter. Note that the password you choose here can't be altered or retrieved. -To generate a new SSH key, use the following commandGitLab```bash +To generate a new SSH key, use the following command: +```bash ssh-keygen -t rsa -C "$your_email" ``` This command will prompt you for a location and filename to store the key @@ -108,4 +109,4 @@ Note in the gitlab.com example above a username was specified to override the de Due to the wide variety of SSH clients and their very large number of configuration options, further explanation of these topics is beyond the scope of this document. Public SSH keys need to be unique, as they will bind to your account. Your SSH key is the only identifier you'll -have when pushing code via SSH. That's why it needs to uniquely map to a single user. +have when pushing code via SSH. That's why it needs to uniquely map to a single user. \ No newline at end of file -- cgit v1.2.1 From 85ad2140fa4ef2209b9d217403e183e33ce4dd37 Mon Sep 17 00:00:00 2001 From: Robert Schilling Date: Sat, 28 Nov 2015 20:53:17 +0100 Subject: Remove reverence to satellites [ci skip] --- doc/raketasks/backup_restore.md | 3 --- 1 file changed, 3 deletions(-) (limited to 'doc') diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 4e645b21a85..b4d2786bd76 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -274,9 +274,6 @@ sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186 # Start GitLab sudo gitlab-ctl start -# Create satellites -sudo gitlab-rake gitlab:satellites:create - # Check GitLab sudo gitlab-rake gitlab:check SANITIZE=true ``` -- cgit v1.2.1 From 7ba00b50827d8d30ea4c54acb2623d6fb387dcf1 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Mon, 30 Nov 2015 19:04:44 +0100 Subject: Use gitlab-shell 2.6.8 in update guide --- doc/update/8.1-to-8.2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index b57e999cfb7..7b228d6a22f 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -68,7 +68,7 @@ sudo -u git -H git checkout 8-2-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch -sudo -u git -H git checkout v2.6.7 +sudo -u git -H git checkout v2.6.8 ``` ### 5. Replace gitlab-git-http-server with gitlab-workhorse -- cgit v1.2.1 From a9642ba046db9aa4490d3361b32cdba48b3fbb9c Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Mon, 30 Nov 2015 19:05:28 +0100 Subject: Install gitlab-shell 2.6.8 --- doc/install/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 3f5c03a890a..61647780607 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -312,7 +312,7 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da GitLab Shell is an SSH access and repository management software developed specially for GitLab. # Run the installation task for gitlab-shell (replace `REDIS_URL` if needed): - sudo -u git -H bundle exec rake gitlab:shell:install[v2.6.7] REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:shell:install[v2.6.8] REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production # By default, the gitlab-shell config is generated from your main GitLab config. # You can review (and modify) the gitlab-shell config as follows: -- cgit v1.2.1 From 5f6117c0aa391a6e9c96493ca01a8a23eb40f0cd Mon Sep 17 00:00:00 2001 From: Job van der Voort Date: Thu, 5 Nov 2015 16:53:05 +0100 Subject: update ci docs with yml reason and better quickstart --- doc/ci/quick_start/README.md | 202 +++++++++++++++++++++++++++++-------------- doc/ci/yaml/README.md | 30 ++++--- 2 files changed, 156 insertions(+), 76 deletions(-) (limited to 'doc') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index d69064a91fd..74626c8f8a4 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,44 +1,62 @@ # Quick Start -To start building projects with GitLab CI a few steps needs to be done. +GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You +only need to enable it in the Services settings of your project. -## 1. Install GitLab and CI +This guide assumes that you: -First you need to have a working GitLab and GitLab CI instance. +- have a working GitLab instance of version 8.0 or higher or are using + [GitLab.com](https://gitlab.com/users/sign_in) +- have a project in GitLab that you would like to use CI for -You can omit this step if you use [GitLab.com](https://GitLab.com/). +In brief, the steps needed to have a working CI can be summed up to: -## 2. Create repository on GitLab +1. Create a new project +1. Add `.gitlab-ci.yml` to the git repository and push to GitLab +1. Configure a Runner -Once you login on your GitLab add a new repository where you will store your source code. -Push your application to that repository. +From there on, on every push to your git repository the build will be +automagically started by the Runner and will appear under the project's +`/builds` page. -## 3. Add project to CI +Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -The next part is to login to GitLab CI. -Point your browser to the URL you have set GitLab or use [gitlab.com/ci](https://gitlab.com/ci/). +## 1. Creating a `.gitlab-ci.yml` file -On the first screen you will see a list of GitLab's projects that you have access to: + **GitLab CI** service is enabled automatically on the first push of a + `.gitlab-ci.yml` file in your repository and this is the recommended way. -![Projects](projects.png) +For other methods read [how to enable the GitLab CI service](../enable_ci.md). -Click **Add Project to CI**. -This will create project in CI and authorize GitLab CI to fetch sources from GitLab. +### What is `.gitlab-ci.yml` -> GitLab CI creates unique token that is used to configure GitLab CI service in GitLab. -> This token allows to access GitLab's repository and configures GitLab to trigger GitLab CI webhook on **Push events** and **Tag push events**. -> You can see that token by going to Project's Settings > Services > GitLab CI. -> You will see there token, the same token is assigned in GitLab CI settings of project. +The `.gitlab-ci.yml` file is where you configure what CI does with your project. +It lives in the root of your repository. -## 4. Create project's configuration - .gitlab-ci.yml +On any push to your repository, GitLab will look for the `.gitlab-ci.yml` +file and start builds on _Runners_ according to the contents of the file, +for that commit. -The next: You have to define how your project will be built. -GitLab CI uses [YAML](https://en.wikipedia.org/wiki/YAML) file to store build configuration. -You need to create `.gitlab-ci.yml` in root directory of your repository: +Because `.gitlab-ci.yml` is in the repository, it is version controlled, +old versions still build succesfully, forks can easily make use of CI, +branches can have separate builds and you have a single source of truth for CI. +You can read more about the reasons why we are using `.gitlab-ci.yml` +[in our blog about it][blog-ci]. + +`.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file. + +### Creating a simple `.gitlab-ci.yml` file + +You need to create a file named `.gitlab-ci.yml` in the root directory of your +repository. Below is an example for a Ruby on Rails project. ```yaml before_script: - - bundle install + - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs + - ruby -v + - which ruby + - gem install bundler --no-ri --no-rdoc + - bundle install --jobs $(nproc) "${FLAGS[@]}" rspec: script: @@ -49,71 +67,129 @@ rubocop: - bundle exec rubocop ``` -This is the simplest possible build configuration that will work for most Ruby applications: -1. Define two jobs `rspec` and `rubocop` with two different commands to be executed. -1. Before every job execute commands defined by `before_script`. +This is the simplest possible build configuration that will work for most Ruby +applications: + +1. Define two jobs `rspec` and `rubocop` with different commands to be executed. +1. Before every job, the commands defined by `before_script` are executed. + +The `.gitlab-ci.yml` file defines sets of jobs with constraints of how and when +they should be run. The jobs are defined as top-level elements with a name and +always have to contain the `script` name. Jobs are used to create builds, +which are then picked by [Runners](../runners/README.md) and executed within +the environment of the Runner. + +What is important is that each job is run independently from each other. -The `.gitlab-ci.yml` defines set of jobs with constrains how and when they should be run. -The jobs are defined as top-level elements with name and always have to contain the `script`. -Jobs are used to create builds, which are then picked by [runners](../runners/README.md) and executed within environment of the runner. -What is important that each job is run independently from each other. +If you want to check whether your `.gitlab-ci.yml` file is valid, there is a +Lint tool under the page `/ci/lint` of your GitLab instance. You can also find +the link under **Settings** -> **CI settings** in your project. -For more information and complete `.gitlab-ci.yml` syntax, please check the [Configuring project (.gitlab-ci.yml)](../yaml/README.md). +For more information and a complete `.gitlab-ci.yml` syntax, please check +[the documentation on .gitlab-ci.yml](../yaml/README.md). -## 5. Add file and push .gitlab-ci.yml to repository +### Push `.gitlab-ci.yml` to GitLab -Once you created `.gitlab-ci.yml` you should add it to git repository and push it to GitLab. +Once you've created `.gitlab-ci.yml`, you should add it to your git repository +and push it to GitLab. ```bash git add .gitlab-ci.yml -git commit +git commit -m "Add .gitlab-ci.yml" git push origin master ``` -If you refresh the project's page on GitLab CI you will notice a one new commit: +Now if you go to the **Builds** page you will see that the builds are pending. + +You can also go to the **Commits** page and notice the little clock icon next +to the commit SHA. + +![New commit pending](img/new_commit.png) + +Clicking on the clock icon you will be directed to the builds page for that +specific commit. + +![Single commit builds page](img/single_commit_status_pending.png) + +Notice that there are two jobs pending which are named after what we wrote in +`.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured +yet for these builds. + +The next step is to configure a Runner so that it picks the pending jobs. + +## 2. Configuring a Runner + +In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. +A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker +container or even a cluster of containers. GitLab and the Runners communicate +through an API, so the only needed requirement is that the machine on which the +Runner is configured to have Internet access. + +A Runner can be specific to a certain project or serve multiple projects in +GitLab. If it serves all projects it's called a _Shared Runner_. + +Find more information about different Runners in the +[Runners](../runners/README.md) documentation. + +You can find whether any Runners are assigned to your project by going to +**Settings** -> **Runners**. + +Setting up a Runner is easy and straightforward. The official Runner supported +by GitLab is written in Go and can be found at +. + +In order to have a functional Runner you need to: + +1. [Install it][runner-install] +2. [Configure it](../runners/README.md#registering-a-specific-runner) + +For other types of unofficial Runners written in other languages, see the +[instructions for the various GitLab Runners](https://about.gitlab.com/gitlab-ci/#gitlab-runner). + +Once the Runner has been set up, you should see it on the Runners page of your +project, following **Settings** -> **Runners**. -![](new_commit.png) +![Activated runners](img/runners_activated.png) -However the commit has status **pending** which means that commit was not yet picked by runner. +### Shared Runners -## 6. Configure runner +If you use [GitLab.com](https://gitlab.com/) you can use **Shared Runners** +provided by GitLab Inc. -In GitLab CI, Runners run your builds. -A runner is a machine (can be virtual, bare-metal or VPS) that picks up builds through the coordinator API of GitLab CI. +These are special virtual machines that are run on GitLab's infrastructure that +can build any project. -A runner can be specific to a certain project or serve any project in GitLab CI. -A runner that serves all projects is called a shared runner. -More information about different runner types can be found in [Configuring runner](../runners/README.md). +To enable **Shared Runners** you have to go to your project's +**Settings** -> **Runners** and click **Enable shared runners**. -To check if you have runners assigned to your project go to **Runners**. You will find there information how to setup project specific runner: +[Read more on Shared Runners](../runners/README.md). -1. Install GitLab Runner software. Checkout the [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner) section to install it. -1. Specify following URL during runner setup: https://gitlab.com/ci/ -1. Use the following registration token during setup: TOKEN +## 3. Seeing the status of your build -If you do it correctly your runner should be shown under **Runners activated for this project**: +After configuring the Runner succesfully, you should see the status of your +last commit change from _pending_ to either _running_, _success_ or _failed_. -![](runners_activated.png) +You can view all builds, by going to the **Builds** page in your project. -### Shared runners +![Commit status](img/builds_status.png) -If you use [gitlab.com/ci](https://gitlab.com/ci/) you can use **Shared runners** provided by GitLab Inc. -These are special virtual machines that are run on GitLab's infrastructure that can build any project. -To enable **Shared runners** you have to go to **Runners** and click **Enable shared runners** for this project. +By clicking on a Build ID, you will be able to see the log of that build. +This is important to diagnose why a build failed or acted differently than +you expected. -## 7. Check status of commit +![Build log](img/build_log.png) -If everything went OK and you go to commit, the status of the commit should change from **pending** to either **running**, **success** or **failed**. +You are also able to view the status of any commit in the various pages in +GitLab, such as **Commits** and **Merge Requests**. -![](commit_status.png) +## Next steps -You can click **Build ID** to view build log for specific job. +Awesome! You started using CI in GitLab! -## 8. Congratulations! +Next you can look into doing more with the CI. Many people are using GitLab +to package, containerize, test and deploy software. -You managed to build your first project using GitLab CI. -You may need to tune your `.gitlab-ci.yml` file to implement build plan for your project. -A few examples how it can be done you can find on [Examples](../examples/README.md) page. +We have a number of [examples](../examples/README.md) available. -GitLab CI also offers **the Lint** tool to verify validity of your `.gitlab-ci.yml` which can be useful to troubleshoot potential problems. -The Lint is available from project's settings or by adding `/lint` to GitLab CI url. +[runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation +[blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3dbf1afc7a9..9ee26c41e6d 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -20,6 +20,22 @@ Of course a command can execute code directly (`./configure;make;make install`) Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. What is important, is that each job is run independently from each other. +## Why `.gitlab-ci.yml` + +By placing a single configuration file in the root of your repository, +it is version controlled and you get all the advantages of git. + +In addition, builds for older versions of the repository will work just fine, +as GitLab look at the `.gitlab-ci.yml` of the pushed commit. +This means that forks also build without any problem. + +You can even set up different builds for different branches. This allows you +to only deploy the `production` branch, for instance. + +By having a single source of truth, everyone can view and contribute to the +stability of your CI builds, eventually improving the quality of your development +cycle. + ## .gitlab-ci.yml The YAML syntax allows for using more complex job specifications than in the above example: @@ -185,7 +201,7 @@ This are two parameters that allow for setting a refs policy to limit when jobs There are a few rules that apply to usage of refs policy: -1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. +1. `only` and `except` are exclusive. If both `only` and `except` are defined in job specification only `only` is taken into account. 1. `only` and `except` allow for using the regexp expressions. 1. `only` and `except` allow for using special keywords: `branches` and `tags`. These names can be used for example to exclude all tags and all branches. @@ -198,18 +214,6 @@ job: - branches # use special keyword ``` -1. `only` and `except` allow for specify repository path to filter jobs for forks. -The repository path can be used to have jobs executed only for parent repository. - -```yaml -job: - only: - - branches@gitlab-org/gitlab-ce - except: - - master@gitlab-org/gitlab-ce -``` -The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . - ### tags `tags` is used to select specific runners from the list of all runners that are allowed to run this project. -- cgit v1.2.1 From 2c30d11e80a4bb4112d01be7e1fbe51a321b3beb Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 5 Nov 2015 22:49:57 +0200 Subject: Clean up intro --- doc/ci/quick_start/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 74626c8f8a4..92dad2be3e8 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,8 +16,8 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the Runner and will appear under the project's -`/builds` page. +automagically started by the runner and will appear under the project's `/builds` +page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -- cgit v1.2.1 From 1058652a920d1106d29452aadfef953937a188e5 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 6 Nov 2015 10:43:28 +0200 Subject: Add section of enabling GitLab CI --- doc/ci/quick_start/README.md | 7 +++++++ doc/ci/quick_start/builds_tab.png | Bin 0 -> 3845 bytes doc/ci/quick_start/ci_service_disabled.png | Bin 0 -> 3598 bytes doc/ci/quick_start/ci_service_enabled.png | Bin 0 -> 3545 bytes doc/ci/quick_start/ci_service_mark_active.png | Bin 0 -> 17193 bytes doc/ci/quick_start/enable_ci.md | 24 ++++++++++++++++++++++++ 6 files changed, 31 insertions(+) create mode 100644 doc/ci/quick_start/builds_tab.png create mode 100644 doc/ci/quick_start/ci_service_disabled.png create mode 100644 doc/ci/quick_start/ci_service_enabled.png create mode 100644 doc/ci/quick_start/ci_service_mark_active.png create mode 100644 doc/ci/quick_start/enable_ci.md (limited to 'doc') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 92dad2be3e8..d45ad3e5e23 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -21,6 +21,13 @@ page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. +## 1. Enable GitLab CI + +After creating a new project, the first thing to do is enable the **GitLab CI** +service in your project's settings if it isn't already enabled. + +Read [how to enable the GitLab CI service](enable_ci.md). + ## 1. Creating a `.gitlab-ci.yml` file **GitLab CI** service is enabled automatically on the first push of a diff --git a/doc/ci/quick_start/builds_tab.png b/doc/ci/quick_start/builds_tab.png new file mode 100644 index 00000000000..d088b8b329d Binary files /dev/null and b/doc/ci/quick_start/builds_tab.png differ diff --git a/doc/ci/quick_start/ci_service_disabled.png b/doc/ci/quick_start/ci_service_disabled.png new file mode 100644 index 00000000000..351de4267a4 Binary files /dev/null and b/doc/ci/quick_start/ci_service_disabled.png differ diff --git a/doc/ci/quick_start/ci_service_enabled.png b/doc/ci/quick_start/ci_service_enabled.png new file mode 100644 index 00000000000..dfe7488c1ba Binary files /dev/null and b/doc/ci/quick_start/ci_service_enabled.png differ diff --git a/doc/ci/quick_start/ci_service_mark_active.png b/doc/ci/quick_start/ci_service_mark_active.png new file mode 100644 index 00000000000..8bc8694cec3 Binary files /dev/null and b/doc/ci/quick_start/ci_service_mark_active.png differ diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md new file mode 100644 index 00000000000..6f539c752d8 --- /dev/null +++ b/doc/ci/quick_start/enable_ci.md @@ -0,0 +1,24 @@ +# Enable GitLab CI + +GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You +only need to enable it in the **Services** settings of your project. + +First, head over your project's page that you would like to enable CI for. +If you can see the **Builds** tab in the sidebar, then CI is enabled. + +![Builds tab](builds_tab.png) + +If not, go to **Settings > Services** and search for **GitLab CI**. Its state +should be disabled. + +![CI service disabled](ci_service_disabled.png) + +Click on **GitLab CI** to enter its settings, mark it as active and hit +**Save**. + +![Mark CI service as active](ci_service_mark_active.png) + +Do you see that green dot? Then good, the service is now enabled! You can also +check its status under **Services**. + +![CI service enabled](ci_service_enabled.png) -- cgit v1.2.1 From 3b4806bad95e55a40d3efe02e329d271995d8a70 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Sat, 7 Nov 2015 09:11:37 +0200 Subject: Add option to enable GitLab CI via .gitlab-ci.yml --- doc/ci/quick_start/enable_ci.md | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) (limited to 'doc') diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md index 6f539c752d8..42635e35d5d 100644 --- a/doc/ci/quick_start/enable_ci.md +++ b/doc/ci/quick_start/enable_ci.md @@ -1,15 +1,30 @@ # Enable GitLab CI -GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You -only need to enable it in the **Services** settings of your project. +GitLab Continuous Integration (CI) is fully integrated into GitLab itself as +of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). -First, head over your project's page that you would like to enable CI for. -If you can see the **Builds** tab in the sidebar, then CI is enabled. +First, head over your project's page that you would like to enable GitLab CI +for. If you can see the **Builds** tab in the sidebar, then GitLab CI is +already enabled and you can skip reading the rest of this guide. ![Builds tab](builds_tab.png) -If not, go to **Settings > Services** and search for **GitLab CI**. Its state -should be disabled. +If not, there are two ways to enable it in your project. + +## Use .gitlab-ci.yml to enable GitLab CI + +GitLab CI will be automatically enabled if you just push a +[`.gitlab-ci.yml`](../yaml/README.md) in your git repository. GitLab will +pick up the change immediately and GitLab CI will be enabled for this project. +This is the recommended way. + +## Manually enable GitLab CI + +The second way is to manually enable it in the project's **Services** settings +and this is also the way to disable it if needed. + +Go to **Settings > Services** and search for **GitLab CI**. Its state should +be disabled. ![CI service disabled](ci_service_disabled.png) -- cgit v1.2.1 From 32a3c3e068e957716eac56a755f44268c07c5957 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 03:26:51 +0200 Subject: Place images in their own dir --- doc/ci/enable_ci.md | 39 ++++++++++++++++++++++++++ doc/ci/img/builds_tab.png | Bin 0 -> 3845 bytes doc/ci/img/ci_service_disabled.png | Bin 0 -> 3598 bytes doc/ci/img/ci_service_enabled.png | Bin 0 -> 3545 bytes doc/ci/img/ci_service_mark_active.png | Bin 0 -> 17193 bytes doc/ci/quick_start/README.md | 14 ++++----- doc/ci/quick_start/build_status.png | Bin 62140 -> 0 bytes doc/ci/quick_start/builds_tab.png | Bin 3845 -> 0 bytes doc/ci/quick_start/ci_service_disabled.png | Bin 3598 -> 0 bytes doc/ci/quick_start/ci_service_enabled.png | Bin 3545 -> 0 bytes doc/ci/quick_start/ci_service_mark_active.png | Bin 17193 -> 0 bytes doc/ci/quick_start/commit_status.png | Bin 33492 -> 0 bytes doc/ci/quick_start/enable_ci.md | 39 -------------------------- doc/ci/quick_start/img/build_status.png | Bin 0 -> 62140 bytes doc/ci/quick_start/img/commit_status.png | Bin 0 -> 33492 bytes doc/ci/quick_start/img/new_commit.png | Bin 0 -> 47527 bytes doc/ci/quick_start/img/projects.png | Bin 0 -> 37014 bytes doc/ci/quick_start/img/runners.png | Bin 0 -> 123048 bytes doc/ci/quick_start/img/runners_activated.png | Bin 0 -> 60769 bytes doc/ci/quick_start/new_commit.png | Bin 47527 -> 0 bytes doc/ci/quick_start/projects.png | Bin 37014 -> 0 bytes doc/ci/quick_start/runners.png | Bin 123048 -> 0 bytes doc/ci/quick_start/runners_activated.png | Bin 60769 -> 0 bytes 23 files changed, 45 insertions(+), 47 deletions(-) create mode 100644 doc/ci/enable_ci.md create mode 100644 doc/ci/img/builds_tab.png create mode 100644 doc/ci/img/ci_service_disabled.png create mode 100644 doc/ci/img/ci_service_enabled.png create mode 100644 doc/ci/img/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/build_status.png delete mode 100644 doc/ci/quick_start/builds_tab.png delete mode 100644 doc/ci/quick_start/ci_service_disabled.png delete mode 100644 doc/ci/quick_start/ci_service_enabled.png delete mode 100644 doc/ci/quick_start/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/commit_status.png delete mode 100644 doc/ci/quick_start/enable_ci.md create mode 100644 doc/ci/quick_start/img/build_status.png create mode 100644 doc/ci/quick_start/img/commit_status.png create mode 100644 doc/ci/quick_start/img/new_commit.png create mode 100644 doc/ci/quick_start/img/projects.png create mode 100644 doc/ci/quick_start/img/runners.png create mode 100644 doc/ci/quick_start/img/runners_activated.png delete mode 100644 doc/ci/quick_start/new_commit.png delete mode 100644 doc/ci/quick_start/projects.png delete mode 100644 doc/ci/quick_start/runners.png delete mode 100644 doc/ci/quick_start/runners_activated.png (limited to 'doc') diff --git a/doc/ci/enable_ci.md b/doc/ci/enable_ci.md new file mode 100644 index 00000000000..01c684dabec --- /dev/null +++ b/doc/ci/enable_ci.md @@ -0,0 +1,39 @@ +# Enable GitLab CI + +GitLab Continuous Integration (CI) is fully integrated into GitLab itself as +of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). + +First, head over your project's page that you would like to enable GitLab CI +for. If you can see the **Builds** tab in the sidebar, then GitLab CI is +already enabled and you can skip reading the rest of this guide. + +![Builds tab](img/builds_tab.png) + +If not, there are two ways to enable it in your project. + +## Use .gitlab-ci.yml to enable GitLab CI + +GitLab CI will be automatically enabled if you just push a +[`.gitlab-ci.yml`](yaml/README.md) in your git repository. GitLab will +pick up the change immediately and GitLab CI will be enabled for this project. +This is the recommended way. + +## Manually enable GitLab CI + +The second way is to manually enable it in the project's **Services** settings +and this is also the way to disable it if needed. + +Go to **Settings > Services** and search for **GitLab CI**. Its state should +be disabled. + +![CI service disabled](img/ci_service_disabled.png) + +Click on **GitLab CI** to enter its settings, mark it as active and hit +**Save**. + +![Mark CI service as active](img/ci_service_mark_active.png) + +Do you see that green dot? Then good, the service is now enabled! You can also +check its status under **Services**. + +![CI service enabled](img/ci_service_enabled.png) diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png new file mode 100644 index 00000000000..d088b8b329d Binary files /dev/null and b/doc/ci/img/builds_tab.png differ diff --git a/doc/ci/img/ci_service_disabled.png b/doc/ci/img/ci_service_disabled.png new file mode 100644 index 00000000000..351de4267a4 Binary files /dev/null and b/doc/ci/img/ci_service_disabled.png differ diff --git a/doc/ci/img/ci_service_enabled.png b/doc/ci/img/ci_service_enabled.png new file mode 100644 index 00000000000..dfe7488c1ba Binary files /dev/null and b/doc/ci/img/ci_service_enabled.png differ diff --git a/doc/ci/img/ci_service_mark_active.png b/doc/ci/img/ci_service_mark_active.png new file mode 100644 index 00000000000..8bc8694cec3 Binary files /dev/null and b/doc/ci/img/ci_service_mark_active.png differ diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index d45ad3e5e23..ce05a6f417c 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,19 +16,17 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the runner and will appear under the project's `/builds` -page. +automagically started by the runner and will appear under the project's +`/builds` page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -## 1. Enable GitLab CI - -After creating a new project, the first thing to do is enable the **GitLab CI** -service in your project's settings if it isn't already enabled. +## 1. Creating a `.gitlab-ci.yml` file -Read [how to enable the GitLab CI service](enable_ci.md). + **GitLab CI** service is enabled automatically on the first push of a + `.gitlab-ci.yml` file in your repository and this is the recommended way. -## 1. Creating a `.gitlab-ci.yml` file +For other methods read [how to enable the GitLab CI service](../enable_ci.md). **GitLab CI** service is enabled automatically on the first push of a `.gitlab-ci.yml` file in your repository and this is the recommended way. diff --git a/doc/ci/quick_start/build_status.png b/doc/ci/quick_start/build_status.png deleted file mode 100644 index 333259e6acd..00000000000 Binary files a/doc/ci/quick_start/build_status.png and /dev/null differ diff --git a/doc/ci/quick_start/builds_tab.png b/doc/ci/quick_start/builds_tab.png deleted file mode 100644 index d088b8b329d..00000000000 Binary files a/doc/ci/quick_start/builds_tab.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_disabled.png b/doc/ci/quick_start/ci_service_disabled.png deleted file mode 100644 index 351de4267a4..00000000000 Binary files a/doc/ci/quick_start/ci_service_disabled.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_enabled.png b/doc/ci/quick_start/ci_service_enabled.png deleted file mode 100644 index dfe7488c1ba..00000000000 Binary files a/doc/ci/quick_start/ci_service_enabled.png and /dev/null differ diff --git a/doc/ci/quick_start/ci_service_mark_active.png b/doc/ci/quick_start/ci_service_mark_active.png deleted file mode 100644 index 8bc8694cec3..00000000000 Binary files a/doc/ci/quick_start/ci_service_mark_active.png and /dev/null differ diff --git a/doc/ci/quick_start/commit_status.png b/doc/ci/quick_start/commit_status.png deleted file mode 100644 index 725b79e6f91..00000000000 Binary files a/doc/ci/quick_start/commit_status.png and /dev/null differ diff --git a/doc/ci/quick_start/enable_ci.md b/doc/ci/quick_start/enable_ci.md deleted file mode 100644 index 42635e35d5d..00000000000 --- a/doc/ci/quick_start/enable_ci.md +++ /dev/null @@ -1,39 +0,0 @@ -# Enable GitLab CI - -GitLab Continuous Integration (CI) is fully integrated into GitLab itself as -of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). - -First, head over your project's page that you would like to enable GitLab CI -for. If you can see the **Builds** tab in the sidebar, then GitLab CI is -already enabled and you can skip reading the rest of this guide. - -![Builds tab](builds_tab.png) - -If not, there are two ways to enable it in your project. - -## Use .gitlab-ci.yml to enable GitLab CI - -GitLab CI will be automatically enabled if you just push a -[`.gitlab-ci.yml`](../yaml/README.md) in your git repository. GitLab will -pick up the change immediately and GitLab CI will be enabled for this project. -This is the recommended way. - -## Manually enable GitLab CI - -The second way is to manually enable it in the project's **Services** settings -and this is also the way to disable it if needed. - -Go to **Settings > Services** and search for **GitLab CI**. Its state should -be disabled. - -![CI service disabled](ci_service_disabled.png) - -Click on **GitLab CI** to enter its settings, mark it as active and hit -**Save**. - -![Mark CI service as active](ci_service_mark_active.png) - -Do you see that green dot? Then good, the service is now enabled! You can also -check its status under **Services**. - -![CI service enabled](ci_service_enabled.png) diff --git a/doc/ci/quick_start/img/build_status.png b/doc/ci/quick_start/img/build_status.png new file mode 100644 index 00000000000..333259e6acd Binary files /dev/null and b/doc/ci/quick_start/img/build_status.png differ diff --git a/doc/ci/quick_start/img/commit_status.png b/doc/ci/quick_start/img/commit_status.png new file mode 100644 index 00000000000..725b79e6f91 Binary files /dev/null and b/doc/ci/quick_start/img/commit_status.png differ diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png new file mode 100644 index 00000000000..3839e893c17 Binary files /dev/null and b/doc/ci/quick_start/img/new_commit.png differ diff --git a/doc/ci/quick_start/img/projects.png b/doc/ci/quick_start/img/projects.png new file mode 100644 index 00000000000..0b3430a69db Binary files /dev/null and b/doc/ci/quick_start/img/projects.png differ diff --git a/doc/ci/quick_start/img/runners.png b/doc/ci/quick_start/img/runners.png new file mode 100644 index 00000000000..25b4046bc00 Binary files /dev/null and b/doc/ci/quick_start/img/runners.png differ diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png new file mode 100644 index 00000000000..c934bd12f41 Binary files /dev/null and b/doc/ci/quick_start/img/runners_activated.png differ diff --git a/doc/ci/quick_start/new_commit.png b/doc/ci/quick_start/new_commit.png deleted file mode 100644 index 3839e893c17..00000000000 Binary files a/doc/ci/quick_start/new_commit.png and /dev/null differ diff --git a/doc/ci/quick_start/projects.png b/doc/ci/quick_start/projects.png deleted file mode 100644 index 0b3430a69db..00000000000 Binary files a/doc/ci/quick_start/projects.png and /dev/null differ diff --git a/doc/ci/quick_start/runners.png b/doc/ci/quick_start/runners.png deleted file mode 100644 index 25b4046bc00..00000000000 Binary files a/doc/ci/quick_start/runners.png and /dev/null differ diff --git a/doc/ci/quick_start/runners_activated.png b/doc/ci/quick_start/runners_activated.png deleted file mode 100644 index c934bd12f41..00000000000 Binary files a/doc/ci/quick_start/runners_activated.png and /dev/null differ -- cgit v1.2.1 From e70fb9cf75d08e8439d94637726dbee7e2324ab6 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 03:49:32 +0200 Subject: New images for the commit build status --- doc/ci/quick_start/README.md | 2 -- doc/ci/quick_start/img/new_commit.png | Bin 47527 -> 9033 bytes .../quick_start/img/single_commit_status_pending.png | Bin 0 -> 36431 bytes doc/ci/quick_start/img/status_pending.png | Bin 0 -> 19782 bytes 4 files changed, 2 deletions(-) create mode 100644 doc/ci/quick_start/img/single_commit_status_pending.png create mode 100644 doc/ci/quick_start/img/status_pending.png (limited to 'doc') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index ce05a6f417c..073146b9d15 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -120,8 +120,6 @@ Notice that there are two jobs pending which are named after what we wrote in `.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured yet for these builds. -The next step is to configure a Runner so that it picks the pending jobs. - ## 2. Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png index 3839e893c17..3d3c9d5c0bd 100644 Binary files a/doc/ci/quick_start/img/new_commit.png and b/doc/ci/quick_start/img/new_commit.png differ diff --git a/doc/ci/quick_start/img/single_commit_status_pending.png b/doc/ci/quick_start/img/single_commit_status_pending.png new file mode 100644 index 00000000000..23b3bb5acfc Binary files /dev/null and b/doc/ci/quick_start/img/single_commit_status_pending.png differ diff --git a/doc/ci/quick_start/img/status_pending.png b/doc/ci/quick_start/img/status_pending.png new file mode 100644 index 00000000000..a049ec2a5ba Binary files /dev/null and b/doc/ci/quick_start/img/status_pending.png differ -- cgit v1.2.1 From 11b204f1312c3d6aa579b7a04cb4bcf794ad5239 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 10 Nov 2015 21:16:12 +0200 Subject: Final touches for the quick start quide --- doc/ci/quick_start/README.md | 6 ++++-- doc/ci/quick_start/img/build_log.png | Bin 0 -> 63272 bytes doc/ci/quick_start/img/builds_status.png | Bin 0 -> 49121 bytes doc/ci/quick_start/img/runners_activated.png | Bin 60769 -> 27597 bytes 4 files changed, 4 insertions(+), 2 deletions(-) create mode 100644 doc/ci/quick_start/img/build_log.png create mode 100644 doc/ci/quick_start/img/builds_status.png (limited to 'doc') diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 073146b9d15..7771d78d91f 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -16,7 +16,7 @@ In brief, the steps needed to have a working CI can be summed up to: 1. Configure a Runner From there on, on every push to your git repository the build will be -automagically started by the runner and will appear under the project's +automagically started by the Runner and will appear under the project's `/builds` page. Now, let's break it down to pieces and work on solving the GitLab CI puzzle. @@ -120,6 +120,8 @@ Notice that there are two jobs pending which are named after what we wrote in `.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured yet for these builds. +The next step is to configure a Runner so that it picks the pending jobs. + ## 2. Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. @@ -136,8 +138,8 @@ Find more information about different Runners in the You can find whether any Runners are assigned to your project by going to **Settings** -> **Runners**. - Setting up a Runner is easy and straightforward. The official Runner supported + by GitLab is written in Go and can be found at . diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png new file mode 100644 index 00000000000..89e6cd40cb6 Binary files /dev/null and b/doc/ci/quick_start/img/build_log.png differ diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png new file mode 100644 index 00000000000..b8e6c2a361a Binary files /dev/null and b/doc/ci/quick_start/img/builds_status.png differ diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png index c934bd12f41..eafcfd6ecd5 100644 Binary files a/doc/ci/quick_start/img/runners_activated.png and b/doc/ci/quick_start/img/runners_activated.png differ -- cgit v1.2.1 From aea5b72faafa6911377cb0c9fc0801269ac59e1e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 11 Nov 2015 09:57:21 +0200 Subject: Remove old images --- doc/ci/quick_start/img/build_status.png | Bin 62140 -> 0 bytes doc/ci/quick_start/img/commit_status.png | Bin 33492 -> 0 bytes doc/ci/quick_start/img/projects.png | Bin 37014 -> 0 bytes 3 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 doc/ci/quick_start/img/build_status.png delete mode 100644 doc/ci/quick_start/img/commit_status.png delete mode 100644 doc/ci/quick_start/img/projects.png (limited to 'doc') diff --git a/doc/ci/quick_start/img/build_status.png b/doc/ci/quick_start/img/build_status.png deleted file mode 100644 index 333259e6acd..00000000000 Binary files a/doc/ci/quick_start/img/build_status.png and /dev/null differ diff --git a/doc/ci/quick_start/img/commit_status.png b/doc/ci/quick_start/img/commit_status.png deleted file mode 100644 index 725b79e6f91..00000000000 Binary files a/doc/ci/quick_start/img/commit_status.png and /dev/null differ diff --git a/doc/ci/quick_start/img/projects.png b/doc/ci/quick_start/img/projects.png deleted file mode 100644 index 0b3430a69db..00000000000 Binary files a/doc/ci/quick_start/img/projects.png and /dev/null differ -- cgit v1.2.1 From 6805f1bc47640886020e5febf015a73e5862114c Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 1 Dec 2015 10:05:24 +0200 Subject: Clean up quick start quide [ci skip] * Remove references to enabling CI since it it will be deprecated * Revert changes in yaml/README.md, refine it in a separate MR --- doc/ci/enable_ci.md | 39 ---------------------- doc/ci/img/ci_service_disabled.png | Bin 3598 -> 0 bytes doc/ci/img/ci_service_enabled.png | Bin 3545 -> 0 bytes doc/ci/img/ci_service_mark_active.png | Bin 17193 -> 0 bytes doc/ci/quick_start/README.md | 61 ++++++++++++++++------------------ doc/ci/quick_start/img/runners.png | Bin 123048 -> 0 bytes doc/ci/yaml/README.md | 30 ++++++++--------- 7 files changed, 42 insertions(+), 88 deletions(-) delete mode 100644 doc/ci/enable_ci.md delete mode 100644 doc/ci/img/ci_service_disabled.png delete mode 100644 doc/ci/img/ci_service_enabled.png delete mode 100644 doc/ci/img/ci_service_mark_active.png delete mode 100644 doc/ci/quick_start/img/runners.png (limited to 'doc') diff --git a/doc/ci/enable_ci.md b/doc/ci/enable_ci.md deleted file mode 100644 index 01c684dabec..00000000000 --- a/doc/ci/enable_ci.md +++ /dev/null @@ -1,39 +0,0 @@ -# Enable GitLab CI - -GitLab Continuous Integration (CI) is fully integrated into GitLab itself as -of [version 8.0](https://about.gitlab.com/2015/09/22/gitlab-8-0-released/). - -First, head over your project's page that you would like to enable GitLab CI -for. If you can see the **Builds** tab in the sidebar, then GitLab CI is -already enabled and you can skip reading the rest of this guide. - -![Builds tab](img/builds_tab.png) - -If not, there are two ways to enable it in your project. - -## Use .gitlab-ci.yml to enable GitLab CI - -GitLab CI will be automatically enabled if you just push a -[`.gitlab-ci.yml`](yaml/README.md) in your git repository. GitLab will -pick up the change immediately and GitLab CI will be enabled for this project. -This is the recommended way. - -## Manually enable GitLab CI - -The second way is to manually enable it in the project's **Services** settings -and this is also the way to disable it if needed. - -Go to **Settings > Services** and search for **GitLab CI**. Its state should -be disabled. - -![CI service disabled](img/ci_service_disabled.png) - -Click on **GitLab CI** to enter its settings, mark it as active and hit -**Save**. - -![Mark CI service as active](img/ci_service_mark_active.png) - -Do you see that green dot? Then good, the service is now enabled! You can also -check its status under **Services**. - -![CI service enabled](img/ci_service_enabled.png) diff --git a/doc/ci/img/ci_service_disabled.png b/doc/ci/img/ci_service_disabled.png deleted file mode 100644 index 351de4267a4..00000000000 Binary files a/doc/ci/img/ci_service_disabled.png and /dev/null differ diff --git a/doc/ci/img/ci_service_enabled.png b/doc/ci/img/ci_service_enabled.png deleted file mode 100644 index dfe7488c1ba..00000000000 Binary files a/doc/ci/img/ci_service_enabled.png and /dev/null differ diff --git a/doc/ci/img/ci_service_mark_active.png b/doc/ci/img/ci_service_mark_active.png deleted file mode 100644 index 8bc8694cec3..00000000000 Binary files a/doc/ci/img/ci_service_mark_active.png and /dev/null differ diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 7771d78d91f..a9b36139de9 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,7 +1,7 @@ # Quick Start -GitLab Continuous Integration (CI) is fully integrated into GitLab itself. You -only need to enable it in the Services settings of your project. +Starting from version 8.0, GitLab Continuous Integration (CI) is fully +integrated into GitLab itself and is enabled by default on all projects. This guide assumes that you: @@ -21,17 +21,10 @@ automagically started by the Runner and will appear under the project's Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -## 1. Creating a `.gitlab-ci.yml` file +## Creating a `.gitlab-ci.yml` file - **GitLab CI** service is enabled automatically on the first push of a - `.gitlab-ci.yml` file in your repository and this is the recommended way. - -For other methods read [how to enable the GitLab CI service](../enable_ci.md). - - **GitLab CI** service is enabled automatically on the first push of a - `.gitlab-ci.yml` file in your repository and this is the recommended way. - -For other methods read [how to enable the GitLab CI service](../enable_ci.md). +Before you create `.gitlab-ci.yml` let's first explain in brief what this is +all about. ### What is `.gitlab-ci.yml` @@ -48,7 +41,9 @@ branches can have separate builds and you have a single source of truth for CI. You can read more about the reasons why we are using `.gitlab-ci.yml` [in our blog about it][blog-ci]. -`.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file. +**Note:** `.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file +so you have to pay extra attention to the identation. Always use spaces, not +tabs. ### Creating a simple `.gitlab-ci.yml` file @@ -75,20 +70,21 @@ rubocop: This is the simplest possible build configuration that will work for most Ruby applications: -1. Define two jobs `rspec` and `rubocop` with different commands to be executed. +1. Define two jobs `rspec` and `rubocop` (the names are arbitrary) with + different commands to be executed. 1. Before every job, the commands defined by `before_script` are executed. The `.gitlab-ci.yml` file defines sets of jobs with constraints of how and when -they should be run. The jobs are defined as top-level elements with a name and -always have to contain the `script` name. Jobs are used to create builds, -which are then picked by [Runners](../runners/README.md) and executed within -the environment of the Runner. +they should be run. The jobs are defined as top-level elements with a name (in +our case `rspec` and `rubocop`) and always have to contain the `script` keyword. +Jobs are used to create builds, which are then picked by +[Runners](../runners/README.md) and executed within the environment of the Runner. What is important is that each job is run independently from each other. If you want to check whether your `.gitlab-ci.yml` file is valid, there is a Lint tool under the page `/ci/lint` of your GitLab instance. You can also find -the link under **Settings** -> **CI settings** in your project. +the link under **Settings > CI settings** in your project. For more information and a complete `.gitlab-ci.yml` syntax, please check [the documentation on .gitlab-ci.yml](../yaml/README.md). @@ -122,13 +118,13 @@ yet for these builds. The next step is to configure a Runner so that it picks the pending jobs. -## 2. Configuring a Runner +## Configuring a Runner In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker container or even a cluster of containers. GitLab and the Runners communicate through an API, so the only needed requirement is that the machine on which the -Runner is configured to have Internet access. +Runner is configured to has Internet access. A Runner can be specific to a certain project or serve multiple projects in GitLab. If it serves all projects it's called a _Shared Runner_. @@ -137,22 +133,23 @@ Find more information about different Runners in the [Runners](../runners/README.md) documentation. You can find whether any Runners are assigned to your project by going to -**Settings** -> **Runners**. -Setting up a Runner is easy and straightforward. The official Runner supported - -by GitLab is written in Go and can be found at +**Settings > Runners**. Setting up a Runner is easy and straightforward. The +official Runner supported by GitLab is written in Go and can be found at . -In order to have a functional Runner you need to: +In order to have a functional Runner you need to follow two steps: 1. [Install it][runner-install] 2. [Configure it](../runners/README.md#registering-a-specific-runner) +Follow the links above to set up your own Runner or use a Shared Runner as +described in the next section. + For other types of unofficial Runners written in other languages, see the [instructions for the various GitLab Runners](https://about.gitlab.com/gitlab-ci/#gitlab-runner). Once the Runner has been set up, you should see it on the Runners page of your -project, following **Settings** -> **Runners**. +project, following **Settings > Runners**. ![Activated runners](img/runners_activated.png) @@ -161,15 +158,15 @@ project, following **Settings** -> **Runners**. If you use [GitLab.com](https://gitlab.com/) you can use **Shared Runners** provided by GitLab Inc. -These are special virtual machines that are run on GitLab's infrastructure that -can build any project. +These are special virtual machines that run on GitLab's infrastructure and can +build any project. To enable **Shared Runners** you have to go to your project's -**Settings** -> **Runners** and click **Enable shared runners**. +**Settings > Runners** and click **Enable shared runners**. [Read more on Shared Runners](../runners/README.md). -## 3. Seeing the status of your build +## Seeing the status of your build After configuring the Runner succesfully, you should see the status of your last commit change from _pending_ to either _running_, _success_ or _failed_. @@ -194,7 +191,7 @@ Awesome! You started using CI in GitLab! Next you can look into doing more with the CI. Many people are using GitLab to package, containerize, test and deploy software. -We have a number of [examples](../examples/README.md) available. +Visit our various languages examples at . [runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation [blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ diff --git a/doc/ci/quick_start/img/runners.png b/doc/ci/quick_start/img/runners.png deleted file mode 100644 index 25b4046bc00..00000000000 Binary files a/doc/ci/quick_start/img/runners.png and /dev/null differ diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 9ee26c41e6d..3dbf1afc7a9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -20,22 +20,6 @@ Of course a command can execute code directly (`./configure;make;make install`) Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. What is important, is that each job is run independently from each other. -## Why `.gitlab-ci.yml` - -By placing a single configuration file in the root of your repository, -it is version controlled and you get all the advantages of git. - -In addition, builds for older versions of the repository will work just fine, -as GitLab look at the `.gitlab-ci.yml` of the pushed commit. -This means that forks also build without any problem. - -You can even set up different builds for different branches. This allows you -to only deploy the `production` branch, for instance. - -By having a single source of truth, everyone can view and contribute to the -stability of your CI builds, eventually improving the quality of your development -cycle. - ## .gitlab-ci.yml The YAML syntax allows for using more complex job specifications than in the above example: @@ -201,7 +185,7 @@ This are two parameters that allow for setting a refs policy to limit when jobs There are a few rules that apply to usage of refs policy: -1. `only` and `except` are exclusive. If both `only` and `except` are defined in job specification only `only` is taken into account. +1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. 1. `only` and `except` allow for using the regexp expressions. 1. `only` and `except` allow for using special keywords: `branches` and `tags`. These names can be used for example to exclude all tags and all branches. @@ -214,6 +198,18 @@ job: - branches # use special keyword ``` +1. `only` and `except` allow for specify repository path to filter jobs for forks. +The repository path can be used to have jobs executed only for parent repository. + +```yaml +job: + only: + - branches@gitlab-org/gitlab-ce + except: + - master@gitlab-org/gitlab-ce +``` +The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . + ### tags `tags` is used to select specific runners from the list of all runners that are allowed to run this project. -- cgit v1.2.1 From 55d0e04ffd3f178f41ae3ef13790eb6882b1ef8e Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Tue, 1 Dec 2015 14:26:57 +0100 Subject: Auto-detect the required gitlab-shell version --- doc/install/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 61647780607..618391e16d2 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -312,7 +312,7 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da GitLab Shell is an SSH access and repository management software developed specially for GitLab. # Run the installation task for gitlab-shell (replace `REDIS_URL` if needed): - sudo -u git -H bundle exec rake gitlab:shell:install[v2.6.8] REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production # By default, the gitlab-shell config is generated from your main GitLab config. # You can review (and modify) the gitlab-shell config as follows: -- cgit v1.2.1 From 25907ebe476a24bfdd2c451f18227d4fcf314b07 Mon Sep 17 00:00:00 2001 From: Zeger-Jan van de Weg Date: Wed, 2 Dec 2015 13:28:21 +0100 Subject: Doc feature Merge When Build Succeeds --- .../disable_merge_when_build_succeeds.png | Bin 0 -> 20551 bytes .../enable_merge_when_build_succeeds.png | Bin 0 -> 13150 bytes doc/workflow/merge_when_build_succeeds.md | 20 ++++++++++++++++++++ 3 files changed, 20 insertions(+) create mode 100644 doc/workflow/merge_requests/disable_merge_when_build_succeeds.png create mode 100644 doc/workflow/merge_requests/enable_merge_when_build_succeeds.png create mode 100644 doc/workflow/merge_when_build_succeeds.md (limited to 'doc') diff --git a/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png b/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png new file mode 100644 index 00000000000..a45a4890b62 Binary files /dev/null and b/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png differ diff --git a/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png b/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png new file mode 100644 index 00000000000..62a46c9508b Binary files /dev/null and b/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png differ diff --git a/doc/workflow/merge_when_build_succeeds.md b/doc/workflow/merge_when_build_succeeds.md new file mode 100644 index 00000000000..9bf6ddcc569 --- /dev/null +++ b/doc/workflow/merge_when_build_succeeds.md @@ -0,0 +1,20 @@ +# Merge When Build Succeeds + +Select a Merge Request to be merged if the build succeeds so the user does not have to wait for the build to finish and revisit the Merge Request to merge it after the build is done. + +## Enabling for a Merge Request + +Given an active build for a Merge Request, thus pending or running, a `Merge When Build Succeeds` button will appear to any user which can merge it. Once clicked, it ensures this merge request is merged when the build is successful. +When clicking the button, the merge parameters are also saved to allow the merge user to edit the commit message and remove the source branch if he can remove that branch. + +When this feature is enabled, a message will appear to notify other users. Also a note is posted on the thread. + +![Enable Merge When Build Succceeds](merge_requests/enable_merge_when_build_succeeds.png) + +## Canceling + +The automatic merge can be disabled by clicking the `Cancel Automatic Merge` button, or when a new commit is added to the Merge Request. In the former case a note is posted. In the latter case a user able to merge can enable the feature again. + +![Disable the automatic merge](merge_requests/disable_merge_when_build_succeeds.png) + +A failed build does not reset the automatic build so a build can be retried. -- cgit v1.2.1 From 0ec0c93824096fc6b26de369e170ffc729967bd3 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 2 Dec 2015 15:54:44 +0200 Subject: Also update gitlab-workhorse in patch updates [ci skip] --- doc/update/patch_versions.md | 45 +++++++++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 593722eb01f..957354decb7 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -6,7 +6,8 @@ For example from 7.14.0 to 7.14.3, also see the [semantic versioning specificati ### 0. Backup It's useful to make a backup just in case things go south: -(With MySQL, this may require granting "LOCK TABLES" privileges to the GitLab user on the database version) +(With MySQL, this may require granting "LOCK TABLES" privileges to the GitLab +user on the database version) ```bash cd /home/git/gitlab @@ -15,19 +16,23 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ### 1. Stop server - sudo service gitlab stop +```bash +sudo service gitlab stop +``` ### 2. Get latest code for the stable branch +In the commands below, replace `LATEST_TAG` with the latest GitLab tag you want +to update to, for example `v8.0.3`. Use `git tag -l 'v*.[0-9]' --sort='v:refname'` +to see a list of all tags. Make sure to update patch versions only (check your +current version with `cat VERSION`). + ```bash cd /home/git/gitlab sudo -u git -H git fetch --all sudo -u git -H git checkout -- Gemfile.lock db/schema.rb sudo -u git -H git checkout LATEST_TAG -b LATEST_TAG ``` -Replace `LATEST_TAG` with the latest GitLab tag you want to update to, for example `v8.0.3`. -Use `git tag -l 'v*.[0-9]' --sort='v:refname'` to see a list of all tags. -Make sure to update patch versions only (check your current version with `cat VERSION`) ### 3. Update gitlab-shell to the corresponding version @@ -37,12 +42,20 @@ sudo -u git -H git fetch sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` ``` -### 4. Install libs, migrations, etc. +### 4. Update gitlab-workhorse to the corresponding version + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch +sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` +``` + +### 5. Install libs, migrations, etc. ```bash cd /home/git/gitlab -#PostgreSQL +# PostgreSQL sudo -u git -H bundle install --without development test mysql --deployment # MySQL @@ -52,19 +65,25 @@ sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production ``` -### 5. Start application +### 6. Start application - sudo service gitlab start - sudo service nginx restart +```bash +sudo service gitlab start +sudo service nginx restart +``` -### 6. Check application status +### 7. Check application status Check if GitLab and its environment are configured correctly: - sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +```bash +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` To make sure you didn't miss anything run a more thorough check with: - sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +```bash +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` If all items are green, then congratulations upgrade complete! -- cgit v1.2.1 From e3fe255c057e86d841a68f9a96b1a62b07036781 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hannes=20Rosen=C3=B6gger?= <123haynes@gmail.com> Date: Wed, 2 Dec 2015 15:21:02 +0100 Subject: fixed the documentation of the Guest role in permission.md --- doc/permissions/permissions.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index 8d4c2ceab7d..bcd00cfc6bf 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -6,6 +6,9 @@ If a user is both in a project group and in the project itself, the highest perm If a user is a GitLab administrator they receive all permissions. +On public projects the Guest role is not enforced. +All users will be able to create issues, leave comments, and pull or download the project code. + To add or import a user, you can follow the [project users and members documentation](doc/workflow/add-user/add-user.md). @@ -15,8 +18,8 @@ documentation](doc/workflow/add-user/add-user.md). |---------------------------------------|---------|------------|-------------|----------|--------| | Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | -| Pull project code | ✓ | ✓ | ✓ | ✓ | ✓ | -| Download project | ✓ | ✓ | ✓ | ✓ | ✓ | +| Pull project code | | ✓ | ✓ | ✓ | ✓ | +| Download project | | ✓ | ✓ | ✓ | ✓ | | Create code snippets | | ✓ | ✓ | ✓ | ✓ | | Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | | Manage labels | | ✓ | ✓ | ✓ | ✓ | -- cgit v1.2.1 From 387e5656a1e158eaa1c010f22d332d758b336179 Mon Sep 17 00:00:00 2001 From: Kevin Pankonen Date: Thu, 3 Dec 2015 15:40:08 -0700 Subject: fixes #3263 slashes are replaced with two underscores --- doc/ci/docker/using_docker_images.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index ef8a7ec1e86..64e52eba3a2 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -60,11 +60,11 @@ This is image that have fully preconfigured `wordpress` and have `MySQL` server ``` Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum_wordpress`. +and you will have access to it from your build container under hostname: `tutum__wordpress`. Alias hostname for the service is made from the image name: 1. Everything after `:` is stripped, -2. '/' is replaced to `_`. +2. '/' is replaced with `__`. ### Configuring services Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. -- cgit v1.2.1 From 0ccd7de7f369d98615833867abe84142bcc25193 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 4 Dec 2015 10:55:36 +0100 Subject: Fix wrong doc in merge request API Signed-off-by: Dmitriy Zaporozhets --- doc/api/merge_requests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 0cef09d5b27..3a1fc406fd1 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -159,7 +159,7 @@ Parameters: "updated_at": "2015-02-02T19:49:26.013Z", "due_date": null }, - "files": [ + "changes": [ { "old_path": "VERSION", "new_path": "VERSION", -- cgit v1.2.1 From 0b68a0e79e1cbe12c44beb6c23e79d48b71f219e Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 4 Dec 2015 11:08:10 +0100 Subject: Add API endpoint to fetch merge request commits list Signed-off-by: Dmitriy Zaporozhets --- doc/api/merge_requests.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 3a1fc406fd1..2b1498c85a0 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -101,6 +101,45 @@ Parameters: } ``` +## Get single MR commits + +Get a list of repository commits in a merge request. + +``` +GET /projects/:id/merge_request/:merge_request_id/commits +``` + +Parameters: + +- `id` (required) - The ID of a project +- `merge_request_id` (required) - The ID of MR + + +```json +[ + { + "id": "ed899a2f4b50b4370feeea94676502b42383c746", + "short_id": "ed899a2f4b5", + "title": "Replace sanitize with escape once", + "author_name": "Dmitriy Zaporozhets", + "author_email": "dzaporozhets@sphereconsultinginc.com", + "created_at": "2012-09-20T11:50:22+03:00", + "message": "Replace sanitize with escape once", + "allow_failure": false + }, + { + "id": "6104942438c14ec7bd21c6cd5bd995272b3faff6", + "short_id": "6104942438c", + "title": "Sanitize for network graph", + "author_name": "randx", + "author_email": "dmitriy.zaporozhets@gmail.com", + "created_at": "2012-09-20T09:06:12+03:00", + "message": "Sanitize for network graph", + "allow_failure": false + } +] +``` + ## Get single MR changes Shows information about the merge request including its files and changes. -- cgit v1.2.1 From c366e81da7fb6c9ef921cbd57e5ac662999f0bc0 Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 4 Dec 2015 11:26:35 +0100 Subject: Improve docs Signed-off-by: Dmitriy Zaporozhets --- doc/api/merge_requests.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 2b1498c85a0..82f2cef969f 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -103,7 +103,7 @@ Parameters: ## Get single MR commits -Get a list of repository commits in a merge request. +Get a list of merge request commits. ``` GET /projects/:id/merge_request/:merge_request_id/commits @@ -124,8 +124,7 @@ Parameters: "author_name": "Dmitriy Zaporozhets", "author_email": "dzaporozhets@sphereconsultinginc.com", "created_at": "2012-09-20T11:50:22+03:00", - "message": "Replace sanitize with escape once", - "allow_failure": false + "message": "Replace sanitize with escape once" }, { "id": "6104942438c14ec7bd21c6cd5bd995272b3faff6", @@ -134,8 +133,7 @@ Parameters: "author_name": "randx", "author_email": "dmitriy.zaporozhets@gmail.com", "created_at": "2012-09-20T09:06:12+03:00", - "message": "Sanitize for network graph", - "allow_failure": false + "message": "Sanitize for network graph" } ] ``` -- cgit v1.2.1 From 3227a5ead22c90873794b0c0e4c788436283fb3e Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Fri, 4 Dec 2015 12:21:06 +0100 Subject: Extent Event and Note API * add note to Events API * add author section to Events API * add noteable_id and noteable_type to Notes API Signed-off-by: Dmitriy Zaporozhets --- doc/api/notes.md | 15 ++++++++--- doc/api/projects.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 79 insertions(+), 7 deletions(-) (limited to 'doc') diff --git a/doc/api/notes.md b/doc/api/notes.md index e7f299c0994..4d7ef288df8 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -35,7 +35,9 @@ Parameters: "created_at": "2013-10-02T09:22:45Z", "system": true, "upvote": false, - "downvote": false + "downvote": false, + "noteable_id": 377, + "noteable_type": "Issue" }, { "id": 305, @@ -52,7 +54,9 @@ Parameters: "created_at": "2013-10-02T09:56:03Z", "system": true, "upvote": false, - "downvote": false + "downvote": false, + "noteable_id": 121, + "noteable_type": "Issue" } ] ``` @@ -219,7 +223,12 @@ Parameters: "state": "active", "created_at": "2013-09-30T13:46:01Z" }, - "created_at": "2013-10-02T08:57:14Z" + "created_at": "2013-10-02T08:57:14Z", + "system": false, + "upvote": false, + "downvote": false, + "noteable_id": 2, + "noteable_type": "MergeRequest" } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 755cc6525c2..42919a312ae 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -245,9 +245,17 @@ Parameters: "target_id": 830, "target_type": "Issue", "author_id": 1, - "author_username": "john", "data": null, - "target_title": "Public project search field" + "target_title": "Public project search field", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" }, { "title": null, @@ -256,6 +264,14 @@ Parameters: "target_id": null, "target_type": null, "author_id": 1, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, "author_username": "john", "data": { "before": "50d4420237a9de7be1304607147aec22e4a14af7", @@ -292,9 +308,56 @@ Parameters: "target_id": 840, "target_type": "Issue", "author_id": 1, - "author_username": "john", "data": null, - "target_title": "Finish & merge Code search PR" + "target_title": "Finish & merge Code search PR", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" + }, + { + "title": null, + "project_id": 15, + "action_name": "commented on", + "target_id": 1312, + "target_type": "Note", + "author_id": 1, + "data": null, + "target_title": null, + "created_at": "2015-12-04T10:33:58.089Z", + "note": { + "id": 1312, + "body": "What an awesome day!", + "attachment": null, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "created_at": "2015-12-04T10:33:56.698Z", + "system": false, + "upvote": false, + "downvote": false, + "noteable_id": 377, + "noteable_type": "Issue" + }, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" } ] ``` -- cgit v1.2.1 From 5c1b49f494f07bf37ba3c60f3b9f70d1842d8b60 Mon Sep 17 00:00:00 2001 From: Valery Sizov Date: Fri, 4 Dec 2015 16:23:21 +0200 Subject: Add added, modified and removed properties to commit object in webhook --- doc/web_hooks/web_hooks.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index 7d838187a26..03746dd9df3 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -57,6 +57,9 @@ X-Gitlab-Event: Push Hook "name": "Jordi Mallach", "email": "jordi@softcatala.org" } + "added": ["CHANGELOG"], + "modified": ["app/controller/application.rb"], + "removed": [] }, { "id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", @@ -66,13 +69,14 @@ X-Gitlab-Event: Push Hook "author": { "name": "GitLab dev user", "email": "gitlabdev@dv6700.(none)" - } + }, + "added": ["CHANGELOG"], + "modified": ["app/controller/application.rb"], + "removed": [] } ], - "total_commits_count": 4, - "added": ["CHANGELOG"], - "modified": ["app/controller/application.rb"], - "removed": [] + "total_commits_count": 4 + } ``` -- cgit v1.2.1 From ee134d09e7cedb57cd021314d5de459a4adf3d4d Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Sat, 5 Dec 2015 15:06:32 -0800 Subject: Move release cycle comments to the documentation. --- doc/release/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/release/README.md b/doc/release/README.md index 1342b90f3b3..52eca7c02a6 100644 --- a/doc/release/README.md +++ b/doc/release/README.md @@ -1,4 +1,8 @@ -GitLab has the following updates: +## Release cycle + +Since 2011 a minor or major version of GitLab is released on the 22nd of every month. Patch and security releases are published when needed. New features are detailed on the [blog](https://about.gitlab.com/blog/) and in the [changelog](CHANGELOG). Features that will likely be in the next releases can be found on the [direction page](https://about.gitlab.com/direction/). + +## Release process documentation - [Monthly release](monthly.md), every month on the 22nd. - [Patch release](patch.md), if there are serious regressions. -- cgit v1.2.1 From 900d3a09a73b953e3271a233aa6fc4937ac9d59c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tobias=20St=C3=B6ckler?= Date: Mon, 7 Dec 2015 10:28:47 +0100 Subject: Fix typos in integration docs --- doc/integration/bitbucket.md | 2 +- doc/integration/crowd.md | 2 +- doc/integration/github.md | 2 +- doc/integration/gitlab.md | 2 +- doc/integration/google.md | 2 +- doc/integration/saml.md | 2 +- doc/integration/twitter.md | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) (limited to 'doc') diff --git a/doc/integration/bitbucket.md b/doc/integration/bitbucket.md index 6a0fa4ce015..63432b04432 100644 --- a/doc/integration/bitbucket.md +++ b/doc/integration/bitbucket.md @@ -30,7 +30,7 @@ Bitbucket will generate an application ID and secret key for you to use. sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/crowd.md b/doc/integration/crowd.md index 2ecc8795ac1..40d93aef2a9 100644 --- a/doc/integration/crowd.md +++ b/doc/integration/crowd.md @@ -10,7 +10,7 @@ To enable the Crowd OmniAuth provider you must register your application with Cr sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/github.md b/doc/integration/github.md index b64501c2aaa..a789d2c814f 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -32,7 +32,7 @@ GitHub will generate an application ID and secret key for you to use. sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index 216f1f11a9b..80e3c0142a0 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -38,7 +38,7 @@ GitLab.com will generate an application ID and secret key for you to use. sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/google.md b/doc/integration/google.md index e1c14c7c948..91e9b2495cc 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -35,7 +35,7 @@ To enable the Google OAuth2 OmniAuth provider you must register your application sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 4aa6dbe758a..1b8c28dd0f4 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -14,7 +14,7 @@ First configure SAML 2.0 support in GitLab, then register the GitLab application sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index 1350c8f693c..52ed4a22339 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -37,7 +37,7 @@ To enable the Twitter OmniAuth provider you must register your application with sudo editor /etc/gitlab/gitlab.rb ``` - For instalations from source: + For installations from source: ```sh cd /home/git/gitlab -- cgit v1.2.1 From a426fb1596978221510c8c78a17703658ad7d161 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Mon, 7 Dec 2015 14:11:13 +0100 Subject: Update documentation about automatic issue closing --- doc/customization/issue_closing.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/doc/customization/issue_closing.md b/doc/customization/issue_closing.md index 64f128f5a63..00edfc97ed9 100644 --- a/doc/customization/issue_closing.md +++ b/doc/customization/issue_closing.md @@ -1,38 +1,39 @@ # Issue closing pattern -Here's how to close multiple issues in one commit message: +When a commit or merge request resolves one or more issues, it is possible to automatically have these issues closed when the commit or merge request lands in the project's default branch. -If a commit message matches the regular expression below, all issues referenced from -the matched text will be closed. This happens when the commit is pushed or merged -into the default branch of a project. +If a commit message or merge request description contains a sentence matching the regular expression below, all issues referenced from +the matched text will be closed. This happens when the commit is pushed to a project's default branch, or when a commit or merge request is merged into there. -When not specified, the default issue_closing_pattern as shown below will be used: +When not specified, the default `issue_closing_pattern` as shown below will be used: ```bash -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?#\d+(?:(?:, *| +and +)?))+) +((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+) ``` +Here, `%{issue_ref}` is a complex regular expression defined inside GitLab, that matches a reference to a local issue (`#123`), cross-project issue (`group/project#123`) or a link to an issue (`https://gitlab.example.com/group/project/issues/123`). + For example: ``` -git commit -m "Awesome commit message (Fix #20, Fixes #21 and Closes #22). This commit is also related to #17 and fixes #18, #19 and #23." +git commit -m "Awesome commit message (Fix #20, Fixes #21 and Closes group/otherproject#2). This commit is also related to #17 and fixes #18, #19 and https://gitlab.example.com/group/otherproject/issues/23." ``` -will close `#20`, `#21`, `#22`, `#18`, `#19` and `#23`, but `#17` won't be closed -as it does not match the pattern. It also works with multiline commit messages. +will close `#18`, `#19`, `#20`, and `#21` in the project this commit is pushed to, as well as `#22` and `#23` in group/otherproject. `#17` won't be closed as it does not match the pattern. It also works with multiline commit messages. Tip: you can test this closing pattern at [http://rubular.com][1]. Use this site to test your own patterns. +Because Rubular doesn't understand `%{issue_ref}`, you can replace this by `#\d+` in testing, which matches only local issue references like `#123`. ## Change the pattern For Omnibus installs you can change the default pattern in `/etc/gitlab/gitlab.rb`: ``` -issue_closing_pattern: '((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?#\d+(?:(?:, *| +and +)?))+)' +issue_closing_pattern: '((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)' ``` -For manual installs you can customize the pattern in [gitlab.yml][0]. +For manual installs you can customize the pattern in [gitlab.yml][0] using the `issue_closing_pattern` key. -[0]: https://gitlab.com/gitlab-org/gitlab-ce/blob/40c3675372320febf5264061c9bcd63db2dfd13c/config/gitlab.yml.example#L65 -[1]: http://rubular.com/r/Xmbexed1OJ \ No newline at end of file +[0]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example +[1]: http://rubular.com/r/Xmbexed1OJ -- cgit v1.2.1 From 8a0507d523293289ae4b7d61bb3fce8a873e3dcc Mon Sep 17 00:00:00 2001 From: "Michael A. Smith" Date: Mon, 7 Dec 2015 10:29:42 -0500 Subject: Update Docker Syntax --- doc/ci/docker/using_docker_images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 64e52eba3a2..1feae62b1c7 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -190,7 +190,7 @@ This will create two service containers (MySQL and PostgreSQL). 1. Create a build container and execute script in its context: ``` -$ cat build_script | docker run -n build -i -l mysql:service-mysql -l postgres:service-postgres ruby:2.1 /bin/bash +$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` This will create build container that has two service containers linked. The build_script is piped using STDIN to bash interpreter which executes the build script in container. -- cgit v1.2.1 From 8f817c7b08bcad23e1b047f84cc60d1748104e2a Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Mon, 7 Dec 2015 17:10:40 +0100 Subject: Add API group projects endpoint. --- doc/api/groups.md | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 67 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/api/groups.md b/doc/api/groups.md index 0b9f6406d8d..808675d8605 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -1,6 +1,6 @@ # Groups -## List project groups +## List groups Get a list of groups. (As user: my groups, as admin: all groups) @@ -21,6 +21,70 @@ GET /groups You can search for groups by name or path, see below. + +## List a group's projects + +Get a list of projects in this group. + +``` +GET /groups/:id/projects +``` + +Parameters: + +- `archived` (optional) - if passed, limit by archived status +- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` +- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` +- `search` (optional) - Return list of authorized projects according to a search criteria +- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first + +```json +[ + { + "id": 4, + "description": null, + "default_branch": "master", + "public": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", + "web_url": "http://example.com/diaspora/diaspora-client", + "tag_list": [ + "example", + "disapora client" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13: 46: 02Z" + }, + "name": "Diaspora Client", + "name_with_namespace": "Diaspora / Diaspora Client", + "path": "diaspora-client", + "path_with_namespace": "diaspora/diaspora-client", + "issues_enabled": true, + "merge_requests_enabled": true, + "builds_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "created_at": "2013-09-30T13: 46: 02Z", + "last_activity_at": "2013-09-30T13: 46: 02Z", + "creator_id": 3, + "namespace": { + "created_at": "2013-09-30T13: 46: 02Z", + "description": "", + "id": 3, + "name": "Diaspora", + "owner_id": 1, + "path": "diaspora", + "updated_at": "2013-09-30T13: 46: 02Z" + }, + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png" + } +] +``` + ## Details of a group Get all details of a group. @@ -186,7 +250,7 @@ To get more (up to 100), pass the following as an argument to the API call: /groups?per_page=100 ``` -And to switch pages add: +And to switch pages add: ``` /groups?per_page=100&page=2 -``` \ No newline at end of file +``` -- cgit v1.2.1 From b2b548de9d74b01816baca822d39f9dd543bbbf7 Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 8 Dec 2015 14:02:34 +0100 Subject: Rewrite docs --- doc/workflow/README.md | 1 + .../disable_merge_when_build_succeeds.png | Bin 20551 -> 0 bytes .../enable_merge_when_build_succeeds.png | Bin 13150 -> 0 bytes doc/workflow/merge_when_build_succeeds.md | 19 +++++++------------ doc/workflow/merge_when_build_succeeds/enable.png | Bin 0 -> 151112 bytes doc/workflow/merge_when_build_succeeds/status.png | Bin 0 -> 180318 bytes 6 files changed, 8 insertions(+), 12 deletions(-) delete mode 100644 doc/workflow/merge_requests/disable_merge_when_build_succeeds.png delete mode 100644 doc/workflow/merge_requests/enable_merge_when_build_succeeds.png create mode 100644 doc/workflow/merge_when_build_succeeds/enable.png create mode 100644 doc/workflow/merge_when_build_succeeds/status.png (limited to 'doc') diff --git a/doc/workflow/README.md b/doc/workflow/README.md index a6b4d951188..d2642495c9a 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -17,4 +17,5 @@ - [Milestones](milestones.md) - [Merge Requests](merge_requests.md) - ["Work In Progress" Merge Requests](wip_merge_requests.md) +- [Merge When Build Succeeds](merge_when_build_succeeds.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) diff --git a/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png b/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png deleted file mode 100644 index a45a4890b62..00000000000 Binary files a/doc/workflow/merge_requests/disable_merge_when_build_succeeds.png and /dev/null differ diff --git a/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png b/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png deleted file mode 100644 index 62a46c9508b..00000000000 Binary files a/doc/workflow/merge_requests/enable_merge_when_build_succeeds.png and /dev/null differ diff --git a/doc/workflow/merge_when_build_succeeds.md b/doc/workflow/merge_when_build_succeeds.md index 9bf6ddcc569..3c055650c49 100644 --- a/doc/workflow/merge_when_build_succeeds.md +++ b/doc/workflow/merge_when_build_succeeds.md @@ -1,20 +1,15 @@ # Merge When Build Succeeds -Select a Merge Request to be merged if the build succeeds so the user does not have to wait for the build to finish and revisit the Merge Request to merge it after the build is done. +When reviewing a merge request that looks ready to merge but still has one or more CI builds running, you can set it to be merged automatically when the build succeeds. This way, you don't have to wait for the build to finish and remember to merge the merge request then. -## Enabling for a Merge Request +![Enable](merge_when_build_succeeds/enable.png) -Given an active build for a Merge Request, thus pending or running, a `Merge When Build Succeeds` button will appear to any user which can merge it. Once clicked, it ensures this merge request is merged when the build is successful. -When clicking the button, the merge parameters are also saved to allow the merge user to edit the commit message and remove the source branch if he can remove that branch. +When you hit the "Merge When Build Succeeds" button, the status of the Merge Request will be updated to represent the impending merge. If you cannot wait for the build to succeed and want to build immediately, this option is available in the dropdown menu on the right of the main button. -When this feature is enabled, a message will appear to notify other users. Also a note is posted on the thread. +Both team developers and the author of the merge request have the option to cancel the automatic merge when they find a reason it shouldn't be merged after all. -![Enable Merge When Build Succceeds](merge_requests/enable_merge_when_build_succeeds.png) +![Status](merge_when_build_succeeds/status.png) -## Canceling +When the build succeeds, the merge request will automatically be merged. When the build fails, the author gets a chance to retry any failed builds, or to push new commits to fix the failure. -The automatic merge can be disabled by clicking the `Cancel Automatic Merge` button, or when a new commit is added to the Merge Request. In the former case a note is posted. In the latter case a user able to merge can enable the feature again. - -![Disable the automatic merge](merge_requests/disable_merge_when_build_succeeds.png) - -A failed build does not reset the automatic build so a build can be retried. +When the builds are retried and succeed on the second try, the merge request will automatically be merged after all. When the merge request is updated with new commits, the automatic merge is automatically canceled to allow the new changes to be reviewed. diff --git a/doc/workflow/merge_when_build_succeeds/enable.png b/doc/workflow/merge_when_build_succeeds/enable.png new file mode 100644 index 00000000000..633efa1246f Binary files /dev/null and b/doc/workflow/merge_when_build_succeeds/enable.png differ diff --git a/doc/workflow/merge_when_build_succeeds/status.png b/doc/workflow/merge_when_build_succeeds/status.png new file mode 100644 index 00000000000..c856c7d14dc Binary files /dev/null and b/doc/workflow/merge_when_build_succeeds/status.png differ -- cgit v1.2.1 From f3ca92a062424e0cda2c077d9c30a4edbd6bf4c8 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Tue, 8 Dec 2015 15:08:22 +0100 Subject: Add 'resume' capability to parallel-rsync-repos --- doc/operations/moving_repositories.md | 50 ++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) (limited to 'doc') diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md index a89602b367f..39086b7a251 100644 --- a/doc/operations/moving_repositories.md +++ b/doc/operations/moving_repositories.md @@ -96,25 +96,59 @@ after switching to the new repository storage directory. ### Parallel rsync for all repositories known to GitLab -This will sync repositories with 10 rsync processes at a time. +This will sync repositories with 10 rsync processes at a time. We keep +track of progress so that the transfer can be restarted if necessary. + +First we create a new directory, owned by 'git', to hold transfer +logs. We assume the directory is empty before we start the transfer +procedure, and that we are the only ones writing files in it. ``` # Omnibus -sudo gitlab-rake gitlab:list_repos |\ - sudo -u git \ +sudo mkdir /var/opt/gitlab/transfer-logs +sudo chown git:git /var/opt/gitlab/transfer-logs + +# Source +sudo -u git -H mkdir /home/git/transfer-logs +``` + +We seed the process with a list of the directories we want to copy. + +``` +# Omnibus +sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt' + +# Source +cd /home/git/gitlab +sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt' +``` + +Now we can start the transfer. The command below is idempotent, and +the number of jobs done by GNU Parallel should converge to zero. If it +does not some repositories listed in all-repos-1234.txt may have been +deleted/renamed before they could be copied. + +``` +# Omnibus +sudo -u git sh -c ' +cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\ /usr/bin/env JOBS=10 \ - /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repoos \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ + /var/opt/gitlab/transfer-logs/succes-$(date +%s).log \ /var/opt/gitlab/git-data/repositories \ /mnt/gitlab/repositories +' # Source cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:list_repos |\ - sudo -u git -H \ +sudo -u git -H sh -c ' +cat /home/git/transfer-logs/* | sort | uniq -u |\ /usr/bin/env JOBS=10 \ bin/parallel-rsync-repos \ + /home/git/transfer-logs/succes-$(date +%s).log \ /home/git/repositories \ /mnt/gitlab/repositories +` ``` ### Parallel rsync only for repositories with recent activity @@ -129,7 +163,8 @@ gitlab:list_repos' to only print repositories with recent activity. sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ sudo -u git \ /usr/bin/env JOBS=10 \ - /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repoos \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ + succes-$(date +%s).log \ /var/opt/gitlab/git-data/repositories \ /mnt/gitlab/repositories @@ -139,6 +174,7 @@ sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ sudo -u git -H \ /usr/bin/env JOBS=10 \ bin/parallel-rsync-repos \ + succes-$(date +%s).log \ /home/git/repositories \ /mnt/gitlab/repositories ``` -- cgit v1.2.1 From d00d3444a45f81a5e79a1161acb44fca2df3c1c7 Mon Sep 17 00:00:00 2001 From: Kelvin Date: Tue, 8 Dec 2015 18:23:30 +0300 Subject: Remove the prepended v on GitLab Workhorse upgrade doc --- doc/update/patch_versions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 957354decb7..c19ee49f9e0 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -47,7 +47,7 @@ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`ca ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch -sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` +sudo -u git -H git checkout `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b `cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` ``` ### 5. Install libs, migrations, etc. -- cgit v1.2.1 From 4f074aaa14faa8a866f18a80f58b66cd023a141f Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Wed, 25 Nov 2015 14:41:14 +0100 Subject: Introduce CI documentation for services and languages --- doc/README.md | 12 +++ doc/ci/README.md | 12 +++ doc/ci/docker/using_docker_images.md | 115 +++++++++------------- doc/ci/languages/README.md | 3 + doc/ci/languages/php.md | 178 +++++++++++++++++++++++++++++++++++ doc/ci/services/README.md | 6 ++ doc/ci/services/docker-services.md | 5 + doc/ci/services/mysql.md | 72 ++++++++++++++ doc/ci/services/postgres.md | 70 ++++++++++++++ doc/ci/services/redis.md | 40 ++++++++ doc/ci/ssh_keys/README.md | 114 ++++++++++++++++++++++ 11 files changed, 559 insertions(+), 68 deletions(-) create mode 100644 doc/ci/languages/README.md create mode 100644 doc/ci/languages/php.md create mode 100644 doc/ci/services/README.md create mode 100644 doc/ci/services/docker-services.md create mode 100644 doc/ci/services/mysql.md create mode 100644 doc/ci/services/postgres.md create mode 100644 doc/ci/services/redis.md create mode 100644 doc/ci/ssh_keys/README.md (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index 58ab5dd08e0..a7025f7af10 100644 --- a/doc/README.md +++ b/doc/README.md @@ -24,9 +24,21 @@ - [Using Docker Images](ci/docker/using_docker_images.md) - [Using Docker Build](ci/docker/using_docker_build.md) - [Using Variables](ci/variables/README.md) +- [Using SSH keys](ci/ssh_keys/README.md) - [User permissions](ci/permissions/README.md) - [API](ci/api/README.md) +### CI Languages + ++ [Testing PHP](ci/languages/php.md) + +## CI Services + ++ [Using MySQL](ci/services/mysql.md) ++ [Using PostgreSQL](ci/services/postgres.md) ++ [Using Redis](ci/services/redis.md) ++ [Using Other Services](ci/docker/using_docker_images.html#how-to-use-other-images-as-services) + ### CI Examples - [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md) diff --git a/doc/ci/README.md b/doc/ci/README.md index 97325069ceb..ae921b6f988 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -9,6 +9,18 @@ + [Using Docker Images](docker/using_docker_images.md) + [Using Docker Build](docker/using_docker_build.md) + [Using Variables](variables/README.md) ++ [Using SSH keys](ssh_keys/README.md) + +### Languages + ++ [Testing PHP](languages/php.md) + +### Services + ++ [Using MySQL](services/mysql.md) ++ [Using PostgreSQL](services/postgres.md) ++ [Using Redis](services/redis.md) ++ [Using Other Services](docker/using_docker_images.html#how-to-use-other-images-as-services) ### Examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 1feae62b1c7..2f0ca19cd0f 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -42,7 +42,44 @@ So, **to access your database service you have to connect to host: `mysql` inste ### How to use other images as services? You are not limited to have only database services. -You can hand modify `config.toml` to add any image as service found at [Docker Hub](https://registry.hub.docker.com/). +You can add the services to `.gitlab-ci.yml` or hand modify the `config.toml`. +You can use any image as service found at [Docker Hub](https://registry.hub.docker.com/). + +### Define image and services from `.gitlab-ci.yml` +You can simply define image or list services that you want to use for the build time. +``` +image: ruby:2.2 +services: + - postgres:9.3 +before_install: + - bundle install + +test: + script: + - bundle exec rake spec +``` + +It's possible to define image and service per-job: +``` +before_install: + - bundle install + +test:2.1: + image: ruby:2.1 + services: + - postgres:9.3 + script: + - bundle exec rake spec + +test:2.2: + image: ruby:2.2 + services: + - postgres:9.4 + script: + - bundle exec rake spec +``` + +### Define image and services in `config.toml` Look for `[runners.docker]` section: ``` [runners.docker] @@ -50,13 +87,16 @@ Look for `[runners.docker]` section: services = ["mysql:latest", "postgres:latest"] ``` +The image and services defined these way will be added to all builds run by that runner. + +### Accessing the services For example you need `wordpress` instance to test some API integration with `Wordpress`. -You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). -This is image that have fully preconfigured `wordpress` and have `MySQL` server built-in: +You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). + ``` -[runners.docker] - image = "ruby:2.1" - services = ["mysql:latest", "postgres:latest", "tutum/wordpress:latest"] +# .gitlab-ci.yml +services: +- tutum/wordpress:latest ``` Next time when you run your application the `tutum/wordpress` will be started @@ -64,7 +104,7 @@ and you will have access to it from your build container under hostname: `tutum_ Alias hostname for the service is made from the image name: 1. Everything after `:` is stripped, -2. '/' is replaced with `__`. +2. '/' is replaced to `__`. ### Configuring services Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. @@ -99,67 +139,6 @@ or README page for any other Docker image. **Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** -### Overwrite image and services -It's possible to overwrite `docker-image` and specify services from `.gitlab-ci.yml`. -If you add to your YAML the `image` and the `services` these parameters -be used instead of the ones that were specified during runner's registration. -``` -image: ruby:2.2 -services: - - postgres:9.3 -before_install: - - bundle install - -test: - script: - - bundle exec rake spec -``` - -It's possible to define image and service per-job: -``` -before_install: - - bundle install - -test:2.1: - image: ruby:2.1 - services: - - postgres:9.3 - script: - - bundle exec rake spec - -test:2.2: - image: ruby:2.2 - services: - - postgres:9.4 - script: - - bundle exec rake spec -``` - -#### How to enable overwriting? -To enable overwriting you have to **enable it first** (it's disabled by default for security reasons). -You can do that by hand modifying runner configuration: `config.toml`. -Please go to section where is `[runners.docker]` definition for your runner. -Add `allowed_images` and `allowed_services` to specify what images are allowed to be picked from `.gitlab-ci.yml`: -``` -[runners.docker] - image = "ruby:2.1" - allowed_images = ["ruby:*", "python:*"] - allowed_services = ["mysql:*", "redis:*"] -``` -This enables you to use in your `.gitlab-ci.yml` any image that matches above wildcards. -You will be able to pick only `ruby` and `python` images. -The same rule can be applied to limit services. - -If you are courageous enough, you can make it fully open and accept everything: -``` -[runners.docker] - image = "ruby:2.1" - allowed_images = ["*", "*/*"] - allowed_services = ["*", "*/*"] -``` - -**It the feature is not enabled, or image isn't allowed the error message will be put into the build log.** - ### How Docker integration works 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. 1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). diff --git a/doc/ci/languages/README.md b/doc/ci/languages/README.md new file mode 100644 index 00000000000..375adf58d18 --- /dev/null +++ b/doc/ci/languages/README.md @@ -0,0 +1,3 @@ +### Languages + ++ [Testing PHP](php.md) diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md new file mode 100644 index 00000000000..e0589182003 --- /dev/null +++ b/doc/ci/languages/php.md @@ -0,0 +1,178 @@ +## Testing PHP projects + +This guide covers basic of building PHP projects. + +Is it possible to test PHP apps on any system. +However, it will require manual configuration. +The simplest is to use Docker executor as described below. + +### PHP projects on Docker executor +It's possible to official [PHP](https://hub.docker.com/_/php/) repositories on Docker Hub. +They allow to test PHP projects against different versions of the runtime. +However, they require additional configuration. + +To build PHP project you need to create valid `.gitlab-ci.yml` describing the build environment: +1. First you need to specify PHP image as described here: http://doc.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-image. To your `.gitlab-ci.yml` add: + + image: php:5.6 + +2. The official images are great, but they are lacking a few useful tools for testing. We need to install them first in build environment. Create `ci/docker_install.sh` file with following content: + + #!/bin/bash + + # We need to install dependencies only for Docker + [[ ! -e /.dockerinit ]] && exit 0 + + set -xe + + # Install git (the php image doesn't have it) which is required by composer + apt-get update -yqq + apt-get install git -yqq + + # Install phpunit, the tool that we will use for testing + curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar + chmod +x /usr/local/bin/phpunit + + # Install mysql driver + # Here you can install any other extension that you need + docker-php-ext-install pdo_mysql + +3. From your `.gitlab-ci.yml` run the created script: + + before_script: + - bash ci/docker_install.sh > /dev/null + +4. Now you can run your tests. Usually it will be `phpunit` with arguments: + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +5. Commit your files, and push them to GitLab to see if it works. With GitLab Runner 1.0 you can also test the changes locally. From your terminal execute: + + # Check using docker executor + gitlab-runner exec docker test:app + + # Check using shell executor + gitlab-runner exec shell test:app + +The final `.gitlab-ci.yml` should look similar to this: + + # Select image from https://hub.docker.com/_/php/ + image: php:5.6 + + before_script: + # Install dependencies + - ci/docker_install.sh > /dev/null + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Test against different PHP versions in Docker builds + +You can also test against multiple version of PHP runtime: + + before_script: + # Install dependencies + - ci/docker_install.sh > /dev/null + + # We test PHP5.6 + test:5.6: + image: php:5.6 + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + + # We test PHP7.0 + test:7.0: + image: php:7.0 + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Custom PHP configuration in Docker builds + +You can customise your PHP environment by putting your .ini file into `/usr/local/etc/php/conf.d/`: + + before_script: + - cp my_php.ini /usr/local/etc/php/conf.d/test.ini + +### Test PHP projects using Shell + +Shell executor runs your builds in terminal session of your server. Thus in order to test your projects you need to have all dependencies installed as root. + +1. Install PHP dependencies: + + sudo apt-get update -qy + sudo apt-get install phpunit php5-mysql -y + + This will install the PHP version available for your distribution. + +2. Now you can run your tests. Usually it will be `phpunit` with arguments: + + test:app: + script: + - phpunit --configuration phpunit_myapp.xml --coverage-text + +#### Test against different PHP versions in Shell builds + +The [phpenv](https://github.com/phpenv/phpenv) allows you to easily manage different PHP with they own configs. +This is specially usefull when testing PHP project with Shell executor. + +Login as `gitlab-runner` user and follow [the installation guide](https://github.com/phpenv/phpenv#installation). + +Using phpenv also allows to easily configure PHP environment with: `phpenv config-add my_config.ini`. + +#### Install custom extensions + +Since we have pretty bare installation of our PHP environment you may need some extensions that are not present on your installation. + +To install additional extensions simply execute.: + + pecl install + + It's not advised to add this to the `.gitlab-ci.yml`. + You should execute this command once, only to setup the build environment. + +### Extend your tests + +#### Using atoum + +Instead of PHPUnit, you can use any other tool to run unit tests. For example [atoum](https://github.com/atoum/atoum): + + before_script: + - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar + + test:atoum: + script: + - php mageekguy.atoum.phar + +#### Using Composer + +Majority of the PHP projects use Composer for managing the packages. +It's very simple to execute the Composer before running your tests. +To your `.gitlab-ci.yml` add: + + # The composer stores all downloaded packages in vendor/ + # Remove it if you committed the vendor/ directory + cache: + paths: + - vendor/ + + before_script: + # Install composer dependencies + - curl -sS https://getcomposer.org/installer | php + - php composer.phar install + +### Access private packages / dependencies + +You need to configure [the SSH keys](../ssh_keys/README.md) in order to checkout the repositories. + +### Use databases or other services + +Please checkout the docs about configuring [the CI services](../services/README.md). + +### Example project + +You maybe interested in our [Example Project](https://gitlab.com/gitlab-examples/php) that runs on [GitLab.com](https://gitlab.com) using our publically available shared runners. + +Want to hack it? Simply fork it, commit and push changes. Within a few moments the changes will be picked and rebuilt by public runners. diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md new file mode 100644 index 00000000000..0550e9435a3 --- /dev/null +++ b/doc/ci/services/README.md @@ -0,0 +1,6 @@ +## GitLab CI Services + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) ++ [Using Other Services](../docker/using_docker_images.html#how-to-use-other-images-as-services) diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md new file mode 100644 index 00000000000..df36ebaf7d4 --- /dev/null +++ b/doc/ci/services/docker-services.md @@ -0,0 +1,5 @@ +## GitLab CI Services + ++ [Using MySQL](mysql.md) ++ [Using PostgreSQL](postgres.md) ++ [Using Redis](redis.md) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md new file mode 100644 index 00000000000..3155af6b3e1 --- /dev/null +++ b/doc/ci/services/mysql.md @@ -0,0 +1,72 @@ +## Using MySQL + +It's possible to use MySQL database test your apps during builds. + +### Use MySQL with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - mysql + + variables: + # Configure mysql service (https://hub.docker.com/_/mysql/) + MYSQL_DATABASE: hello_world_test + MYSQL_ROOT_PASSWORD: mysql + +2. Configure your application to use the database: + + Host: mysql + User: root + Password: mysql + Database: hello_world_test + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/mysql/). For example: `mysql:5.5`. + +Example: https://gitlab.com/gitlab-examples/mysql/blob/master/.gitlab-ci.yml + +### Use MySQL with Shell executor + +It's possible to use MySQL on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the MySQL server: + + sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev + + # Pick a MySQL root password (can be anything), type it and press enter + # Retype the MySQL root password and press enter + +2. Create an user: + + mysql -u root -p + + # Create a user which will be used by your apps + # do not type the 'mysql>', this is part of the prompt + # change $password in the command below to a real password you pick + mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; + + # Ensure you can use the InnoDB engine which is necessary to support long indexes + # If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting "innodb = off" + mysql> SET storage_engine=INNODB; + + # Create the database + mysql> CREATE DATABASE IF NOT EXISTS `hello_world_test` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; + + # Grant necessary permissions on the database + mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `hello_world_test`.* TO 'runner'@'localhost'; + + # Quit the database session + mysql> \q + +3. Try to connect to database: + + sudo -u gitlab-runner -H mysql -u runner -p -D hello_world_test + +4. Configure your application to use the database: + + Host: localhost + User: runner + Password: $password + Database: hello_world_test diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md new file mode 100644 index 00000000000..e57f8c5944a --- /dev/null +++ b/doc/ci/services/postgres.md @@ -0,0 +1,70 @@ +## Using PostgreSQL + +It's possible to use PostgreSQL database test your apps during builds. + +### Use PostgreSQL with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - postgres + + variables: + # Configure postgres service (https://hub.docker.com/_/postgres/) + POSTGRES_DB: hello_world_test + POSTGRES_USER: postgres + POSTGRES_PASSWORD: "" + +2. Configure your application to use the database: + + Host: postgres + User: postgres + Password: postgres + Database: hello_world_test + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/postgres/). For example: `postgres:9.3`. + +Example: https://gitlab.com/gitlab-examples/postgres/blob/master/.gitlab-ci.yml + +### Use PostgreSQL with Shell executor + +It's possible to use PostgreSQL on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the PostgreSQL server: + + sudo apt-get install -y postgresql postgresql-client libpq-dev + +2. Create an user: + + # Install the database packages + sudo apt-get install -y postgresql postgresql-client libpq-dev + + # Login to PostgreSQL + sudo -u postgres psql -d template1 + + # Create a user for runner + # Do not type the 'template1=#', this is part of the prompt + template1=# CREATE USER runner CREATEDB; + + # Create the database & grant all privileges on database + template1=# CREATE DATABASE hello_world_test OWNER runner; + + # Quit the database session + template1=# \q + +3. Try to connect to database: + + # Try connecting to the new database with the new user + sudo -u gitlab-runner -H psql -d hello_world_test + + # Quit the database session + hello_world_test> \q + +4. Configure your application to use the database: + + Host: localhost + User: runner + Password: + Database: hello_world_test diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md new file mode 100644 index 00000000000..523634a457e --- /dev/null +++ b/doc/ci/services/redis.md @@ -0,0 +1,40 @@ +## Using Redis + +It's possible to use Redis database test your apps during builds. + +### Use Redis with Docker executor + +If you are using our Docker integration you basically have everything already. + +1. Add this to your `.gitlab-ci.yml`: + + services: + - redis + +2. Configure your application to use the database: + + Host: redis + +3. You can also use any other available on [DockerHub](https://hub.docker.com/_/redis/). For example: `redis:2.6`. + +Example: https://gitlab.com/gitlab-examples/redis/blob/master/.gitlab-ci.yml + +### Use Redis with Shell executor + +It's possible to use Redis on manually configured servers that are using GitLab Runner with Shell executor. + +1. First install the Redis server: + + sudo apt-get install redis-server + +2. Try to connect to the server: + + # Try connecting the the Redis server + sudo -u gitlab-runner -H redis-cli + + # Quit the session + 127.0.0.1:6379> quit + +4. Configure your application to use the database: + + Host: localhost diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md new file mode 100644 index 00000000000..515194e5f5e --- /dev/null +++ b/doc/ci/ssh_keys/README.md @@ -0,0 +1,114 @@ +# Using SSH keys + +GitLab currently doesn't have built-in support for SSH keys in build environment. + +The SSH keys can be useful when: +1. You want to checkout internal submodules, +2. You want to download private packages using your package manager (ie. bundler), +3. You want to deploy your app (ex. to Heroku or own server), +4. You want to execute ssh commands from build environment on remote server, +5. You want to rsync files from your build to remote server. + +If anyone of the above holds true, then you most likely need SSH key. + +There are two possibilities to add SSH keys to build environment. + +## Inject keys in your build environment +The most widely supported is to inject SSH key into your build environment by extending your .gitlab-ci.yml. +This is the universal solution which works with any type of executor (docker, shell, etc.). + +### How it works? +1. We create a new SSH private key with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen). +2. We add the private key as the Secure Variable to project. +3. We run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during build to load the private key. + +The example [.gitlab-ci.yml](https://gitlab.com/gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml) looks like this. + +### Make it work? +1. First, go to terminal and generate a new SSH key: +```bash +$ ssh-keygen -t rsa -f my_key + +Generating public/private rsa key pair. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in my_key. +Your public key has been saved in my_key.pub. +The key fingerprint is: +SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint +The key's randomart image is: ++---[RSA 2048]----+ +|=*. .o++*= | +|..= +o..o. | +|.+++o + + . | +|+o*=.. + + | +|o+.=. . S | +|*.o .E . | +|o*o . . | +|.o.. | +| . | ++----[SHA256]-----+ +``` + +2. Create a new **Secure Variable** in your project settings on GitLab and name it: `SSH_PRIVATE_KEY`. + +3. Copy the content of `my_key` and paste it as a **Value** of **SSH_PRIVATE_KEY**. + +4. Next you need to modify your `.gitlab-ci.yml` and at the top of the file add: +``` +before_script: +# install ssh-agent (it is required for Docker, change apt-get to yum if you use CentOS-based image) +- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + +# run ssh-agent (in build environment) +- eval $(ssh-agent -s) + +# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store +- ssh-add <(echo "$SSH_PRIVATE_KEY") + +# for Docker builds disable host key checking, by adding that you are suspectible to man-in-the-middle attack +- mkdir -p ~/.ssh +- '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` +``` + +5. Add the public key from `my_key.pub` to services that you want to have an access from build. + +6. If your builds are run using `shell` executor, you may need to login to server and execute the `ssh ` to store the fingerprint of remote server. + +## SSH keys when using Shell executor +If use `shell`, not `docker` it can be easier to have the SSH key. + +We can generate the SSH key for the machine that holds `gitlab-runner` and use that key for all projects that are run on this machine. + +1. First, login to server that runs your builds. + +2. From terminal login as `gitlab-runner` user and generate the SSH private key: +```bash +$ ssh-keygen -t rsa +Generating public/private rsa key pair. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in ~/.ssh/id_rsa. +Your public key has been saved in ~/.ssh/id_rsa.pub. +The key fingerprint is: +SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint +The key's randomart image is: ++---[RSA 2048]----+ +|=*. .o++*= | +|..= +o..o. | +|.+++o + + . | +|+o*=.. + + | +|o+.=. . S | +|*.o .E . | +|o*o . . | +|.o.. | +| . | ++----[SHA256]-----+ +``` + +3. Add the public key from `~/.ssh/id_rsa.pub` to services that you want to have an access from build. + +4. Try to login for the first time and accept fingerprint: +```bash +ssh Date: Wed, 2 Dec 2015 17:47:18 +0200 Subject: Fix heading --- doc/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index a7025f7af10..f85a464a4f0 100644 --- a/doc/README.md +++ b/doc/README.md @@ -32,7 +32,7 @@ + [Testing PHP](ci/languages/php.md) -## CI Services +### CI Services + [Using MySQL](ci/services/mysql.md) + [Using PostgreSQL](ci/services/postgres.md) -- cgit v1.2.1 From dcdc49fe429c0e8a762f1a411e1d5c0a7294c91f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 2 Dec 2015 19:24:15 +0200 Subject: Use .md instead of .html --- doc/README.md | 2 +- doc/ci/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index f85a464a4f0..a3098094210 100644 --- a/doc/README.md +++ b/doc/README.md @@ -37,7 +37,7 @@ + [Using MySQL](ci/services/mysql.md) + [Using PostgreSQL](ci/services/postgres.md) + [Using Redis](ci/services/redis.md) -+ [Using Other Services](ci/docker/using_docker_images.html#how-to-use-other-images-as-services) ++ [Using Other Services](ci/docker/using_docker_images.md#how-to-use-other-images-as-services) ### CI Examples diff --git a/doc/ci/README.md b/doc/ci/README.md index ae921b6f988..5d9d7a81db3 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -20,7 +20,7 @@ + [Using MySQL](services/mysql.md) + [Using PostgreSQL](services/postgres.md) + [Using Redis](services/redis.md) -+ [Using Other Services](docker/using_docker_images.html#how-to-use-other-images-as-services) ++ [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services) ### Examples -- cgit v1.2.1 From 05267b64d224c19f1b6843a715f666aea2bd106a Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 00:36:45 +0200 Subject: Clean up using_docker_images.md --- doc/ci/docker/using_docker_images.md | 268 ++++++++++++++++++++++++----------- 1 file changed, 185 insertions(+), 83 deletions(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 2f0ca19cd0f..6551d47b697 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,16 +1,25 @@ # Using Docker Images -GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. -Docker is an open-source project that allows to use predefined images to run applications -in independent "containers" that are run within a single Linux instance. -[Docker Hub](https://registry.hub.docker.com/) have rich database of built images that can be used to build applications. +GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. -Docker when used with GitLab CI runs each build in separate and isolated container using predefined image and always from scratch. -It makes it easier to have simple and reproducible build environment that can also be run on your workstation. -This allows you to test all commands from your shell, rather than having to test them on a CI server. +Docker is an open-source project that allows to use predefined images to run +applications in independent "containers" that are run within a single Linux +instance. [Docker Hub][hub] has a rich database of prebuilt images that can be +used to test and build your applications. -### Register Docker runner -To use GitLab Runner with Docker you need to register new runner to use `docker` executor: +Docker, when used with GitLab CI, runs each build in a separate and isolated +container using the predefined image that is set up in `.gitlab-ci.yml`. The +container is always euphemeral which means you get only one container per build. + +This makes it easier to have a simple and reproducible build environment that +can also run on your workstation. The added benefit is that you can test all +the commands that we will explore later from your shell, rather than having to +test them on a CI server. + +### Register docker runner + +To use GitLab Runner with docker you need to register a new runner to use the +`docker` executor: ```bash gitlab-ci-multi-runner register \ @@ -23,45 +32,70 @@ gitlab-ci-multi-runner register \ --docker-mysql latest ``` -**The registered runner will use `ruby:2.1` image and will run two services (`postgres:latest` and `mysql:latest`) that will be accessible for time of the build.** +The registered runner will use the `ruby:2.1` docker image and will run two +services, `postgres:latest` and `mysql:latest`, both of which will be +accessible during the build process. + +### What is image + +The `image` keyword is the name of the docker image that is present in the +local Docker Engine (list all images with `docker images`) or any image that +can be found at [Docker Hub][hub]. For more information about images and Docker +Hub please read the [Docker Fundamentals][] documentation. + +In short, with `image` we refer to the docker image, which will be used to +create a container on which your build will run. + +### What is service -### What is image? -The image is the name of any repository that is present in local Docker Engine or any repository that can be found at [Docker Hub](https://registry.hub.docker.com/). -For more information about the image and Docker Hub please read the [Docker Fundamentals](https://docs.docker.com/introduction/understanding-docker/). +The `service` keyword defines just another docker image that is run during +your build and is linked to the docker image that the `image` keyword defines. +This allows you to access the service image during build time. -### What is service? -Service is just another image that is run for time of your build and is linked to your build. This allows you to access the service image during build time. -The service image can run any application, but most common use case is to run some database container, ie.: `mysql`. -It's easier and faster to use existing image, run it as additional container than install `mysql` every time project is built. +The service image can run any application, but the most common use case is to +run a database container, eg. `mysql`. It's easier and faster to use an +existing image and run it as an additional container than install `mysql` every +time the project is built. -#### How is service linked to the build? -There's good document that describes how Docker linking works: [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). -To summarize: if you add `mysql` as service to your application, the image will be used to create container that is linked to build container. -The service container for MySQL will be accessible under hostname `mysql`. -So, **to access your database service you have to connect to host: `mysql` instead of socket or `localhost`**. +#### How is service linked to the build -### How to use other images as services? -You are not limited to have only database services. -You can add the services to `.gitlab-ci.yml` or hand modify the `config.toml`. -You can use any image as service found at [Docker Hub](https://registry.hub.docker.com/). +To better undestand how the container linking works, read +[Linking containers together](https://docs.docker.com/userguide/dockerlinks/). + +To summarize, if you add `mysql` as service to your application, the image will +then be used to create a container that is linked to the build container. + +The service container for MySQL will be accessible under the hostname `mysql`. +So, in order to access your database service you have to connect to the host +named `mysql` instead of a socket or `localhost`. + +### How to use other images as services + +You are not limited to have only database services. You can add as many +services you need to `.gitlab-ci.yml` or manually modify `config.toml`. +Any image found at [Docker Hub][hub] can be used as a service. ### Define image and services from `.gitlab-ci.yml` -You can simply define image or list services that you want to use for the build time. -``` + +You can simply define an image that will be used for all jobs and a list of +services that you want to use during build time. + +```yaml image: ruby:2.2 services: - postgres:9.3 -before_install: +before_script: - bundle install - + test: script: - bundle exec rake spec ``` -It's possible to define image and service per-job: -``` -before_install: +It is also possible to define different images and services per job: + +```yaml +before_script: - bundle install test:2.1: @@ -80,68 +114,106 @@ test:2.2: ``` ### Define image and services in `config.toml` -Look for `[runners.docker]` section: + +Look for the `[runners.docker]` section: + ``` +... + [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] + +... ``` -The image and services defined these way will be added to all builds run by that runner. +The image and services defined this way will be added to all builds run by +that runner. ### Accessing the services -For example you need `wordpress` instance to test some API integration with `Wordpress`. -You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). -``` -# .gitlab-ci.yml +Let's say that you need a Wordpress instance to test some API integration with +your application. + +You can then use for example the [tutum/wordpress][] image in your +`.gitlab-ci.yml`: + +```yaml +... + services: - tutum/wordpress:latest + +... ``` -Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum__wordpress`. +When the build is run, `tutum/wordpress` will be started and you will have +access to it from your build container under the hostname `tutum_wordpress`. + +The alias hostname for the service is made from the image name following these +rules: -Alias hostname for the service is made from the image name: -1. Everything after `:` is stripped, -2. '/' is replaced to `__`. +1. Everything after `:` is stripped +2. Backslash (`/`) is replaced with double underscores (`__`) ### Configuring services -Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. -GitLab Runner 0.5.0 and up passes all YAML-defined variables to created service containers. +Many services accept environment variables which allow you to easily change +database names or set account names depending on the environment. -1. To configure database name for [postgres](https://registry.hub.docker.com/u/library/postgres/) service, -you need to set POSTGRES_DB. +GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created +service containers. - ```yaml - services: - - postgres - - variables: - POSTGRES_DB: gitlab - ``` +For all possible configuration variables check the documentation of each image +provided in their corresponding Docker hub page. -1. To use [mysql](https://registry.hub.docker.com/u/library/mysql/) service with empty password for time of build, -you need to set MYSQL_ALLOW_EMPTY_PASSWORD. +*Note: All variables will be passed to all service containers. It's not designed + to distinguish which variable should go where.* - ```yaml - services: - - mysql - - variables: - MYSQL_ALLOW_EMPTY_PASSWORD: "yes" - ``` +#### PostgreSQL service example -For other possible configuration variables check the -https://registry.hub.docker.com/u/library/mysql/ or https://registry.hub.docker.com/u/library/postgres/ -or README page for any other Docker image. +To configure the database name for [postgres][postgres-hub] service, you need +to set the `POSTGRES_DB` variable: -**Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** +```yaml +... + +services: +- postgres + +variables: + POSTGRES_DB: gitlab + +... +``` + +For a real example visit . + +#### MySQL service example + +To use the [mysql][mysql-hub] service with an empty password during the time of +build, you need to set the `MYSQL_ALLOW_EMPTY_PASSWORD` variable: + +```yaml +... + +services: +- mysql + +variables: + MYSQL_ALLOW_EMPTY_PASSWORD: "yes" + +... +``` ### How Docker integration works + +Below is a high level overview of the steps performed by docker during build +time. + 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. -1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). +1. Create cache container to store all volumes as defined in `config.toml` and + `Dockerfile` of build image (`ruby:2.1` as in above example). 1. Create build container and link any service container to build container. 1. Start build container and send build script to the container. 1. Run build script. @@ -151,32 +223,62 @@ or README page for any other Docker image. 1. Remove build container and all created service containers. ### How to debug a build locally -1. Create a file with build script: + +*Note: The following commands are run without root privileges. You should be +able to run docker with your regular user account.* + +First start with creating a file named `build script`: + ```bash -$ cat < build_script +cat < build_script git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner cd /builds/gitlab-org/gitlab-ci-multi-runner -make <- or any other build step +make EOF ``` -1. Create service containers: +Here we use as an example the GitLab Runner repository which contains a +Makefile, so running `make` will execute the commands defined in the Makefile. +Your mileage may vary, so instead of `make` you could run the command which +is specific to your project. + +Then create some service containers: + ``` -$ docker run -d -n service-mysql mysql:latest -$ docker run -d -n service-postgres postgres:latest +docker run -d -n service-mysql mysql:latest +docker run -d -n service-postgres postgres:latest ``` -This will create two service containers (MySQL and PostgreSQL). -1. Create a build container and execute script in its context: +This will create two service containers, named `service-mysql` and +`service-postgres` which use the latest MySQL and PostgreSQL images +respectively. They will both run in the background (`-d`). + +Finally, create a build container by executing the `build_script` file we +created earlier: + ``` -$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script +docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` -This will create build container that has two service containers linked. -The build_script is piped using STDIN to bash interpreter which executes the build script in container. -1. At the end remove all containers: +The above command will create a container named `build` that is spawned from +the `ruby:2.1` image and has two services linked to it. The `build_script` is +piped using STDIN to the bash interpreter which in turn executes the +`build_script` in the `build` container. + +When you finish testing and no longer need the containers, you can remove them +with: + ``` docker rm -f -v build service-mysql service-postgres ``` -This will forcefully (the `-f` switch) remove build container and service containers -and all volumes (the `-v` switch) that were created with the container creation. + +This will forcefully (`-f`) remove the `build` container, the two service +containers as well as all volumes (`-v`) that were created with the container +creation. + +[Docker Fundamentals]: https://docs.docker.com/engine/introduction/understanding-docker/ +[hub]: https://hub.docker.com/ +[linking-containers]: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/ +[tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ +[postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ +[mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ -- cgit v1.2.1 From 223a02757909b406c1d5ebe42ca4bc3340318a2e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:43:06 +0200 Subject: Bring back removed heading and point to other section --- doc/ci/docker/using_docker_images.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 6551d47b697..63cfa436333 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -69,6 +69,10 @@ The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host named `mysql` instead of a socket or `localhost`. +### Overwrite image and services + +See the section below. + ### How to use other images as services You are not limited to have only database services. You can add as many -- cgit v1.2.1 From 71519d650d0317785f86ed935508d1dd5966cd6e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:43:29 +0200 Subject: Fix wrong example --- doc/ci/docker/using_docker_images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 63cfa436333..488479418b1 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -152,7 +152,7 @@ services: ``` When the build is run, `tutum/wordpress` will be started and you will have -access to it from your build container under the hostname `tutum_wordpress`. +access to it from your build container under the hostname `tutum__wordpress`. The alias hostname for the service is made from the image name following these rules: -- cgit v1.2.1 From bb75dfe38b481607952234212a15d29cd17b7cef Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 15:46:27 +0200 Subject: Add intro about languages --- doc/ci/languages/README.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc') diff --git a/doc/ci/languages/README.md b/doc/ci/languages/README.md index 375adf58d18..54b2343e08b 100644 --- a/doc/ci/languages/README.md +++ b/doc/ci/languages/README.md @@ -1,3 +1,7 @@ ### Languages +This is a list of languages you can test with GitLab CI. Each section has +comprehensive documentation and comes with a test repository hosted on +GitLab.com + + [Testing PHP](php.md) -- cgit v1.2.1 From a08cc70232b1c5071f199106a582c77ba9541a83 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 18:13:19 +0200 Subject: Clean up PHP CI example [ci skip] --- doc/ci/languages/php.md | 325 +++++++++++++++++++++++++++++++----------------- 1 file changed, 211 insertions(+), 114 deletions(-) (limited to 'doc') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index e0589182003..c71ae40786e 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -1,178 +1,275 @@ -## Testing PHP projects +# Testing PHP projects -This guide covers basic of building PHP projects. +This guide covers basic building instructions for PHP projects. -Is it possible to test PHP apps on any system. -However, it will require manual configuration. -The simplest is to use Docker executor as described below. +There are covered two cases: testing using the Docker executor and testing +using the Shell executor. -### PHP projects on Docker executor -It's possible to official [PHP](https://hub.docker.com/_/php/) repositories on Docker Hub. -They allow to test PHP projects against different versions of the runtime. -However, they require additional configuration. +## Test PHP projects using the Docker executor -To build PHP project you need to create valid `.gitlab-ci.yml` describing the build environment: -1. First you need to specify PHP image as described here: http://doc.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-image. To your `.gitlab-ci.yml` add: +While it is possible to test PHP apps on any system, this would require manual +configuration from the developer. To overcome this we will be using the +official [PHP docker image][php-hub] that can be found in Docker Hub. - image: php:5.6 +This will allow us to test PHP projects against different versions of PHP. +However, not everything is plug 'n' play, you still need to onfigure some +things manually. -2. The official images are great, but they are lacking a few useful tools for testing. We need to install them first in build environment. Create `ci/docker_install.sh` file with following content: +As with every build, you need to create a valid `.gitlab-ci.yml` describing the +build environment. - #!/bin/bash +Let's first specify the PHP image that will be used for the build process +(you can read more about what an image means in the Runner's lingo reading +about [Using Docker images](../docker/using_docker_images.md#what-is-image)). - # We need to install dependencies only for Docker - [[ ! -e /.dockerinit ]] && exit 0 +Start by adding the image to your `.gitlab-ci.yml`: - set -xe +```yaml +image: php:5.6 +``` - # Install git (the php image doesn't have it) which is required by composer - apt-get update -yqq - apt-get install git -yqq +The official images are great, but they lack a few useful tools for testing. +We need to first prepare the build environment. A way to overcome this is to +create a script which installs all prerequisites prior the actual testing is +done. - # Install phpunit, the tool that we will use for testing - curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar - chmod +x /usr/local/bin/phpunit +Let's create a `ci/docker_install.sh` file in the root directory of our +repository with the following content: - # Install mysql driver - # Here you can install any other extension that you need - docker-php-ext-install pdo_mysql +```bash +#!/bin/bash -3. From your `.gitlab-ci.yml` run the created script: +# We need to install dependencies only for Docker +[[ ! -e /.dockerinit ]] && exit 0 - before_script: - - bash ci/docker_install.sh > /dev/null +set -xe -4. Now you can run your tests. Usually it will be `phpunit` with arguments: +# Install git (the php image doesn't have it) which is required by composer +apt-get update -yqq +apt-get install git -yqq - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +# Install phpunit, the tool that we will use for testing +curl -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar +chmod +x /usr/local/bin/phpunit -5. Commit your files, and push them to GitLab to see if it works. With GitLab Runner 1.0 you can also test the changes locally. From your terminal execute: +# Install mysql driver +# Here you can install any other extension that you need +docker-php-ext-install pdo_mysql +``` - # Check using docker executor - gitlab-runner exec docker test:app +You might wonder what `docker-php-ext-install` is. In short, it is a script +provided by the official php docker image that you can use to easilly install +extensions. For more information read the the documentation at +. - # Check using shell executor - gitlab-runner exec shell test:app +Now that we created the script that contains all prerequisites for our build +environment, let's add it in `.gitlab-ci.yml`: + +```yaml +... + +before_script: +- bash ci/docker_install.sh > /dev/null + +... +``` + +Last step, run the actual tests using `phpunit`: + +```yaml +... + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml + +... +``` + +Finally, commit your files and push them to GitLab to see your build succeeding +(or failing). The final `.gitlab-ci.yml` should look similar to this: - # Select image from https://hub.docker.com/_/php/ - image: php:5.6 +```yaml +# Select image from https://hub.docker.com/_/php/ +image: php:5.6 + +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Test against different PHP versions in Docker builds + +Testing against multiple versions of PHP is super easy. Just add another job +with a different docker image version and the runner will do the rest: + +```yaml +before_script: +# Install dependencies +- ci/docker_install.sh > /dev/null + +# We test PHP5.6 +test:5.6: + image: php:5.6 + script: + - phpunit --configuration phpunit_myapp.xml + +# We test PHP7.0 (good luck with that) +test:7.0: + image: php:7.0 + script: + - phpunit --configuration phpunit_myapp.xml +``` + +### Custom PHP configuration in Docker builds + +There are times where you will need to customise your PHP environment by +putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose +add a `before_script` action: + +```yaml +before_script: +- cp my_php.ini /usr/local/etc/php/conf.d/test.ini +``` - before_script: - # Install dependencies - - ci/docker_install.sh > /dev/null +Of course, `my_php.ini` must be present in the root directory of your repository. - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +## Test PHP projects using the Shell executor -#### Test against different PHP versions in Docker builds +The shell executor runs your builds in a terminal session on your server. +Thus, in order to test your projects you first need to make sure that all +dependencies are installed. -You can also test against multiple version of PHP runtime: +For example, in a VM running Debian 8 we first update the cache, then we +install `phpunit` and `php5-mysql`: - before_script: - # Install dependencies - - ci/docker_install.sh > /dev/null +```bash +sudo apt-get update -y +sudo apt-get install -y phpunit php5-mysql +``` - # We test PHP5.6 - test:5.6: - image: php:5.6 - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +Next, add the following snippet to your `.gitlab-ci.yml`: - # We test PHP7.0 - test:7.0: - image: php:7.0 - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +```yaml +test:app: + script: + - phpunit --configuration phpunit_myapp.xml +``` -#### Custom PHP configuration in Docker builds +Finally, push to GitLab and let the tests begin! -You can customise your PHP environment by putting your .ini file into `/usr/local/etc/php/conf.d/`: +### Test against different PHP versions in Shell builds - before_script: - - cp my_php.ini /usr/local/etc/php/conf.d/test.ini +The [phpenv][] project allows you to easily manage different versions of PHP +each with its own config. This is specially usefull when testing PHP projects +with the Shell executor. -### Test PHP projects using Shell +You will have to install it on your build machine under the `gitlab-runner` +user following [the upstream installation guide][phpenv-installation]. -Shell executor runs your builds in terminal session of your server. Thus in order to test your projects you need to have all dependencies installed as root. +Using phpenv also allows to easily configure the PHP environment with: -1. Install PHP dependencies: +``` +phpenv config-add my_config.ini +``` - sudo apt-get update -qy - sudo apt-get install phpunit php5-mysql -y +### Install custom extensions - This will install the PHP version available for your distribution. +Since this is a pretty bare installation of the PHP environment, you may need +some extensions that are not currently present on the build machine. -2. Now you can run your tests. Usually it will be `phpunit` with arguments: +To install additional extensions simply execute: - test:app: - script: - - phpunit --configuration phpunit_myapp.xml --coverage-text +```bash +pecl install +``` -#### Test against different PHP versions in Shell builds +It's not advised to add this to `.gitlab-ci.yml`. You should execute this +command once, only to setup the build environment. -The [phpenv](https://github.com/phpenv/phpenv) allows you to easily manage different PHP with they own configs. -This is specially usefull when testing PHP project with Shell executor. +## Extend your tests -Login as `gitlab-runner` user and follow [the installation guide](https://github.com/phpenv/phpenv#installation). +### Using atoum -Using phpenv also allows to easily configure PHP environment with: `phpenv config-add my_config.ini`. +Instead of PHPUnit, you can use any other tool to run unit tests. For example +you can use [atoum](https://github.com/atoum/atoum): -#### Install custom extensions +```yaml +before_script: +- wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar -Since we have pretty bare installation of our PHP environment you may need some extensions that are not present on your installation. +test:atoum: + script: + - php mageekguy.atoum.phar +``` -To install additional extensions simply execute.: +### Using Composer - pecl install +The majority of the PHP projects use Composer for managing their PHP packages. +In order to execute Composer before running your tests, simply add the +following in your `.gitlab-ci.yml`: - It's not advised to add this to the `.gitlab-ci.yml`. - You should execute this command once, only to setup the build environment. +```yaml +... -### Extend your tests +# Composer stores all downloaded packages in the vendor/ directory. +# Do not use the following if the vendor/ directory is commited to +# your git repository. +cache: + paths: + - vendor/ -#### Using atoum +before_script: +# Install composer dependencies +- curl -sS https://getcomposer.org/installer | php +- php composer.phar install -Instead of PHPUnit, you can use any other tool to run unit tests. For example [atoum](https://github.com/atoum/atoum): +... +``` - before_script: - - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar +## Access private packages / dependencies - test:atoum: - script: - - php mageekguy.atoum.phar +If your test suite needs to access a private repository, you need to configure +[the SSH keys](../ssh_keys/README.md) in order to be able to clone it. -#### Using Composer +## Use databases or other services -Majority of the PHP projects use Composer for managing the packages. -It's very simple to execute the Composer before running your tests. -To your `.gitlab-ci.yml` add: +Most of the time you will need a running database in order for your tests to +run. If you are using the Docker executor you can leverage Docker's ability to +connect to other containers. In GitLab Runner lingo, this can be achieved by +defining a `service`. - # The composer stores all downloaded packages in vendor/ - # Remove it if you committed the vendor/ directory - cache: - paths: - - vendor/ +This functionality is covered in [the CI services](../services/README.md) +documentation. - before_script: - # Install composer dependencies - - curl -sS https://getcomposer.org/installer | php - - php composer.phar install +## Testing things locally -### Access private packages / dependencies +With GitLab Runner 1.0 you can also test any changes locally. From your +terminal execute: -You need to configure [the SSH keys](../ssh_keys/README.md) in order to checkout the repositories. +```bash +# Check using docker executor +gitlab-runner exec docker test:app -### Use databases or other services +# Check using shell executor +gitlab-runner exec shell test:app +``` -Please checkout the docs about configuring [the CI services](../services/README.md). +## Example project -### Example project +We have set up an [Example PHP Project](https://gitlab.com/gitlab-examples/php) +for your convenience that runs on [GitLab.com](https://gitlab.com) using our +publicly available [shared runners](../runners/README.md). -You maybe interested in our [Example Project](https://gitlab.com/gitlab-examples/php) that runs on [GitLab.com](https://gitlab.com) using our publically available shared runners. +Want to hack it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. -Want to hack it? Simply fork it, commit and push changes. Within a few moments the changes will be picked and rebuilt by public runners. +[php-hub]: https://hub.docker.com/_/php/ +[phpenv]: https://github.com/phpenv/phpenv +[phpenv-installation]: https://github.com/phpenv/phpenv#installation -- cgit v1.2.1 From 9b8babb601ac6bdb3b7301b017a8ce20c5dc4814 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 3 Dec 2015 21:18:34 +0200 Subject: Use link instead of connect to be more Docker friendly --- doc/ci/languages/php.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index c71ae40786e..589253cfc4c 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -242,7 +242,7 @@ If your test suite needs to access a private repository, you need to configure Most of the time you will need a running database in order for your tests to run. If you are using the Docker executor you can leverage Docker's ability to -connect to other containers. In GitLab Runner lingo, this can be achieved by +link to other containers. In GitLab Runner lingo, this can be achieved by defining a `service`. This functionality is covered in [the CI services](../services/README.md) -- cgit v1.2.1 From d13d43aca9c486365fbe6eab8d30cb1e005c3f61 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 4 Dec 2015 11:46:08 +0200 Subject: Clean up CI ssh keys docs [ci skip] --- doc/ci/ssh_keys/README.md | 167 ++++++++++++++++++++++------------------------ 1 file changed, 81 insertions(+), 86 deletions(-) (limited to 'doc') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 515194e5f5e..210f9c3e849 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,114 +1,109 @@ # Using SSH keys -GitLab currently doesn't have built-in support for SSH keys in build environment. +GitLab currently doesn't have built-in support for managing SSH keys in a build +environment. The SSH keys can be useful when: -1. You want to checkout internal submodules, -2. You want to download private packages using your package manager (ie. bundler), -3. You want to deploy your app (ex. to Heroku or own server), -4. You want to execute ssh commands from build environment on remote server, -5. You want to rsync files from your build to remote server. -If anyone of the above holds true, then you most likely need SSH key. +1. You want to checkout internal submodules +2. You want to download private packages using your package manager (eg. bundler) +3. You want to deploy your application to eg. Heroku or your own server +4. You want to execute SSH commands from the build server to the remote server +5. You want to rsync files from your build server to the remote server -There are two possibilities to add SSH keys to build environment. +If anything of the above rings a bell, then you most likely need an SSH key. -## Inject keys in your build environment -The most widely supported is to inject SSH key into your build environment by extending your .gitlab-ci.yml. -This is the universal solution which works with any type of executor (docker, shell, etc.). +## Inject keys in your build server -### How it works? -1. We create a new SSH private key with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen). -2. We add the private key as the Secure Variable to project. -3. We run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during build to load the private key. +The most widely supported method is to inject an SSH key into your build +environment by extending your `.gitlab-ci.yml`. -The example [.gitlab-ci.yml](https://gitlab.com/gitlab-examples/ssh-private-key/blob/master/.gitlab-ci.yml) looks like this. +This is the universal solution which works with any type of executor +(docker, shell, etc.). -### Make it work? -1. First, go to terminal and generate a new SSH key: -```bash -$ ssh-keygen -t rsa -f my_key - -Generating public/private rsa key pair. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in my_key. -Your public key has been saved in my_key.pub. -The key fingerprint is: -SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint -The key's randomart image is: -+---[RSA 2048]----+ -|=*. .o++*= | -|..= +o..o. | -|.+++o + + . | -|+o*=.. + + | -|o+.=. . S | -|*.o .E . | -|o*o . . | -|.o.. | -| . | -+----[SHA256]-----+ -``` +### How it works + +1. Create a new SSH key pair with [ssh-keygen][] +2. Add the private key as a **Secret Variable** to the project +3. Run the [ssh-agent][] during build to load the private key. + +## SSH keys when using the Docker executor -2. Create a new **Secure Variable** in your project settings on GitLab and name it: `SSH_PRIVATE_KEY`. +You will first need to create an SSH key pair. For more information, follow the +instructions to [generate an SSH key](../ssh/README.md). -3. Copy the content of `my_key` and paste it as a **Value** of **SSH_PRIVATE_KEY**. +Then, create a new **Secret Variable** in your project settings on GitLab +following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` +and in the **Value** field paste the content of your _private_ key that you +created earlier. + +Next you need to modify your `.gitlab-ci.yml` with a `before_script` action. +Add it to the top: -4. Next you need to modify your `.gitlab-ci.yml` and at the top of the file add: ``` before_script: -# install ssh-agent (it is required for Docker, change apt-get to yum if you use CentOS-based image) -- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + # Install ssh-agent if not already installed, it is required by Docker. + # (change apt-get to yum if you use a CentOS-based image) + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + + # Run ssh-agent (inside the build environment) + - eval $(ssh-agent -s) + + # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + - ssh-add <(echo "$SSH_PRIVATE_KEY") + + # For Docker builds disable host key checking. Be aware that by adding that + # you are suspectible to man-in-the-middle attacks. + # WARNING: Use this only with the Docker executor, if you use it with shell + # you will overwrite your user's SSH config. + - mkdir -p ~/.ssh + - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` +``` -# run ssh-agent (in build environment) -- eval $(ssh-agent -s) +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). -# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store -- ssh-add <(echo "$SSH_PRIVATE_KEY") +That's it! You can now have access to private servers or repositories in your +build environment. -# for Docker builds disable host key checking, by adding that you are suspectible to man-in-the-middle attack -- mkdir -p ~/.ssh -- '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` -``` +## SSH keys when using the Shell executor -5. Add the public key from `my_key.pub` to services that you want to have an access from build. +If you are using the Shell executor and not Docker, it is easier to set up an +SSH key. -6. If your builds are run using `shell` executor, you may need to login to server and execute the `ssh ` to store the fingerprint of remote server. +You can generate the SSH key from the machine that GitLab Runner is installed +on, and use that key for all projects that are run on this machine. -## SSH keys when using Shell executor -If use `shell`, not `docker` it can be easier to have the SSH key. +First, you need to login to the server that runs your builds. -We can generate the SSH key for the machine that holds `gitlab-runner` and use that key for all projects that are run on this machine. +Then from the terminal login as the `gitlab-runner` user and generate the SSH +key pair as described in the [SSH keys documentation](../ssh/README.md). -1. First, login to server that runs your builds. +As a final step, add the _public_ key from the one you created earlier to the +services that you want to have an access to from within the build environment. +If you are accessing a private GitLab repository you need to add it as a +[deploy key](../ssh/README.md#deploy-keys). + +Once done, try to login to the remote server in order to accept the fingerprint: -2. From terminal login as `gitlab-runner` user and generate the SSH private key: ```bash -$ ssh-keygen -t rsa -Generating public/private rsa key pair. -Enter passphrase (empty for no passphrase): -Enter same passphrase again: -Your identification has been saved in ~/.ssh/id_rsa. -Your public key has been saved in ~/.ssh/id_rsa.pub. -The key fingerprint is: -SHA256:tBJEfyJUGTMNmPCiPg4UHywHs67MxlM2iEBAlI/W+TY fingeprint -The key's randomart image is: -+---[RSA 2048]----+ -|=*. .o++*= | -|..= +o..o. | -|.+++o + + . | -|+o*=.. + + | -|o+.=. . S | -|*.o .E . | -|o*o . . | -|.o.. | -| . | -+----[SHA256]-----+ +ssh ``` -3. Add the public key from `~/.ssh/id_rsa.pub` to services that you want to have an access from build. +For accessing repositories on GitLab.com, the `` would be +`git@gitlab.com`. -4. Try to login for the first time and accept fingerprint: -```bash -ssh Date: Fri, 4 Dec 2015 11:47:02 +0200 Subject: Move markdown link to the bottom [ci skip] --- doc/ci/languages/php.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index 589253cfc4c..60079c090d3 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -263,13 +263,14 @@ gitlab-runner exec shell test:app ## Example project -We have set up an [Example PHP Project](https://gitlab.com/gitlab-examples/php) -for your convenience that runs on [GitLab.com](https://gitlab.com) using our -publicly available [shared runners](../runners/README.md). +We have set up an [Example PHP Project][php-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). -Want to hack it? Simply fork it, commit and push your changes. Within a few +Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. [php-hub]: https://hub.docker.com/_/php/ [phpenv]: https://github.com/phpenv/phpenv [phpenv-installation]: https://github.com/phpenv/phpenv#installation +[php-example-repo]: https://gitlab.com/gitlab-examples/php -- cgit v1.2.1 From 4fc9e6944e363e3167fede07c0d6898ff6c7e19f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 4 Dec 2015 11:47:50 +0200 Subject: Add an intro to CI services documentation [ci skip] --- doc/ci/services/README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 0550e9435a3..0856b7679c2 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,6 +1,9 @@ ## GitLab CI Services +GitLab CI uses the `service` keyword to define what docker containers should be +linked with your base image. Below is a list of examples you may use. + + [Using MySQL](mysql.md) + [Using PostgreSQL](postgres.md) + [Using Redis](redis.md) -+ [Using Other Services](../docker/using_docker_images.html#how-to-use-other-images-as-services) ++ [Using Other Services](../docker/using_docker_images.md#how-to-use-other-images-as-services) -- cgit v1.2.1 From 4236861df03ff86fc1b2bb8a6a0835b0f3a03244 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 7 Dec 2015 09:39:14 +0200 Subject: Clean up Redis CI service example [ci skip] --- doc/ci/services/redis.md | 72 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 50 insertions(+), 22 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 523634a457e..b0d04f95408 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -1,40 +1,68 @@ -## Using Redis +# Using Redis -It's possible to use Redis database test your apps during builds. +As many applications depend on Redis as their key-value store, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. -### Use Redis with Docker executor +## Use Redis with Docker executor -If you are using our Docker integration you basically have everything already. +If you are using our Docker integration you basically have everything set up +already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - redis +```yaml +services: + - redis:latest +``` -2. Configure your application to use the database: +Then you need to configure your application to use the Redis database, for +example: - Host: redis +```bash +Host: redis +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/redis/). For example: `redis:2.6`. +And that's it. Redis will now be available to be used within your testing +framework. -Example: https://gitlab.com/gitlab-examples/redis/blob/master/.gitlab-ci.yml +If you want to use any other version of Redis, check the available versions +on [Docker Hub](https://hub.docker.com/_/redis/). -### Use Redis with Shell executor +## Use Redis with Shell executor -It's possible to use Redis on manually configured servers that are using GitLab Runner with Shell executor. +Redis can also be used on manually configured servers that are using GitLab +Runner with the Shell executor. -1. First install the Redis server: +In your build machine install the Redis server: - sudo apt-get install redis-server +```bash +sudo apt-get install redis-server +``` -2. Try to connect to the server: +Verify that you can connect to the server with the `gitlab-runner` user: - # Try connecting the the Redis server - sudo -u gitlab-runner -H redis-cli +```bash +# Try connecting the the Redis server +sudo -u gitlab-runner -H redis-cli - # Quit the session - 127.0.0.1:6379> quit +# Quit the session +127.0.0.1:6379> quit +``` -4. Configure your application to use the database: +Finally, configure your application to use the database, for example: - Host: localhost +```bash +Host: localhost +``` + +## Example project + +We have set up an [Example Redis Project][redis-example-repo] for your convenience +that runs on [GitLab.com](https://gitlab.com) using our publicly available +[shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[redis-example-repo]: https://gitlab.com/gitlab-examples/redis -- cgit v1.2.1 From 149c934a1d38662358b48cdf844430de7845d1c4 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 15:59:41 +0200 Subject: More cleanup on Redis example --- doc/ci/services/redis.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index b0d04f95408..e6add57ca76 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -4,10 +4,10 @@ As many applications depend on Redis as their key-value store, you will eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. -## Use Redis with Docker executor +## Use Redis with the Docker executor -If you are using our Docker integration you basically have everything set up -already. +If you are using GitLab's Runner Docker integration you basically have +everything set up already. First, in your `.gitlab-ci.yml` add: @@ -29,7 +29,7 @@ framework. If you want to use any other version of Redis, check the available versions on [Docker Hub](https://hub.docker.com/_/redis/). -## Use Redis with Shell executor +## Use Redis with the Shell executor Redis can also be used on manually configured servers that are using GitLab Runner with the Shell executor. -- cgit v1.2.1 From 06b86de996232b956129f2bb5dde4a7647f94f69 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 18:22:45 +0200 Subject: Clean up postgres CI example [ci skip] --- doc/ci/services/postgres.md | 116 +++++++++++++++++++++++++++----------------- 1 file changed, 71 insertions(+), 45 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index e57f8c5944a..f82a828756c 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -1,70 +1,96 @@ -## Using PostgreSQL +# Using PostgreSQL -It's possible to use PostgreSQL database test your apps during builds. +As many applications depend on PostgreSQL as their database, you will +eventually need it in order for your tests to run. Below you are guided how to +do this with the Docker and Shell executors of GitLab Runner. -### Use PostgreSQL with Docker executor +## Use PostgreSQL with the Docker executor -If you are using our Docker integration you basically have everything already. +If you are using GitLab's Runner with the Docker executor you basically have +everything set up already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - postgres +```yaml +services: + - postgres - variables: - # Configure postgres service (https://hub.docker.com/_/postgres/) - POSTGRES_DB: hello_world_test - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "" +variables: + POSTGRES_DB: nice_marmot + POSTGRES_USER: gitlab_runner + POSTGRES_PASSWORD: "" +``` -2. Configure your application to use the database: +And then configure your application to use PostgreSQL, for example: - Host: postgres - User: postgres - Password: postgres - Database: hello_world_test +```yaml +Host: localhost +User: gitlab_runner +Password: +Database: nice_marmot +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/postgres/). For example: `postgres:9.3`. +You can also use any other docker image available on [Docker Hub][hub-pg]. +For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`. -Example: https://gitlab.com/gitlab-examples/postgres/blob/master/.gitlab-ci.yml +The `postgres` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-pg]. -### Use PostgreSQL with Shell executor +## Use PostgreSQL with the Shell executor -It's possible to use PostgreSQL on manually configured servers that are using GitLab Runner with Shell executor. +You can also use PostgreSQL on manually configured servers that are using +GitLab Runner with the Shell executor. -1. First install the PostgreSQL server: +First install the PostgreSQL server: - sudo apt-get install -y postgresql postgresql-client libpq-dev +```bash +sudo apt-get install -y postgresql postgresql-client libpq-dev +``` -2. Create an user: +Then create a user: - # Install the database packages - sudo apt-get install -y postgresql postgresql-client libpq-dev +```bash +# Login to PostgreSQL +sudo -u postgres psql -d template1 - # Login to PostgreSQL - sudo -u postgres psql -d template1 +# Create a user for GitLab Runner that can create databases +# Do not type the 'template1=#', this is part of the prompt +template1=# CREATE USER gitlab_runner CREATEDB; - # Create a user for runner - # Do not type the 'template1=#', this is part of the prompt - template1=# CREATE USER runner CREATEDB; +# Create the database & grant all privileges on database +template1=# CREATE DATABASE nice_marmot OWNER gitlab_runner; - # Create the database & grant all privileges on database - template1=# CREATE DATABASE hello_world_test OWNER runner; +# Quit the database session +template1=# \q +``` - # Quit the database session - template1=# \q +Try to connect to database: -3. Try to connect to database: +```bash +# Try connecting to the new database with the new user +sudo -u gitlab_runner -H psql -d nice_marmot - # Try connecting to the new database with the new user - sudo -u gitlab-runner -H psql -d hello_world_test +# Quit the database session +nice_marmot> \q +``` - # Quit the database session - hello_world_test> \q +Finally, configure your application to use the database: -4. Configure your application to use the database: +```bash +Host: localhost +User: gitlab_runner +Password: +Database: nice_marmot +``` - Host: localhost - User: runner - Password: - Database: hello_world_test +## Example project + +We have set up an [Example PostgreSQL Project][postgres-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-pg]: https://hub.docker.com/_/postgres/ +[postgres-example-repo]: https://gitlab.com/gitlab-examples/postgres -- cgit v1.2.1 From bf5683f8892c4aefc4c996812ece6291b701dada Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Tue, 8 Dec 2015 09:47:42 -0600 Subject: Block LDAP user when they are no longer found in the LDAP server --- doc/integration/ldap.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/integration/ldap.md b/doc/integration/ldap.md index 7e2920b8865..845f588f913 100644 --- a/doc/integration/ldap.md +++ b/doc/integration/ldap.md @@ -13,6 +13,12 @@ An LDAP user who is allowed to change their email on the LDAP server can [take o We recommend against using GitLab LDAP integration if your LDAP users are allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on the LDAP server. +If a user is deleted from the LDAP server, they will be blocked in GitLab as well. +Users will be immediately blocked from logging in. However, there is an LDAP check +cache time of one hour. The means users that are already logged in or are using Git +over SSH will still be able to access GitLab for up to one hour. Manually block +the user in the GitLab Admin area to immediately block all access. + ## Configuring GitLab for LDAP integration To enable GitLab LDAP integration you need to add your LDAP server settings in `/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`. @@ -192,4 +198,4 @@ Not supported by GitLab's configuration options. When setting `method: ssl`, the underlying authentication method used by `omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with the LDAP server before any LDAP-protocol data is exchanged but no validation of -the LDAP server's SSL certificate is performed. \ No newline at end of file +the LDAP server's SSL certificate is performed. -- cgit v1.2.1 From 47e81da8c2746196ad0506d30720c775a538ebdb Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:11:11 +0200 Subject: Clean up mysql CI example --- doc/ci/services/mysql.md | 137 +++++++++++++++++++++++++++++++---------------- 1 file changed, 90 insertions(+), 47 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 3155af6b3e1..7f50b4d76c8 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -1,72 +1,115 @@ -## Using MySQL +# Using MySQL -It's possible to use MySQL database test your apps during builds. +As many applications depend on MySQL as their database, you will eventually +need it in order for your tests to run. Below you are guided how to do this +with the Docker and Shell executors of GitLab Runner. -### Use MySQL with Docker executor +## Use MySQL with the Docker executor -If you are using our Docker integration you basically have everything already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. -1. Add this to your `.gitlab-ci.yml`: +First, in your `.gitlab-ci.yml` add: - services: - - mysql +```yaml +services: + - mysql - variables: - # Configure mysql service (https://hub.docker.com/_/mysql/) - MYSQL_DATABASE: hello_world_test - MYSQL_ROOT_PASSWORD: mysql +variables: + # Configure mysql environment variables (https://hub.docker.com/_/mysql/) + MYSQL_DATABASE: el_duderino + MYSQL_ROOT_PASSWORD: mysql_strong_password +``` -2. Configure your application to use the database: +And then configure your application to use the database, for example: - Host: mysql - User: root - Password: mysql - Database: hello_world_test +```yaml +Host: localhost +User: root +Password: mysql_strong_password +Database: el_duderino +``` -3. You can also use any other available on [DockerHub](https://hub.docker.com/_/mysql/). For example: `mysql:5.5`. +You can also use any other docker image available on [Docker Hub][hub-mysql]. +For example, to use MySQL 5.5 the service becomes `mysql:5.5`. -Example: https://gitlab.com/gitlab-examples/mysql/blob/master/.gitlab-ci.yml +The `mysql` image can accept some environment variables. For more details +check the documentation on [Docker Hub][hub-mysql]. -### Use MySQL with Shell executor +## Use MySQL with the Shell executor -It's possible to use MySQL on manually configured servers that are using GitLab Runner with Shell executor. +You can also use MySQL on manually configured servers that are using +GitLab Runner with the Shell executor. -1. First install the MySQL server: - - sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev +First install the MySQL server: - # Pick a MySQL root password (can be anything), type it and press enter - # Retype the MySQL root password and press enter +```bash +sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev +``` -2. Create an user: +Pick a MySQL root password (can be anything), and type it twice when asked. - mysql -u root -p +*Note: As a security measure you can run `mysql_secure_installation` to +remove anonymous users, drop the test database and disable remote logins with +the root user.* - # Create a user which will be used by your apps - # do not type the 'mysql>', this is part of the prompt - # change $password in the command below to a real password you pick - mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; +The next step is to create a user, so login to MySQL as root: - # Ensure you can use the InnoDB engine which is necessary to support long indexes - # If this fails, check your MySQL config files (e.g. `/etc/mysql/*.cnf`, `/etc/mysql/conf.d/*`) for the setting "innodb = off" - mysql> SET storage_engine=INNODB; +```bash +mysql -u root -p +``` - # Create the database - mysql> CREATE DATABASE IF NOT EXISTS `hello_world_test` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. - # Grant necessary permissions on the database - mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `hello_world_test`.* TO 'runner'@'localhost'; +*Note: Do not type `mysql>`, this is part of the MySQL prompt.* - # Quit the database session - mysql> \q +```bash +mysql> CREATE USER 'runner'@'localhost' IDENTIFIED BY '$password'; +``` -3. Try to connect to database: +Create the database: - sudo -u gitlab-runner -H mysql -u runner -p -D hello_world_test +```bash +mysql> CREATE DATABASE IF NOT EXISTS `el_duderino` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_unicode_ci`; +``` -4. Configure your application to use the database: +Grant the necessary permissions on the database: - Host: localhost - User: runner - Password: $password - Database: hello_world_test +```bash +mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES ON `el_duderino`.* TO 'runner'@'localhost'; +``` + +If all went well you can now quit the database session: + +```bash +mysql> \q +``` + +Now, try to connect to the newly created database to check that everything is +in place: + +```bash +mysql -u runner -p -D el_duderino +``` + +As a final step, configure your application to use the database, for example: + +```bash +Host: localhost +User: runner +Password: $password +Database: el_duderino +``` + +## Example project + +We have set up an [Example MySQL Project][mysql-example-repo] for your +convenience that runs on [GitLab.com](https://gitlab.com) using our publicly +available [shared runners](../runners/README.md). + +Want to hack on it? Simply fork it, commit and push your changes. Within a few +moments the changes will be picked by a public runner and the build will begin. + +[hub-mysql]: https://hub.docker.com/_/mysql/ +[mysql-example-repo]: https://gitlab.com/gitlab-examples/mysql -- cgit v1.2.1 From 9dc91f46df377d6220928d6292dac73bc6bae295 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:11:27 +0200 Subject: Add link to runners doc [ci skip] --- doc/ci/services/postgres.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index f82a828756c..b8436f4f7ca 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -6,8 +6,8 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use PostgreSQL with the Docker executor -If you are using GitLab's Runner with the Docker executor you basically have -everything set up already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -21,7 +21,7 @@ variables: POSTGRES_PASSWORD: "" ``` -And then configure your application to use PostgreSQL, for example: +And then configure your application to use the database, for example: ```yaml Host: localhost -- cgit v1.2.1 From cf44dc510786775a1303384d858eb0a7bf33c34f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:35:08 +0200 Subject: Add note about the various phpenv tools --- doc/ci/languages/php.md | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'doc') diff --git a/doc/ci/languages/php.md b/doc/ci/languages/php.md index 60079c090d3..dacb67fa3ff 100644 --- a/doc/ci/languages/php.md +++ b/doc/ci/languages/php.md @@ -179,6 +179,14 @@ Using phpenv also allows to easily configure the PHP environment with: phpenv config-add my_config.ini ``` +*__Important note:__ It seems `phpenv/phpenv` + [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork + at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring + the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also + seems like a good alternative. Picking any of the mentioned tools will work + with the basic phpenv commands. Guiding you to choose the right phpenv is out + of the scope of this tutorial.* + ### Install custom extensions Since this is a pretty bare installation of the PHP environment, you may need -- cgit v1.2.1 From 4ea0f06470620064c5d23e28c778f83296aeb551 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:35:25 +0200 Subject: Fix wrong naming of services keyword [ci skip] --- doc/ci/services/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 0856b7679c2..1ebb0a4a250 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,6 +1,6 @@ ## GitLab CI Services -GitLab CI uses the `service` keyword to define what docker containers should be +GitLab CI uses the `services` keyword to define what docker containers should be linked with your base image. Below is a list of examples you may use. + [Using MySQL](mysql.md) -- cgit v1.2.1 From 0c22635aa2036ab3394c7e7bd99fd61ddd99c43c Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 21:41:56 +0200 Subject: More redis CI example clean up --- doc/ci/services/redis.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index e6add57ca76..b281e8f9f60 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -6,8 +6,8 @@ do this with the Docker and Shell executors of GitLab Runner. ## Use Redis with the Docker executor -If you are using GitLab's Runner Docker integration you basically have -everything set up already. +If you are using [GitLab Runner](../runners/README.md) with the Docker executor +you basically have everything set up already. First, in your `.gitlab-ci.yml` add: @@ -19,15 +19,15 @@ services: Then you need to configure your application to use the Redis database, for example: -```bash +```yaml Host: redis ``` And that's it. Redis will now be available to be used within your testing framework. -If you want to use any other version of Redis, check the available versions -on [Docker Hub](https://hub.docker.com/_/redis/). +You can also use any other docker image available on [Docker Hub][hub-redis]. +For example, to use Redis 2.8 the service becomes `redis:2.8`. ## Use Redis with the Shell executor @@ -52,7 +52,7 @@ sudo -u gitlab-runner -H redis-cli Finally, configure your application to use the database, for example: -```bash +```yaml Host: localhost ``` @@ -65,4 +65,5 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available Want to hack on it? Simply fork it, commit and push your changes. Within a few moments the changes will be picked by a public runner and the build will begin. +[hub-redis]: https://hub.docker.com/_/redis/ [redis-example-repo]: https://gitlab.com/gitlab-examples/redis -- cgit v1.2.1 From bbe652cdaf11dae7fa917bac1ffa4a02f2099026 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 22:26:41 +0200 Subject: More postgres CI service example cleanup --- doc/ci/services/postgres.md | 55 ++++++++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 20 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index b8436f4f7ca..db5be070ee7 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -13,11 +13,11 @@ First, in your `.gitlab-ci.yml` add: ```yaml services: - - postgres + - postgres:latest variables: POSTGRES_DB: nice_marmot - POSTGRES_USER: gitlab_runner + POSTGRES_USER: runner POSTGRES_PASSWORD: "" ``` @@ -25,7 +25,7 @@ And then configure your application to use the database, for example: ```yaml Host: localhost -User: gitlab_runner +User: runner Password: Database: nice_marmot ``` @@ -47,39 +47,54 @@ First install the PostgreSQL server: sudo apt-get install -y postgresql postgresql-client libpq-dev ``` -Then create a user: +The next step is to create a user, so login to PostgreSQL: ```bash -# Login to PostgreSQL sudo -u postgres psql -d template1 +``` -# Create a user for GitLab Runner that can create databases -# Do not type the 'template1=#', this is part of the prompt -template1=# CREATE USER gitlab_runner CREATEDB; +Then create a user (in our case `runner`) which will be used by your +application. Change `$password` in the command below to a real strong password. -# Create the database & grant all privileges on database -template1=# CREATE DATABASE nice_marmot OWNER gitlab_runner; +*__Note:__ Do not type `template1=#`, this is part of the PostgreSQL prompt.* -# Quit the database session -template1=# \q +```bash +template1=# CREATE USER runner WITH PASSWORD '$password' CREATEDB; ``` -Try to connect to database: +*__Note:__ Notice that we created the user with the privilege to be able to +create databases (`CREATEDB`). In the following steps we will create a database +explicitly for that user but having that privilege can be useful if in your +testing framework you have tools that drop and create databases.* + +Create the database and grant all privileges on it for the user `runner`: ```bash -# Try connecting to the new database with the new user -sudo -u gitlab_runner -H psql -d nice_marmot +template1=# CREATE DATABASE nice_marmot OWNER runner; +``` + +If all went well you can now quit the database session: -# Quit the database session -nice_marmot> \q +```bash +template1=# \q ``` -Finally, configure your application to use the database: +Now, try to connect to the newly created database with the user `runner` to +check that everything is in place. ```bash +psql -U runner -h localhost -d nice_marmot -W +``` + +*__Note:__ We are explicitly telling `psql` to connect to localhost in order +to use the md5 authentication. If you omit this step you will be denied access.* + +Finally, configure your application to use the database, for example: + +```yaml Host: localhost -User: gitlab_runner -Password: +User: runner +Password: $password Database: nice_marmot ``` -- cgit v1.2.1 From d55dcd11d59ebfadf49290746c5f30c2e4780c6b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 22:27:23 +0200 Subject: Use the latest docker image in mysql CI example [ci skip] --- doc/ci/services/mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 7f50b4d76c8..44bc613e6f7 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -13,7 +13,7 @@ First, in your `.gitlab-ci.yml` add: ```yaml services: - - mysql + - mysql:latest variables: # Configure mysql environment variables (https://hub.docker.com/_/mysql/) -- cgit v1.2.1 From b312edaf440b729dad02749259577f6e0589c061 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 23:29:49 +0200 Subject: More clean up to the CI example on how to use docker images [ci skip] --- doc/ci/docker/using_docker_images.md | 100 +++++++++++++---------------------- 1 file changed, 36 insertions(+), 64 deletions(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 488479418b1..8d4bd44053e 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,28 +1,29 @@ # Using Docker Images -GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. +GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use +[Docker Engine](https://www.docker.com/) to test and build any application. -Docker is an open-source project that allows to use predefined images to run -applications in independent "containers" that are run within a single Linux +Docker is an open-source project that allows you to use predefined images to +run applications in independent "containers" that are run within a single Linux instance. [Docker Hub][hub] has a rich database of prebuilt images that can be used to test and build your applications. Docker, when used with GitLab CI, runs each build in a separate and isolated -container using the predefined image that is set up in `.gitlab-ci.yml`. The -container is always euphemeral which means you get only one container per build. +container using the predefined image that is set up in +[`.gitlab-ci.yml`](../yaml/README.md). This makes it easier to have a simple and reproducible build environment that can also run on your workstation. The added benefit is that you can test all the commands that we will explore later from your shell, rather than having to -test them on a CI server. +test them on a dedicated CI server. -### Register docker runner +## Register docker runner To use GitLab Runner with docker you need to register a new runner to use the `docker` executor: ```bash -gitlab-ci-multi-runner register \ +gitlab-runner register \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ @@ -36,7 +37,7 @@ The registered runner will use the `ruby:2.1` docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. -### What is image +## What is image The `image` keyword is the name of the docker image that is present in the local Docker Engine (list all images with `docker images`) or any image that @@ -46,9 +47,9 @@ Hub please read the [Docker Fundamentals][] documentation. In short, with `image` we refer to the docker image, which will be used to create a container on which your build will run. -### What is service +## What is service -The `service` keyword defines just another docker image that is run during +The `services` keyword defines just another docker image that is run during your build and is linked to the docker image that the `image` keyword defines. This allows you to access the service image during build time. @@ -57,9 +58,12 @@ run a database container, eg. `mysql`. It's easier and faster to use an existing image and run it as an additional container than install `mysql` every time the project is built. -#### How is service linked to the build +You can see some widely used services examples in the relevant documentation of +[CI services examples](../services/README.md). -To better undestand how the container linking works, read +### How is service linked to the build + +To better understand how the container linking works, read [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). To summarize, if you add `mysql` as service to your application, the image will @@ -69,25 +73,27 @@ The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host named `mysql` instead of a socket or `localhost`. -### Overwrite image and services +## Overwrite image and services -See the section below. +See [How to use other images as services](#how-to-use-other-images-as-services). -### How to use other images as services +## How to use other images as services You are not limited to have only database services. You can add as many services you need to `.gitlab-ci.yml` or manually modify `config.toml`. Any image found at [Docker Hub][hub] can be used as a service. -### Define image and services from `.gitlab-ci.yml` +## Define image and services from `.gitlab-ci.yml` You can simply define an image that will be used for all jobs and a list of services that you want to use during build time. ```yaml image: ruby:2.2 + services: - postgres:9.3 + before_script: - bundle install @@ -117,24 +123,20 @@ test:2.2: - bundle exec rake spec ``` -### Define image and services in `config.toml` +## Define image and services in `config.toml` Look for the `[runners.docker]` section: ``` -... - [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] - -... ``` The image and services defined this way will be added to all builds run by that runner. -### Accessing the services +## Accessing the services Let's say that you need a Wordpress instance to test some API integration with your application. @@ -143,12 +145,8 @@ You can then use for example the [tutum/wordpress][] image in your `.gitlab-ci.yml`: ```yaml -... - services: - tutum/wordpress:latest - -... ``` When the build is run, `tutum/wordpress` will be started and you will have @@ -160,7 +158,7 @@ rules: 1. Everything after `:` is stripped 2. Backslash (`/`) is replaced with double underscores (`__`) -### Configuring services +## Configuring services Many services accept environment variables which allow you to easily change database names or set account names depending on the environment. @@ -171,46 +169,20 @@ service containers. For all possible configuration variables check the documentation of each image provided in their corresponding Docker hub page. -*Note: All variables will be passed to all service containers. It's not designed - to distinguish which variable should go where.* - -#### PostgreSQL service example - -To configure the database name for [postgres][postgres-hub] service, you need -to set the `POSTGRES_DB` variable: - -```yaml -... - -services: -- postgres - -variables: - POSTGRES_DB: gitlab +*Note: All variables will be passed to all services containers. It's not +designed to distinguish which variable should go where.* -... -``` - -For a real example visit . +### PostgreSQL service example -#### MySQL service example +See the specific documentation for +[using PostgreSQL as a service](../services/postgres.md). -To use the [mysql][mysql-hub] service with an empty password during the time of -build, you need to set the `MYSQL_ALLOW_EMPTY_PASSWORD` variable: +### MySQL service example -```yaml -... - -services: -- mysql - -variables: - MYSQL_ALLOW_EMPTY_PASSWORD: "yes" - -... -``` +See the specific documentation for +[using MySQL as a service](../services/mysql.md). -### How Docker integration works +## How Docker integration works Below is a high level overview of the steps performed by docker during build time. @@ -226,7 +198,7 @@ time. 1. Check exit status of build script. 1. Remove build container and all created service containers. -### How to debug a build locally +## How to debug a build locally *Note: The following commands are run without root privileges. You should be able to run docker with your regular user account.* -- cgit v1.2.1 From 1464a69c76cf492ad8b9674e24260e917dc7d2ef Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 8 Dec 2015 22:37:07 +0100 Subject: Tweak text of documentation --- doc/workflow/merge_when_build_succeeds.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/workflow/merge_when_build_succeeds.md b/doc/workflow/merge_when_build_succeeds.md index 3c055650c49..75e1fdff2b2 100644 --- a/doc/workflow/merge_when_build_succeeds.md +++ b/doc/workflow/merge_when_build_succeeds.md @@ -1,12 +1,12 @@ # Merge When Build Succeeds -When reviewing a merge request that looks ready to merge but still has one or more CI builds running, you can set it to be merged automatically when the build succeeds. This way, you don't have to wait for the build to finish and remember to merge the merge request then. +When reviewing a merge request that looks ready to merge but still has one or more CI builds running, you can set it to be merged automatically when all builds succeed. This way, you don't have to wait for the builds to finish and remember to merge the request manually. ![Enable](merge_when_build_succeeds/enable.png) -When you hit the "Merge When Build Succeeds" button, the status of the Merge Request will be updated to represent the impending merge. If you cannot wait for the build to succeed and want to build immediately, this option is available in the dropdown menu on the right of the main button. +When you hit the "Merge When Build Succeeds" button, the status of the merge request will be updated to represent the impending merge. If you cannot wait for the build to succeed and want to merge immediately, this option is available in the dropdown menu on the right of the main button. -Both team developers and the author of the merge request have the option to cancel the automatic merge when they find a reason it shouldn't be merged after all. +Both team developers and the author of the merge request have the option to cancel the automatic merge if they find a reason why it shouldn't be merged after all. ![Status](merge_when_build_succeeds/status.png) -- cgit v1.2.1 From 1e7156ed701945349bef484d35c7f53f2ee4474b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 8 Dec 2015 23:49:05 +0200 Subject: Use the name of the linked containers instead of localhost https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1917#note_2833978 --- doc/ci/services/mysql.md | 5 ++++- doc/ci/services/postgres.md | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 44bc613e6f7..c66d77122b2 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -24,12 +24,15 @@ variables: And then configure your application to use the database, for example: ```yaml -Host: localhost +Host: mysql User: root Password: mysql_strong_password Database: el_duderino ``` +If you are wondering why we used `mysql` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + You can also use any other docker image available on [Docker Hub][hub-mysql]. For example, to use MySQL 5.5 the service becomes `mysql:5.5`. diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index db5be070ee7..17d21dbda1c 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -24,12 +24,15 @@ variables: And then configure your application to use the database, for example: ```yaml -Host: localhost +Host: postgres User: runner Password: Database: nice_marmot ``` +If you are wondering why we used `postgres` for the `Host`, read more at +[How is service linked to the build](../docker/using_docker_images.md#how-is-service-linked-to-the-build). + You can also use any other docker image available on [Docker Hub][hub-pg]. For example, to use PostgreSQL 9.3 the service becomes `postgres:9.3`. -- cgit v1.2.1 From 7b0ac5b6b49675790491dcf8764ebfb17b93012c Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Tue, 8 Dec 2015 21:31:22 -0800 Subject: Remove default_branch from project API creation since an empty repository has no branches to start. Closes #3937 --- doc/api/projects.md | 1 - 1 file changed, 1 deletion(-) (limited to 'doc') diff --git a/doc/api/projects.md b/doc/api/projects.md index 42919a312ae..43a50a9a810 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -398,7 +398,6 @@ Parameters: - `user_id` (required) - user_id of owner - `name` (required) - new project name - `description` (optional) - short project description -- `default_branch` (optional) - 'master' by default - `issues_enabled` (optional) - `merge_requests_enabled` (optional) - `builds_enabled` (optional) -- cgit v1.2.1 From 64eaccf10a7ac70c3dc2f9565a638e9f45a8359c Mon Sep 17 00:00:00 2001 From: Felix Eckhofer Date: Wed, 9 Dec 2015 12:26:18 +0100 Subject: Update init script only once The init script is already being updated in section 6. --- doc/update/8.1-to-8.2.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index 7b228d6a22f..b08a79ca0aa 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -85,11 +85,10 @@ sudo -u git -H git checkout 0.4.2 sudo -u git -H make ``` -Update the GitLab init script and 'default' file. +Update the GitLab 'default' file. ``` cd /home/git/gitlab -sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab test -e /etc/default/gitlab && \ sudo sed -i.pre-8.2 's/^\([^=]*\)gitlab_git_http_server/\1gitlab_workhorse/' /etc/default/gitlab ``` -- cgit v1.2.1 From 63a1a581e937ff6d21e7e6ca4774b7907c6a0c1b Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Thu, 10 Dec 2015 10:24:08 -0600 Subject: Document file upload random uuid security --- doc/security/README.md | 3 ++- doc/security/user_file_uploads.md | 11 +++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) create mode 100644 doc/security/user_file_uploads.md (limited to 'doc') diff --git a/doc/security/README.md b/doc/security/README.md index 473f3632dcd..fba6013d9c1 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -4,4 +4,5 @@ - [Rack attack](rack_attack.md) - [Web Hooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) -- [Reset your root password](reset_root_password.md) \ No newline at end of file +- [Reset your root password](reset_root_password.md) +- [User File Uploads](user_file_uploads.md) diff --git a/doc/security/user_file_uploads.md b/doc/security/user_file_uploads.md new file mode 100644 index 00000000000..98493d33b00 --- /dev/null +++ b/doc/security/user_file_uploads.md @@ -0,0 +1,11 @@ +# User File Uploads + +Images attached to issues, merge requests or comments do not require authentication +to be viewed if someone knows the direct URL. This direct URL contains a random +32-character ID that prevents unauthorized people from guessing the URL to an +image containing sensitive information. We don't enable authentication because +these images need to be visible in the body of notification emails, which are +often read from email clients that are not authenticated with GitLab, like +Outlook, Apple Mail, or the Mail app on your mobile device. + +Note that non-image attachments do require authentication to be viewed. -- cgit v1.2.1 From e80e3f5372d6bcad1fbe04a85b3086bb66794828 Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Fri, 4 Dec 2015 12:55:23 +0100 Subject: Migrate CI::Project to Project --- doc/api/projects.md | 3 --- 1 file changed, 3 deletions(-) (limited to 'doc') diff --git a/doc/api/projects.md b/doc/api/projects.md index 43a50a9a810..1a524400627 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -32,7 +32,6 @@ Parameters: - `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` - `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` - `search` (optional) - Return list of authorized projects according to a search criteria -- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first ```json [ @@ -137,7 +136,6 @@ Parameters: - `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` - `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` - `search` (optional) - Return list of authorized projects according to a search criteria -- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first ### List ALL projects @@ -153,7 +151,6 @@ Parameters: - `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` - `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` - `search` (optional) - Return list of authorized projects according to a search criteria -- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first ### Get single project -- cgit v1.2.1 From baa97175abd25eb566d7273bc188a6459639abf3 Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Fri, 11 Dec 2015 11:09:34 -0600 Subject: Clarify cache behavior --- doc/ci/yaml/README.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3dbf1afc7a9..7e2edb945da 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -113,6 +113,9 @@ The YAML-defined variables are also set to all created service containers, thus ### cache `cache` is used to specify list of files and directories which should be cached between builds. +Caches are stored according to the branch/ref and the job name. Caches are not +currently shared between different job names or between branches/refs. This means +caching will benefit you if you push subsequent commits to an existing feature branch. **The global setting allows to specify default cached files for all jobs.** -- cgit v1.2.1 From e6e27f03d77fa57b82f9189f45be7eec1ffe37ac Mon Sep 17 00:00:00 2001 From: Ben Bodenmiller Date: Fri, 11 Dec 2015 23:16:37 +0000 Subject: add details on how to change saml button label --- doc/integration/saml.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 1b8c28dd0f4..1632e42f701 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -38,7 +38,8 @@ First configure SAML 2.0 support in GitLab, then register the GitLab application idp_sso_target_url: 'https://login.example.com/idp', issuer: 'https://gitlab.example.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' - } + }, + "label" => "Company Login" # optional label for SAML login button, defaults to "Saml" } ] ``` @@ -79,4 +80,4 @@ On the sign in page there should now be a SAML button below the regular sign in If you see a "500 error" in GitLab when you are redirected back from the SAML sign in page, this likely indicates that GitLab could not get the email address for the SAML user. -Make sure the IdP provides a claim containing the user's email address, using claim name 'email' or 'mail'. The email will be used to automatically generate the GitLab username. +Make sure the IdP provides a claim containing the user's email address, using claim name 'email' or 'mail'. The email will be used to automatically generate the GitLab username. \ No newline at end of file -- cgit v1.2.1 From 3efae53bd79db118463bfaeceb209bc91f63bd0b Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Fri, 11 Dec 2015 23:17:36 -0800 Subject: Add open_issues_count to project API This is needed to support Huboard and a generally useful value. --- doc/api/projects.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/api/projects.md b/doc/api/projects.md index 43a50a9a810..15956fe6df2 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -59,6 +59,7 @@ Parameters: "path": "diaspora-client", "path_with_namespace": "diaspora/diaspora-client", "issues_enabled": true, + "open_issues_count": 1, "merge_requests_enabled": true, "builds_enabled": true, "wiki_enabled": true, @@ -101,6 +102,7 @@ Parameters: "path": "puppet", "path_with_namespace": "brightbox/puppet", "issues_enabled": true, + "open_issues_count": 1, "merge_requests_enabled": true, "builds_enabled": true, "wiki_enabled": true, @@ -192,6 +194,7 @@ Parameters: "path": "diaspora-project-site", "path_with_namespace": "diaspora/diaspora-project-site", "issues_enabled": true, + "open_issues_count": 1, "merge_requests_enabled": true, "builds_enabled": true, "wiki_enabled": true, -- cgit v1.2.1 From 9c5d8079a367ac24d04466f03f6b9abf5c333f59 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Tue, 24 Nov 2015 08:28:18 -0800 Subject: Bump Redis requirement to 2.8 for Sidekiq 4 requirements Closes #3649 [ci skip] --- doc/install/installation.md | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 618391e16d2..0a19a060a9a 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -62,7 +62,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server redis-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake nodejs + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake nodejs If you want to use Kerberos for user authentication, then install libkrb5-dev: @@ -174,7 +174,23 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da ## 6. Redis - sudo apt-get install redis-server +As of this writing, most Debian/Ubuntu distributions ship with Redis 2.2 or +2.4. GitLab requires at least Redis 2.8. If your platform doesn't provide +this, the following instructions cover building and installing Redis from +scratch. + +Ubuntu users [can also use a PPA](https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server) +to install a recent version of Redis. + + # Build Redis + wget http://download.redis.io/releases/redis-2.8.23.tar.gz + tar xzf redis-2.8.23.tar.gz + cd redis-2.8.23 + make + + # Install Redis + cd utils + sudo ./install_server.sh # Configure redis to use sockets sudo cp /etc/redis/redis.conf /etc/redis/redis.conf.orig @@ -197,7 +213,7 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da fi # Activate the changes to redis.conf - sudo service redis-server restart + sudo service redis_6379 start # Add git to the redis group sudo usermod -aG redis git -- cgit v1.2.1 From 41f5bb035bd648c6fb0023bfbfb5ae40551cb180 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 14 Dec 2015 00:15:29 -0800 Subject: Rename mention of gitlab-git-http-server to gitlab-workhorse [ci skip] Closes https://github.com/gitlabhq/gitlabhq/issues/9904 --- doc/update/8.1-to-8.2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index b08a79ca0aa..46dfa2232b4 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -142,7 +142,7 @@ git diff origin/8-1-stable:lib/support/nginx/gitlab origin/8-2-stable:lib/suppor If you are using Apache instead of NGINX please see the updated [Apache templates]. Also note that because Apache does not support upstreams behind Unix sockets you -will need to let gitlab-git-http-server listen on a TCP port. You can do this +will need to let gitlab-workhorse listen on a TCP port. You can do this via [/etc/default/gitlab]. [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache -- cgit v1.2.1 From 58bb985fe0e3aff148191d2a2a2c0246b60c2ebe Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Sat, 12 Dec 2015 13:59:53 -0800 Subject: Add upgrade guide for 8.2 to 8.3 [ci skip] --- doc/update/8.2-to-8.3.md | 200 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 doc/update/8.2-to-8.3.md (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md new file mode 100644 index 00000000000..071ed02494a --- /dev/null +++ b/doc/update/8.2-to-8.3.md @@ -0,0 +1,200 @@ +# From 8.2 to 8.3 + +**NOTE:** GitLab 8.0 introduced several significant changes related to +installation and configuration which *are not duplicated here*. Be sure you're +already running a working version of at least 8.0 before proceeding with this +guide. + +### 0. Double-check your Git version + +**This notice applies only to /usr/local/bin/git** + +If you compiled Git from source on your GitLab server then please double-check +that you are using a version that protects against CVE-2014-9390. For six +months after this vulnerability became known the GitLab installation guide +still contained instructions that would install the outdated, 'vulnerable' Git +version 2.1.2. + +Run the following command to get your current Git version: + +```sh +/usr/local/bin/git --version +``` + +If you see 'No such file or directory' then you did not install Git according +to the outdated instructions from the GitLab installation guide and you can go +to the next step 'Stop server' below. + +If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4, +v2.2.1 or newer. You can use the [instructions in the GitLab source +installation +guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) +to install a newer version of Git. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-3-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-3-stable-ee +``` + +### 4. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch +sudo -u git -H git checkout v2.6.8 +``` + +### 5. Replace gitlab-git-http-server with gitlab-workhorse + +Install and compile gitlab-workhorse. This requires [Go +1.5](https://golang.org/dl) which should already be on your system +from GitLab 8.1. + +```bash +cd /home/git +sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git +cd gitlab-workhorse +sudo -u git -H git checkout 0.4.2 +sudo -u git -H make +``` + +Update the GitLab init script and 'default' file. + +``` +cd /home/git/gitlab +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +test -e /etc/default/gitlab && \ + sudo sed -i.pre-8.2 's/^\([^=]*\)gitlab_git_http_server/\1gitlab_workhorse/' /etc/default/gitlab +``` + +Make sure that you also update your **NGINX configuration** to use +the new gitlab-workhorse.socket file. + +### 6. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +# Update init.d script +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +### 7. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-2-stable:config/gitlab.yml.example origin/8-3-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +View changes between the previous recommended Nginx configuration and the +current one: + +```sh +# For HTTPS configurations +git diff origin/8-2-stable:lib/support/nginx/gitlab-ssl origin/8-3-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-2-stable:lib/support/nginx/gitlab origin/8-3-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-git-http-server listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/lib/support/init.d/gitlab.default.example#L34 + + +### 8. Use Redis v2.8.0+ + +Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but +GitLab 8.3 uses Sidekiq 4.0, which requires Redis 2.8. You can check your Redis version +with the following command: + + redis-cli info | grep redis_version + +If you need to upgrade, see the [installation guide for Redis](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#6-redis). + +### 9. Start application + + sudo service gitlab start + sudo service nginx restart + +### 10. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.2) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.1 to 8.2](8.1-to-8.2.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +## Troubleshooting + +### "You appear to have cloned an empty repository." + +See the [7.14 to 8.0 update guide](7.14-to-8.0.md#troubleshooting). -- cgit v1.2.1 From 5bca9ec7508414ccba03f29ddca149ea93e62f45 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 14 Dec 2015 00:17:10 -0800 Subject: Rename gitlab-git-http-server mention with gitlab-workhorse [ci skip] --- doc/update/8.2-to-8.3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 071ed02494a..563754ff4f6 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -143,7 +143,7 @@ git diff origin/8-2-stable:lib/support/nginx/gitlab origin/8-3-stable:lib/suppor If you are using Apache instead of NGINX please see the updated [Apache templates]. Also note that because Apache does not support upstreams behind Unix sockets you -will need to let gitlab-git-http-server listen on a TCP port. You can do this +will need to let gitlab-workhorse listen on a TCP port. You can do this via [/etc/default/gitlab]. [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache -- cgit v1.2.1 From b9b35012de171fe03d0be2c3adac6f219bb6927a Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 14 Dec 2015 09:22:42 -0800 Subject: Update gitlab-workhorse and remove mention of gitlab-git-http-server [ci skip] --- doc/update/8.2-to-8.3.md | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 563754ff4f6..f1000ef4650 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -67,36 +67,23 @@ sudo -u git -H git checkout 8-3-stable-ee ```bash cd /home/git/gitlab-shell -sudo -u git -H git fetch +sudo -u git -H git fetch --all sudo -u git -H git checkout v2.6.8 ``` -### 5. Replace gitlab-git-http-server with gitlab-workhorse +### 5. Update gitlab-workhorse Install and compile gitlab-workhorse. This requires [Go 1.5](https://golang.org/dl) which should already be on your system from GitLab 8.1. ```bash -cd /home/git -sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git -cd gitlab-workhorse -sudo -u git -H git checkout 0.4.2 +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch --all +sudo -u git -H git checkout 0.4.3 sudo -u git -H make ``` -Update the GitLab init script and 'default' file. - -``` -cd /home/git/gitlab -sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab -test -e /etc/default/gitlab && \ - sudo sed -i.pre-8.2 's/^\([^=]*\)gitlab_git_http_server/\1gitlab_workhorse/' /etc/default/gitlab -``` - -Make sure that you also update your **NGINX configuration** to use -the new gitlab-workhorse.socket file. - ### 6. Install libs, migrations, etc. ```bash -- cgit v1.2.1 From 739ce7883aaaf2652c7da985254bdec60cb15c51 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 14 Dec 2015 12:28:17 -0500 Subject: Prepare Installation doc for 8.3.0-rc1 [ci skip] --- doc/install/installation.md | 73 ++++++++++++++++++++++++--------------------- 1 file changed, 39 insertions(+), 34 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 0a19a060a9a..f8116a8a31c 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -38,6 +38,7 @@ The GitLab installation consists of setting up the following components: 1. Packages / Dependencies 1. Ruby +1. Go 1. System Users 1. Database 1. Redis @@ -175,48 +176,52 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da ## 6. Redis As of this writing, most Debian/Ubuntu distributions ship with Redis 2.2 or -2.4. GitLab requires at least Redis 2.8. If your platform doesn't provide -this, the following instructions cover building and installing Redis from -scratch. +2.4. GitLab requires at least Redis 2.8. -Ubuntu users [can also use a PPA](https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server) +Ubuntu users [can use a PPA](https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server) to install a recent version of Redis. - # Build Redis - wget http://download.redis.io/releases/redis-2.8.23.tar.gz - tar xzf redis-2.8.23.tar.gz - cd redis-2.8.23 - make +The following instructions cover building and installing Redis from scratch: + +```sh +# Build Redis +wget http://download.redis.io/releases/redis-2.8.23.tar.gz +tar xzf redis-2.8.23.tar.gz +cd redis-2.8.23 +make + +# Install Redis +cd utils +sudo ./install_server.sh + +# Configure redis to use sockets +sudo cp /etc/redis/redis.conf /etc/redis/redis.conf.orig - # Install Redis - cd utils - sudo ./install_server.sh +# Disable Redis listening on TCP by setting 'port' to 0 +sed 's/^port .*/port 0/' /etc/redis/redis.conf.orig | sudo tee /etc/redis/redis.conf - # Configure redis to use sockets - sudo cp /etc/redis/redis.conf /etc/redis/redis.conf.orig +# Enable Redis socket for default Debian / Ubuntu path +echo 'unixsocket /var/run/redis/redis.sock' | sudo tee -a /etc/redis/redis.conf - # Disable Redis listening on TCP by setting 'port' to 0 - sed 's/^port .*/port 0/' /etc/redis/redis.conf.orig | sudo tee /etc/redis/redis.conf +# Grant permission to the socket to all members of the redis group +echo 'unixsocketperm 770' | sudo tee -a /etc/redis/redis.conf - # Enable Redis socket for default Debian / Ubuntu path - echo 'unixsocket /var/run/redis/redis.sock' | sudo tee -a /etc/redis/redis.conf - # Grant permission to the socket to all members of the redis group - echo 'unixsocketperm 770' | sudo tee -a /etc/redis/redis.conf +# Create the directory which contains the socket +mkdir /var/run/redis +chown redis:redis /var/run/redis +chmod 755 /var/run/redis - # Create the directory which contains the socket - mkdir /var/run/redis - chown redis:redis /var/run/redis - chmod 755 /var/run/redis - # Persist the directory which contains the socket, if applicable - if [ -d /etc/tmpfiles.d ]; then - echo 'd /var/run/redis 0755 redis redis 10d -' | sudo tee -a /etc/tmpfiles.d/redis.conf - fi +# Persist the directory which contains the socket, if applicable +if [ -d /etc/tmpfiles.d ]; then + echo 'd /var/run/redis 0755 redis redis 10d -' | sudo tee -a /etc/tmpfiles.d/redis.conf +fi - # Activate the changes to redis.conf - sudo service redis_6379 start +# Activate the changes to redis.conf +sudo service redis_6379 start - # Add git to the redis group - sudo usermod -aG redis git +# Add git to the redis group +sudo usermod -aG redis git +``` ## 7. GitLab @@ -226,9 +231,9 @@ to install a recent version of Redis. ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-2-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-3-stable gitlab -**Note:** You can change `8-2-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `8-3-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It -- cgit v1.2.1 From 381a9951e91275f41ce09a17380222f1af3d6a22 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Mon, 14 Dec 2015 13:17:23 -0500 Subject: Update 8.2-to-8.3 guide [ci skip] --- doc/update/8.2-to-8.3.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index f1000ef4650..e69c4f7ed3c 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -26,8 +26,7 @@ to the outdated instructions from the GitLab installation guide and you can go to the next step 'Stop server' below. If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4, -v2.2.1 or newer. You can use the [instructions in the GitLab source -installation +v2.2.1 or newer. You can use the [instructions in the GitLab source installation guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) to install a newer version of Git. @@ -73,14 +72,13 @@ sudo -u git -H git checkout v2.6.8 ### 5. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires [Go -1.5](https://golang.org/dl) which should already be on your system -from GitLab 8.1. +Install and compile gitlab-workhorse. This requires [Go 1.5](https://golang.org/dl) +which should already be on your system from GitLab 8.1. ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch --all -sudo -u git -H git checkout 0.4.3 +sudo -u git -H git checkout 0.4.2 sudo -u git -H make ``` @@ -136,7 +134,6 @@ via [/etc/default/gitlab]. [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache [/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/lib/support/init.d/gitlab.default.example#L34 - ### 8. Use Redis v2.8.0+ Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but @@ -145,7 +142,7 @@ with the following command: redis-cli info | grep redis_version -If you need to upgrade, see the [installation guide for Redis](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#6-redis). +If you need to upgrade, see the [installation guide for Redis](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/doc/install/installation.md#6-redis). ### 9. Start application -- cgit v1.2.1 From f8bf6c4b2c5fb41ae92df09b9f968d5d5d61bb2b Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Fri, 11 Dec 2015 17:27:20 -0600 Subject: [ci skip] Add user repository integrity check rake task --- doc/raketasks/README.md | 3 +- doc/raketasks/check.md | 63 +++++++++++++++++++++++++++++++++++ doc/raketasks/check_repos_output.png | Bin 0 -> 73786 bytes 3 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 doc/raketasks/check.md create mode 100644 doc/raketasks/check_repos_output.png (limited to 'doc') diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md index a8dc5c24df2..cc8a22cd003 100644 --- a/doc/raketasks/README.md +++ b/doc/raketasks/README.md @@ -1,10 +1,11 @@ # Rake tasks - [Backup restore](backup_restore.md) +- [Check](check.md) - [Cleanup](cleanup.md) - [Features](features.md) - [Maintenance](maintenance.md) and self-checks - [User management](user_management.md) - [Web hooks](web_hooks.md) - [Import](import.md) of git repositories in bulk -- [Rebuild authorized_keys file](http://doc.gitlab.com/ce/raketasks/maintenance.html#rebuild-authorized_keys-file) task for administrators \ No newline at end of file +- [Rebuild authorized_keys file](http://doc.gitlab.com/ce/raketasks/maintenance.html#rebuild-authorized_keys-file) task for administrators diff --git a/doc/raketasks/check.md b/doc/raketasks/check.md new file mode 100644 index 00000000000..3ff3fee6a40 --- /dev/null +++ b/doc/raketasks/check.md @@ -0,0 +1,63 @@ +# Check Rake Tasks + +## Repository Integrity + +Even though Git is very resilient and tries to prevent data integrity issues, +there are times when things go wrong. The following Rake tasks intend to +help GitLab administrators diagnose problem repositories so they can be fixed. + +There are 3 things that are checked to determine integrity. + +1. Git repository file system check ([git fsck](https://git-scm.com/docs/git-fsck)). + This step verifies the connectivity and validity of objects in the repository. +1. Check for `config.lock` in the repository directory. +1. Check for any branch/references lock files in `refs/heads`. + +It's important to note that the existence of `config.lock` or reference locks +alone do not necessarily indicate a problem. Lock files are routinely created +and removed as Git and GitLab perform operations on the repository. They serve +to prevent data integrity issues. However, if a Git operation is interrupted these +locks may not be cleaned up properly. + +The following symptoms may indicate a problem with repository integrity. If users +experience these symptoms you may use the rake tasks described below to determine +exactly which repositories are causing the trouble. + +- Receiving an error when trying to push code - `remote: error: cannot lock ref` +- A 500 error when viewing the GitLab dashboard or when accessing a specific project. + +### Check all GitLab repositories + +This task loops through all repositories on the GitLab server and runs the +3 integrity checks described previously. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:repo:check + +# installation from source +bundle exec rake gitlab:repo:check RAILS_ENV=production +``` + +### Check repositories for a specific user + +This task checks all repositories that a specific user has access to. This is important +because sometimes you know which user is experiencing trouble but you don't know +which project might be the cause. + +If the rake task is executed without brackets at the end, you will be prompted +to enter a username. + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:user:check_repos +sudo gitlab-rake gitlab:user:check_repos[] + +# installation from source +bundle exec rake gitlab:user:check_repos RAILS_ENV=production +bundle exec rake gitlab:user:check_repos[] RAILS_ENV=production +``` + +Example output: + +![gitlab:user:check_repos output](check_repos_output.png) diff --git a/doc/raketasks/check_repos_output.png b/doc/raketasks/check_repos_output.png new file mode 100644 index 00000000000..916b1685101 Binary files /dev/null and b/doc/raketasks/check_repos_output.png differ -- cgit v1.2.1 From 2c130aa087cb837332b2e4b27eb279ac0512a4c1 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Tue, 15 Dec 2015 16:38:11 +0100 Subject: Gitlab-workhorse related 8.3 update changes --- doc/update/8.2-to-8.3.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index e69c4f7ed3c..8ea2b674a1c 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -78,7 +78,7 @@ which should already be on your system from GitLab 8.1. ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch --all -sudo -u git -H git checkout 0.4.2 +sudo -u git -H git checkout 0.5.0 sudo -u git -H make ``` @@ -115,6 +115,12 @@ git diff origin/8-2-stable:config/gitlab.yml.example origin/8-3-stable:config/gi #### Nginx configuration +GitLab 8.3 introduces major changes in the NGINX configuration. +Because all HTTP requests pass through gitlab-workhorse now a lot of +directives need to be removed from NGINX. During future upgrades there +should be much less changes in the NGINX configuration because of +this. + View changes between the previous recommended Nginx configuration and the current one: @@ -134,6 +140,18 @@ via [/etc/default/gitlab]. [Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache [/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-3-stable/lib/support/init.d/gitlab.default.example#L34 +#### Init script + +We updated the init script for GitLab in order to pass new +configuration options to gitlab-workhorse. We let gitlab-workhorse +connect to the Rails application via a Unix domain socket and we tell +it where the 'public' directory of GitLab is. + +``` +cd /home/git/gitlab +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + ### 8. Use Redis v2.8.0+ Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but -- cgit v1.2.1 From f2db1410f58357ae1033d90aae73acb3d5051644 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Tue, 15 Dec 2015 12:21:53 -0500 Subject: Update workhorse version in doc/install/installation.md --- doc/install/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index f8116a8a31c..d4b5c01f72d 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -348,7 +348,7 @@ GitLab Shell is an SSH access and repository management software developed speci cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse - sudo -u git -H git checkout 0.4.2 + sudo -u git -H git checkout 0.5.0 sudo -u git -H make ### Initialize Database and Activate Advanced Features -- cgit v1.2.1 From f98f353436d5f6305ec92de19d22a727b97dd4a0 Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Tue, 8 Dec 2015 16:25:47 -0600 Subject: [ci skip] Add SVN to Git migration documentation --- doc/README.md | 1 + doc/workflow/README.md | 1 + doc/workflow/importing/README.md | 20 ++++--- doc/workflow/importing/migrating_from_svn.md | 79 ++++++++++++++++++++++++---- 4 files changed, 84 insertions(+), 17 deletions(-) (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index a3098094210..8bac00f2f23 100644 --- a/doc/README.md +++ b/doc/README.md @@ -7,6 +7,7 @@ - [GitLab Basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab. - [Importing to GitLab](workflow/importing/README.md). - [Markdown](markdown/markdown.md) GitLab's advanced formatting system. +- [Migrating from SVN](migration/README.md) Convert a SVN repository to Git and GitLab - [Permissions](permissions/permissions.md) Learn what each role in a project (guest/reporter/developer/master/owner) can do. - [Profile Settings](profile/README.md) - [Project Services](project_services/project_services.md) Integrate a project with external services, such as CI and chat. diff --git a/doc/workflow/README.md b/doc/workflow/README.md index d2642495c9a..3651b55f438 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -19,3 +19,4 @@ - ["Work In Progress" Merge Requests](wip_merge_requests.md) - [Merge When Build Succeeds](merge_when_build_succeeds.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) +- [Importing from SVN, GitHub, BitBucket, etc](importing/README.md) diff --git a/doc/workflow/importing/README.md b/doc/workflow/importing/README.md index 7ccf06fbd60..18e5d950866 100644 --- a/doc/workflow/importing/README.md +++ b/doc/workflow/importing/README.md @@ -1,13 +1,17 @@ # Migrating projects to a GitLab instance 1. [Bitbucket](import_projects_from_bitbucket.md) -2. [GitHub](import_projects_from_github.md) -3. [GitLab.com](import_projects_from_gitlab_com.md) -4. [FogBugz](import_projects_from_fogbugz.md) -4. [SVN](migrating_from_svn.md) +1. [GitHub](import_projects_from_github.md) +1. [GitLab.com](import_projects_from_gitlab_com.md) +1. [FogBugz](import_projects_from_fogbugz.md) +1. [SVN](migrating_from_svn.md) -### Note -* If you'd like to migrate from a self-hosted GitLab instance to GitLab.com, you can copy your repos by changing the remote and pushing to the new server; but issues and merge requests can't be imported. +In addition to the specific migration documentation above, you can import any +Git repository via HTTP from the New Project page. Be aware that if the +repository is too large the import can timeout. + +### Migrating from self-hosted GitLab to GitLab.com + +You can copy your repos by changing the remote and pushing to the new server; +but issues and merge requests can't be imported. -* You can import any Git repository via HTTP from the New Project page. -If the repository is too large, it can timeout. diff --git a/doc/workflow/importing/migrating_from_svn.md b/doc/workflow/importing/migrating_from_svn.md index 1938ccd0c26..b355a91b5a6 100644 --- a/doc/workflow/importing/migrating_from_svn.md +++ b/doc/workflow/importing/migrating_from_svn.md @@ -1,17 +1,78 @@ # Migrating from SVN to GitLab -SVN stands for Subversion and is a version control system (VCS). -Git is a distributed version control system. +Subversion (SVN) is a central version control system (VCS) while +Git is a distributed version control system. There are some major differences +between the two, for more information consult your favorite search engine. -There are some major differences between the two, for more information consult your favorite search engine. +If you are currently using an SVN repository, you can migrate the repository +to Git and GitLab. We recommend a hard cut over - run the migration command once +and then have all developers start using the new GitLab repository immediately. +Otherwise, it's hard to keep changing in sync in both directions. The conversion +process should be run on a local workstation. -Git has tools for migrating SVN repositories to git, namely `git svn`. You can read more about this at -[git documentation pages](https://git-scm.com/book/en/Git-and-Other-Systems-Git-and-Subversion). +Install `svn2git`. On all systems you can install as a Ruby gem if you already +have Ruby and Git installed. -Apart from the [official git documentation](https://git-scm.com/book/en/Git-and-Other-Systems-Migrating-to-Git) there is also -user created step by step guide for migrating from SVN to GitLab. +```bash +sudo gem install svn2git +``` -[Benjamin New](https://github.com/leftclickben) wrote [a guide that shows how to do a migration](https://gist.github.com/leftclickben/322b7a3042cbe97ed2af). Mirrors can be found [here](https://gitlab.com/snippets/2168) and [here](https://gist.github.com/maxlazio/f1b593b0d00aa966e9ca). +On Debian-based Linux distributions you can install the native packages: + +```bash +sudo apt-get install git-core git-svn ruby +``` + +Optionally, prepare an authors file so `svn2git` can map SVN authors to Git authors. +If you choose not to create the authors file then commits will not be attributed +to the correct GitLab user. Some users may not consider this a big issue while +others will want to ensure they complete this step. If you choose to map authors +you will be required to map every author that is present on changes in the SVN +repository. If you don't, the conversion will fail and you will have to update +the author file accordingly. The following command will search through the +repository and output a list of authors. + +```bash +svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq +``` + +Use the output from the last command to construct the authors file. +Create a file called `authors.txt` and add one mapping per line. + +``` +janedoe = Jane Doe +johndoe = John Doe +``` + +If your SVN repository is in the standard format (trunk, branches, tags, +not nested) the conversion is simple. For a non-standard repository see +[svn2git documentation](https://github.com/nirvdrum/svn2git). The following +command will checkout the repository and do the conversion in the current +working directory. Be sure to create a new directory for each repository before +running the `svn2git` command. The conversion process will take some time. + +```bash +svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt +``` + +If your SVN repository requires a username and password add the +`--username ` and `--password /.git +git push --all origin +``` ## Contribute to this guide -We welcome all contributions that would expand this guide with instructions on how to migrate from SVN and other version control systems. +We welcome all contributions that would expand this guide with instructions on +how to migrate from SVN and other version control systems. + + -- cgit v1.2.1 From fc9fbdce44570042c79ff9a66289890bcb479524 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 10 Dec 2015 15:56:20 +0200 Subject: Clean up ci yaml doc [ci skip] * Fix headings so that they are picked up by doc.gitlab.com * Stick to 80 characters * Clean up some examples --- doc/ci/yaml/README.md | 345 +++++++++++++++++++++++++++++++------------------- 1 file changed, 212 insertions(+), 133 deletions(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 7e2edb945da..c5bdce2f778 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,9 +1,12 @@ # Configuration of your builds with .gitlab-ci.yml -From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) file (**.gitlab-ci.yml**) for the project configuration. -It is placed in the root of your repository and contains definitions of how your project should be built. -The YAML file defines a set of jobs with constraints stating when they should be run. -The jobs are defined as top-level elements with a name and always have to contain the `script` clause: +From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) +file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root +of your repository and contains definitions of how your project should be built. + +The YAML file defines a set of jobs with constraints stating when they should +be run. The jobs are defined as top-level elements with a name and always have +to contain the `script` clause: ```yaml job1: @@ -13,15 +16,21 @@ job2: script: "execute-script-for-job2" ``` -The above example is the simplest possible CI configuration with two separate jobs, -where each of the jobs executes a different command. -Of course a command can execute code directly (`./configure;make;make install`) or run a script (`test.sh`) in the repository. +The above example is the simplest possible CI configuration with two separate +jobs, where each of the jobs executes a different command. + +Of course a command can execute code directly (`./configure;make;make install`) +or run a script (`test.sh`) in the repository. -Jobs are used to create builds, which are then picked up by [runners](../runners/README.md) and executed within the environment of the runner. -What is important, is that each job is run independently from each other. +Jobs are used to create builds, which are then picked up by +[runners](../runners/README.md) and executed within the environment of the +runner. What is important, is that each job is run independently from each +other. ## .gitlab-ci.yml -The YAML syntax allows for using more complex job specifications than in the above example: + +The YAML syntax allows for using more complex job specifications than in the +above example: ```yaml image: ruby:2.1 @@ -46,26 +55,31 @@ job1: - docker ``` -There are a few `keywords` that can't be used as job names: +There are a few reserved `keywords` that **cannot** be used as job names: -| keyword | required | description | +| Keyword | Required | Description | |---------------|----------|-------------| -| image | optional | Use docker image, covered in [Use Docker](../docker/README.md) | -| services | optional | Use docker services, covered in [Use Docker](../docker/README.md) | -| stages | optional | Define build stages | -| types | optional | Alias for `stages` | -| before_script | optional | Define commands prepended for each job's script | -| variables | optional | Define build variables | -| cache | optional | Define list of files that should be cached between subsequent runs | +| image | no | Use docker image, covered in [Use Docker](../docker/README.md) | +| services | no | Use docker services, covered in [Use Docker](../docker/README.md) | +| stages | no | Define build stages | +| types | no | Alias for `stages` | +| before_script | no | Define commands that run before each job's script | +| variables | no | Define build variables | +| cache | no | Define list of files that should be cached between subsequent runs | ### image and services -This allows to specify a custom Docker image and a list of services that can be used for time of the build. -The configuration of this feature is covered in separate document: [Use Docker](../docker/README.md). + +This allows to specify a custom Docker image and a list of services that can be +used for time of the build. The configuration of this feature is covered in +separate document: [Use Docker](../docker/README.md). ### before_script -`before_script` is used to define the command that should be run before all builds, including deploy builds. This can be an array or a multiline string. + +`before_script` is used to define the command that should be run before all +builds, including deploy builds. This can be an array or a multi-line string. ### stages + `stages` is used to define build stages that can be used by jobs. The specification of `stages` allows for having flexible multi stage pipelines. @@ -75,7 +89,8 @@ The ordering of elements in `stages` defines the ordering of builds' execution: 1. Builds of next stage are run after success. Let's consider the following example, which defines 3 stages: -``` + +```yaml stages: - build - test @@ -86,21 +101,26 @@ stages: 1. If all jobs of `build` succeeds, the `test` jobs are executed in parallel. 1. If all jobs of `test` succeeds, the `deploy` jobs are executed in parallel. 1. If all jobs of `deploy` succeeds, the commit is marked as `success`. -1. If any of the previous jobs fails, the commit is marked as `failed` and no jobs of further stage are executed. +1. If any of the previous jobs fails, the commit is marked as `failed` and no + jobs of further stage are executed. There are also two edge cases worth mentioning: -1. If no `stages` is defined in `.gitlab-ci.yml`, then by default the `build`, `test` and `deploy` are allowed to be used as job's stage by default. +1. If no `stages` is defined in `.gitlab-ci.yml`, then by default the `build`, + `test` and `deploy` are allowed to be used as job's stage by default. 2. If a job doesn't specify `stage`, the job is assigned the `test` stage. ### types + Alias for [stages](#stages). ### variables -**This feature requires `gitlab-runner` with version equal or greater than 0.5.0.** -GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build environment. -The variables are stored in repository and are meant to store non-sensitive project configuration, ie. RAILS_ENV or DATABASE_URL. +_**Note:** Introduced in GitLab Runner v0.5.0._ + +GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build +environment. The variables are stored in the git repository and are meant to +store non-sensitive project configuration, for example: ```yaml variables: @@ -109,18 +129,23 @@ variables: These variables can be later used in all executed commands and scripts. -The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +The YAML-defined variables are also set to all created service containers, +thus allowing to fine tune them. ### cache -`cache` is used to specify list of files and directories which should be cached between builds. -Caches are stored according to the branch/ref and the job name. Caches are not -currently shared between different job names or between branches/refs. This means -caching will benefit you if you push subsequent commits to an existing feature branch. -**The global setting allows to specify default cached files for all jobs.** +`cache` is used to specify a list of files and directories which should be +cached between builds. Caches are stored according to the branch/ref and the +job name. They are not currently shared between different job names or between +branches/refs, which means that caching will benefit you if you push subsequent +commits to an existing feature branch. + +If `cache` is defined outside the scope of the jobs, it means it is set +globally and all jobs will use its definition. To cache all git untracked files and files in `binaries`: -``` + +```yaml cache: untracked: true paths: @@ -128,9 +153,10 @@ cache: ``` ## Jobs -`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. -Each job has to have a unique `job_name`, which is not one of the keywords mentioned above. -A job is defined by a list of parameters that define the build behaviour. + +`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job +must have a unique name, which is not one of the Keywords mentioned above. +A job is defined by a list of parameters that define the build behavior. ```yaml job_name: @@ -148,21 +174,22 @@ job_name: allow_failure: true ``` -| keyword | required | description | +| Keyword | Required | Description | |---------------|----------|-------------| -| script | required | Defines a shell script which is executed by runner | -| stage | optional (default: test) | Defines a build stage | -| type | optional | Alias for `stage` | -| only | optional | Defines a list of git refs for which build is created | -| except | optional | Defines a list of git refs for which build is not created | -| tags | optional | Defines a list of tags which are used to select runner | -| allow_failure | optional | Allow build to fail. Failed build doesn't contribute to commit status | -| when | optional | Define when to run build. Can be `on_success`, `on_failure` or `always` | -| artifacts | optional | Define list build artifacts | -| cache | optional | Define list of files that should be cached between subsequent runs | +| script | yes | Defines a shell script which is executed by runner | +| stage | no (default: `test`) | Defines a build stage | +| type | no | Alias for `stage` | +| only | no | Defines a list of git refs for which build is created | +| except | no | Defines a list of git refs for which build is not created | +| tags | no | Defines a list of tags which are used to select runner | +| allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status | +| when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` | +| artifacts | no | Define list build artifacts | +| cache | no | Define list of files that should be cached between subsequent runs | ### script -`script` is a shell script which is executed by runner. The shell script is prepended with `before_script`. + +`script` is a shell script which is executed by the runner. For example: ```yaml job: @@ -170,6 +197,7 @@ job: ``` This parameter can also contain several commands using an array: + ```yaml job: script: @@ -178,31 +206,45 @@ job: ``` ### stage -`stage` allows to group build into different stages. Builds of the same `stage` are executed in `parallel`. -For more info about the use of `stage` please check the [stages](#stages). + +`stage` allows to group build into different stages. Builds of the same `stage` +are executed in `parallel`. For more info about the use of `stage` please check +[stages](#stages). ### only and except -This are two parameters that allow for setting a refs policy to limit when jobs are built: -1. `only` defines the names of branches and tags for which job will be built. -2. `except` defines the names of branches and tags for which the job wil **not** be built. -There are a few rules that apply to usage of refs policy: +`only` and `except` are two parameters that set a refs policy to limit when +jobs are built: -1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. -1. `only` and `except` allow for using the regexp expressions. -1. `only` and `except` allow for using special keywords: `branches` and `tags`. -These names can be used for example to exclude all tags and all branches. +1. `only` defines the names of branches and tags for which the job will be + built. +2. `except` defines the names of branches and tags for which the job will + **not** be built. + +There are a few rules that apply to the usage of refs policy: + +* `only` and `except` are inclusive. If both `only` and `except` are defined + in a job specification, the ref is filtered by `only` and `except`. +* `only` and `except` allow the use of regular expressions. +* `only` and `except` allow the use of special keywords: `branches` and `tags`. +* `only` and `except` allow to specify a repository path to filter jobs for + forks. + +In the example below, `job` will run only for refs that start with `issue-`, +whereas all branches will be skipped. ```yaml job: + # use regexp only: - - /^issue-.*$/ # use regexp + - /^issue-.*$/ + # use special keyword except: - - branches # use special keyword + - branches ``` -1. `only` and `except` allow for specify repository path to filter jobs for forks. -The repository path can be used to have jobs executed only for parent repository. +The repository path can be used to have jobs executed only for the parent +repository and not forks: ```yaml job: @@ -211,33 +253,47 @@ job: except: - master@gitlab-org/gitlab-ce ``` -The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . + +The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, +except master. ### tags -`tags` is used to select specific runners from the list of all runners that are allowed to run this project. -During registration of a runner, you can specify the runner's tags, ie.: `ruby`, `postgres`, `development`. -`tags` allow you to run builds with runners that have the specified tags assigned: +`tags` is used to select specific runners from the list of all runners that are +allowed to run this project. -``` +During the registration of a runner, you can specify the runner's tags, for +example `ruby`, `postgres`, `development`. + +`tags` allow you to run builds with runners that have the specified tags +assigned to them: + +```yaml job: tags: - ruby - postgres ``` -The above specification will make sure that `job` is built by a runner that have `ruby` AND `postgres` tags defined. +The specification above, will make sure that `job` is built by a runner that +has both `ruby` AND `postgres` tags defined. ### when -`when` is used to implement jobs that are run in case of failure or despite the failure. + +`when` is used to implement jobs that are run in case of failure or despite the +failure. `when` can be set to one of the following values: -1. `on_success` - execute build only when all builds from prior stages succeeded. This is the default. -1. `on_failure` - execute build only when at least one build from prior stages failed. +1. `on_success` - execute build only when all builds from prior stages + succeeded. This is the default. +1. `on_failure` - execute build only when at least one build from prior stages + failed. 1. `always` - execute build despite the status of builds from prior stages. -``` +For example: + +```yaml stages: - build - cleanup_build @@ -245,28 +301,28 @@ stages: - deploy - cleanup -build: +build_job: stage: build script: - make build -cleanup_build: +cleanup_build_job: stage: cleanup_build script: - cleanup build when failed when: on_failure -test: +test_job: stage: test script: - make test -deploy: +deploy_job: stage: deploy script: - make deploy -cleanup: +cleanup_job: stage: cleanup script: - cleanup after builds @@ -274,84 +330,107 @@ cleanup: ``` The above script will: -1. Execute `cleanup_build` only when the `build` failed, -2. Always execute `cleanup` as the last step in pipeline. + +1. Execute `cleanup_build_job` only when `build_job` fails +2. Always execute `cleanup_job` as the last step in pipeline. ### artifacts -`artifacts` is used to specify list of files and directories which should be attached to build after success. -1. Send all files in `binaries` and `.config`: +_**Note:** Introduced in GitLab Runner v0.7.0._ - artifacts: - paths: - - binaries/ - - .config +`artifacts` is used to specify list of files and directories which should be +attached to build after success. Below are some examples. -2. Send all git untracked files: +Send all files in `binaries` and `.config`: - artifacts: - untracked: true +```yaml +artifacts: + paths: + - binaries/ + - .config +``` -3. Send all git untracked files and files in `binaries`: +Send all git untracked files: - artifacts: - untracked: true - paths: - - binaries/ +```yaml +artifacts: + untracked: true +``` + +Send all git untracked files and files in `binaries`: -The artifacts will be send after the build success to GitLab and will be accessible in GitLab interface to download. +```yaml +artifacts: + untracked: true + paths: + - binaries/ +``` -This feature requires GitLab Runner v0.7.0 or higher. +The artifacts will be send after a successful build success to GitLab, and will +be accessible in the GitLab UI to download. ### cache -`cache` is used to specify list of files and directories which should be cached between builds. -1. Cache all files in `binaries` and `.config`: +_**Note:** Introduced in GitLab Runner v0.7.0._ - rspec: - script: test - cache: - paths: - - binaries/ - - .config +`cache` is used to specify list of files and directories which should be cached +between builds. Below are some examples: -2. Cache all git untracked files: +Cache all files in `binaries` and `.config`: - rspec: - script: test - cache: - untracked: true - -3. Cache all git untracked files and files in `binaries`: +```yaml +rspec: + script: test + cache: + paths: + - binaries/ + - .config +``` - rspec: - script: test - cache: - untracked: true - paths: - - binaries/ +Cache all git untracked files: -4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: +```yaml +rspec: + script: test + cache: + untracked: true +``` + +Cache all git untracked files and files in `binaries`: + +```yaml +rspec: + script: test + cache: + untracked: true + paths: + - binaries/ +``` - cache: - paths: - - my/files - - rspec: - script: test - cache: - paths: - - binaries/ +Locally defined cache overwrites globally defined options. This will cache only +`binaries/`: -The cache is provided on best effort basis, so don't expect that cache will be present. -For implementation details please check GitLab Runner. +```yaml +cache: + paths: + - my/files -This feature requires GitLab Runner v0.7.0 or higher. +rspec: + script: test + cache: + paths: + - binaries/ +``` +The cache is provided on best effort basis, so don't expect that cache will be +always present. For implementation details please check GitLab Runner. ## Validate the .gitlab-ci.yml + Each instance of GitLab CI has an embedded debug tool called Lint. -You can find the link to the Lint in the project's settings page or use short url `/lint`. +You can find the link under `/ci/lint` of your gitlab instance. ## Skipping builds -There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped + +If your commit message contains `[ci skip]`, the commit will be created but the +builds will be skipped. -- cgit v1.2.1 From d2b5ef1c1e1011b5ccc8a3142b0147ce29632963 Mon Sep 17 00:00:00 2001 From: Cauan Cabral Date: Wed, 16 Dec 2015 14:23:02 -0300 Subject: Remove CI_BUILD_BEFORE_SHA from CI documentation As pointed in #3210, the environment variable isn't usable any more. --- doc/ci/variables/README.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 022afb70042..b99ea25a3fe 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -27,7 +27,6 @@ The API_TOKEN will take the Secure Variable value: `SECURE`. | **CI_BUILD_TAG** | 0.5 | The commit tag name. Present only when building tags. | | **CI_BUILD_NAME** | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | | **CI_BUILD_STAGE** | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | -| **CI_BUILD_BEFORE_SHA** | all | The first commit that were included in push request | | **CI_BUILD_REF_NAME** | all | The branch or tag name for which project is built | | **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally | | **CI_BUILD_REPO** | all | The URL to clone the Git repository | @@ -40,7 +39,6 @@ The API_TOKEN will take the Secure Variable value: `SECURE`. Example values: ```bash -export CI_BUILD_BEFORE_SHA="9df57456fa9de2a6d335ca5edf9750ed812b9df0" export CI_BUILD_ID="50" export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_BUILD_REF_NAME="master" -- cgit v1.2.1 From c36821df9f571855b477b09c102b90a78b9079e9 Mon Sep 17 00:00:00 2001 From: Zeger-Jan van de Weg Date: Wed, 16 Dec 2015 21:39:27 +0100 Subject: Api support for requesting starred projects for user Fixes #4112 --- doc/api/projects.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) (limited to 'doc') diff --git a/doc/api/projects.md b/doc/api/projects.md index 2c7a3d5c552..658e65c6f01 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -139,6 +139,21 @@ Parameters: - `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` - `search` (optional) - Return list of authorized projects according to a search criteria +### List starred projects + +Get a list of projects which are starred by the authenticated user. + +``` +GET /projects/starred +``` + +Parameters: + +- `archived` (optional) - if passed, limit by archived status +- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` +- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` +- `search` (optional) - Return list of authorized projects according to a search criteria + ### List ALL projects Get a list of all GitLab projects (admin only). -- cgit v1.2.1 From 8579fea2cfe4c822dae12efaf50f9e01adbc752e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 17 Dec 2015 08:53:00 +0200 Subject: Add info on using private Docker registries in CI [ci skip] --- doc/ci/docker/using_docker_images.md | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 8d4bd44053e..31458d61674 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,11 +1,11 @@ # Using Docker Images -GitLab CI in conjuction with [GitLab Runner](../runners/README.md) can use +GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use [Docker Engine](https://www.docker.com/) to test and build any application. Docker is an open-source project that allows you to use predefined images to run applications in independent "containers" that are run within a single Linux -instance. [Docker Hub][hub] has a rich database of prebuilt images that can be +instance. [Docker Hub][hub] has a rich database of pre-built images that can be used to test and build your applications. Docker, when used with GitLab CI, runs each build in a separate and isolated @@ -136,6 +136,24 @@ Look for the `[runners.docker]` section: The image and services defined this way will be added to all builds run by that runner. +## Define an image from a private Docker registry + +Starting with GitLab Runner 0.6.0, you are able to define images located to +private registries that could also require authentication. + +All you have to do is be explicit on the image definition in `.gitlab-ci.yml`. + +```yaml +image: my.registry.tld:5000/namepace/image:tag +``` + +In the example above, GitLab Runner will look at `my.registry.tld:5000` for the +image `namespace/image:tag`. + +If the repository is private you need to authenticate your GitLab Runner in the +registry. Learn how to do that on +[GitLab Runner's documentation][runner-priv-reg]. + ## Accessing the services Let's say that you need a Wordpress instance to test some API integration with @@ -258,3 +276,4 @@ creation. [tutum/wordpress]: https://registry.hub.docker.com/u/tutum/wordpress/ [postgres-hub]: https://registry.hub.docker.com/u/library/postgres/ [mysql-hub]: https://registry.hub.docker.com/u/library/mysql/ +[runner-priv-reg]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry -- cgit v1.2.1 From b336eb01f711f170c20393cc07e7f9a7b80fea87 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Thu, 17 Dec 2015 11:55:13 +0100 Subject: Use gitlab-workhorse 0.5.1 --- doc/install/installation.md | 2 +- doc/update/8.2-to-8.3.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index d4b5c01f72d..81edd8da2b8 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -348,7 +348,7 @@ GitLab Shell is an SSH access and repository management software developed speci cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse - sudo -u git -H git checkout 0.5.0 + sudo -u git -H git checkout 0.5.1 sudo -u git -H make ### Initialize Database and Activate Advanced Features diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 8ea2b674a1c..b7022458396 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -78,7 +78,7 @@ which should already be on your system from GitLab 8.1. ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch --all -sudo -u git -H git checkout 0.5.0 +sudo -u git -H git checkout 0.5.1 sudo -u git -H make ``` -- cgit v1.2.1 From c019154a911e751efba707f0323797b240775b08 Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Thu, 17 Dec 2015 13:01:57 -0600 Subject: Clarify Windows shell executor artifact upload support --- doc/ci/yaml/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index c5bdce2f778..fd0d49de4e4 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -336,7 +336,8 @@ The above script will: ### artifacts -_**Note:** Introduced in GitLab Runner v0.7.0._ +_**Note:** Introduced in GitLab Runner v0.7.0. Also, the Windows shell executor + does not currently support artifact uploads._ `artifacts` is used to specify list of files and directories which should be attached to build after success. Below are some examples. -- cgit v1.2.1 From 56d780cf2c5b351479f002b221755bfb85b3ff70 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Thu, 17 Dec 2015 20:54:27 +0100 Subject: Use gitlab-shell 2.6.9 --- doc/update/8.2-to-8.3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index b7022458396..c4661dc16af 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -67,7 +67,7 @@ sudo -u git -H git checkout 8-3-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch --all -sudo -u git -H git checkout v2.6.8 +sudo -u git -H git checkout v2.6.9 ``` ### 5. Update gitlab-workhorse -- cgit v1.2.1