diff options
Diffstat (limited to 'doc')
56 files changed, 1155 insertions, 193 deletions
diff --git a/doc/README.md b/doc/README.md index ab8ea192a26..9b81c409570 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,12 +1,24 @@ -# GitLab Community Edition +# GitLab Documentation -[GitLab](https://about.gitlab.com/) is a Git-based fully featured platform -for software development. +Welcome to [GitLab](https://about.gitlab.com/), a Git-based fully featured +platform for software development! -**GitLab Community Edition (CE)** is an opensource product, self-hosted, free to use. -All [GitLab products](https://about.gitlab.com/products/) contain the features -available in GitLab CE. Premium features are available in -[GitLab Enterprise Edition (EE)](https://about.gitlab.com/gitlab-ee/). +We offer four different products for you and your company: + +- **GitLab Community Edition (CE)** is an [opensource product](https://gitlab.com/gitlab-org/gitlab-ce/), +self-hosted, free to use. Every feature available in GitLab CE is also available on GitLab Enterprise Edition (Starter and Premium) and GitLab.com. +- **GitLab Enterprise Edition (EE)** is an [opencore product](https://gitlab.com/gitlab-org/gitlab-ee/), +self-hosted, fully featured solution of GitLab, available under distinct [subscriptions](https://about.gitlab.com/products/): **GitLab Enterprise Edition Starter (EES)** and **GitLab Enterprise Edition Premium (EEP)**. +- **GitLab.com**: SaaS GitLab solution, with [free and paid subscriptions](https://about.gitlab.com/gitlab-com/). GitLab.com is hosted by GitLab, Inc., and administrated by GitLab (users don't have access to admin settings). + +**GitLab EE** contains all features available in **GitLab CE**, +plus premium features available in each version: **Enterprise Edition Starter** +(**EES**) and **Enterprise Edition Premium** (**EEP**). Everything available in +**EES** is also available in **EEP**. + +**Note:** _We are unifying the documentation for CE and EE. To check if certain feature is +available in CE or EE, look for a note right below the page title containing the GitLab +version which introduced that feature._ ---- @@ -125,7 +137,7 @@ have access to GitLab administration tools and settings. - [Access restrictions](user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab - [Authentication/Authorization](topics/authentication/index.md#gitlab-administrators): Enforce 2FA, configure external authentication with LDAP, SAML, CAS and additional Omniauth providers. -### GitLab admins' superpowers +### Features - [Container Registry](administration/container_registry.md): Configure Docker Registry with GitLab. - [Custom Git hooks](administration/custom_hooks.md): Custom Git hooks (on the filesystem) for when webhooks aren't enough. @@ -155,6 +167,7 @@ have access to GitLab administration tools and settings. - [Operations](administration/operations.md): Keeping GitLab up and running. - [Polling](administration/polling.md): Configure how often the GitLab UI polls for updates. - [Request Profiling](administration/monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. +- [Performance Bar](administration/monitoring/performance/performance_bar.md): Get performance information for the current page. ### Customization @@ -167,6 +180,7 @@ have access to GitLab administration tools and settings. ### Admin tools +- [Gitaly](administration/gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service - [Raketasks](raketasks/README.md): Backups, maintenance, automatic webhook setup and the importing of projects. - [Backup and restore](raketasks/backup_restore.md): Backup and restore your GitLab instance. - [Reply by email](administration/reply_by_email.md): Allow users to comment on issues and merge requests by replying to notification emails. diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 332457cb384..5732b6a1ca4 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -2,8 +2,7 @@ [Gitaly](https://gitlab.com/gitlab-org/gitaly) (introduced in GitLab 9.0) is a service that provides high-level RPC access to Git -repositories. As of GitLab 9.3 it is still an optional component with -limited scope. +repositories. Gitaly is a mandatory component in GitLab 9.4 and newer. GitLab components that access Git repositories (gitlab-rails, gitlab-shell, gitlab-workhorse) act as clients to Gitaly. End users do @@ -149,6 +148,8 @@ git_data_dirs({ { 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, { 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, }) + +gitlab_rails['gitaly_token'] = 'abc123secret' ``` Source installations: @@ -164,6 +165,9 @@ gitlab: storage1: path: /mnt/gitlab/storage1/repositories gitaly_address: tcp://gitlab.internal:9999 + + gitaly: + token: 'abc123secret' ``` Now reconfigure (Omnibus) or restart (source). When you tail the @@ -172,36 +176,11 @@ Gitaly logs on your Gitaly server (`sudo gitlab-ctl tail gitaly` or coming in. One sure way to trigger a Gitaly request is to clone a repository from your GitLab server over HTTP. -## Configuring GitLab to not use Gitaly - -Gitaly is still an optional component in GitLab 9.3. This means you -can choose to not use it. - -In Omnibus you can make the following change in -`/etc/gitlab/gitlab.rb` and reconfigure. This will both disable the -Gitaly service and configure the rest of GitLab not to use it. - -```ruby -gitaly['enable'] = false -``` - -In source installations, edit `/home/git/gitlab/config/gitlab.yml` and -make sure `enabled` in the `gitaly` section is set to 'false'. This -does not disable the Gitaly service in your init script; it only -prevents it from being used. - -Apply the change with `service gitlab restart`. - -```yaml - gitaly: - enabled: false -``` - ## Disabling or enabling the Gitaly service -Be careful: if you disable Gitaly without instructing the rest of your -GitLab installation not to use Gitaly, you may end up with errors -because GitLab tries to access a service that is not running. +If you are running Gitaly [as a remote +service](#running-gitaly-on-its-own-server) you may want to disable +the local Gitaly service that runs on your Gitlab server by default. To disable the Gitaly service in your Omnibus installation, add the following line to `/etc/gitlab/gitlab.rb`: @@ -220,4 +199,4 @@ following to `/etc/default/gitlab`: gitaly_enabled=false ``` -When you run `service gitlab restart` Gitaly will be disabled.
\ No newline at end of file +When you run `service gitlab restart` Gitaly will be disabled. diff --git a/doc/administration/monitoring/performance/img/performance_bar.png b/doc/administration/monitoring/performance/img/performance_bar.png Binary files differnew file mode 100644 index 00000000000..d38293d2ed6 --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png b/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png Binary files differnew file mode 100644 index 00000000000..2d64ef8c5fc --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png Binary files differnew file mode 100644 index 00000000000..7868e2c46d1 --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png Binary files differnew file mode 100644 index 00000000000..372ae021f6b --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md new file mode 100644 index 00000000000..ee680c7b258 --- /dev/null +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -0,0 +1,35 @@ +# Performance Bar + +A Performance Bar can be displayed, to dig into the performance of a page. When +activated, it looks as follows: + + + +It allows you to: + +- see the current host serving the page +- see the timing of the page (backend, frontend) +- the number of DB queries, the time it took, and the detail of these queries + +- the number of calls to Redis, and the time it took +- the number of background jobs created by Sidekiq, and the time it took +- the number of Ruby GC calls, and the time it took +- profile the code used to generate the page, line by line + + +## Enable the Performance Bar via the Admin panel + +GitLab Performance Bar is disabled by default. To enable it for a given group, +navigate to the Admin area in **Settings > Profiling - Performance Bar** +(`/admin/application_settings`). + +The only required setting you need to set is the full path of the group that +will be allowed to display the Performance Bar. +Make sure _Enable the Performance Bar_ is checked and hit +**Save** to save the changes. + +--- + + + +--- diff --git a/doc/api/README.md b/doc/api/README.md index b7f6ee69193..95e7a457848 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -17,6 +17,7 @@ following locations: - [Deploy Keys](deploy_keys.md) - [Environments](environments.md) - [Events](events.md) +- [Feature flags](features.md) - [Gitignores templates](templates/gitignores.md) - [GitLab CI Config templates](templates/gitlab_ci_ymls.md) - [Groups](groups.md) diff --git a/doc/api/features.md b/doc/api/features.md index 89b8d3ac948..6861dbf00a2 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -1,4 +1,4 @@ -# Features API +# Features flags API All methods require administrator authorization. @@ -58,6 +58,11 @@ POST /features/:name | --------- | ---- | -------- | ----------- | | `name` | string | yes | Name of the feature to create or update | | `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time | +| `feature_group` | string | no | A Feature group name | +| `user` | string | no | A GitLab username | + +Note that you can enable or disable a feature for both a `feature_group` and a +`user` with a single API call. ```bash curl --data "value=30" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/new_library diff --git a/doc/api/issues.md b/doc/api/issues.md index df5666bb7b6..a00a63bad4b 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -964,3 +964,30 @@ Example response: ## Comments on issues Comments are done via the [notes](notes.md) resource. + +## Get user agent details + +Available only for admins. + +``` +GET /projects/:id/issues/:issue_iid/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `issue_iid` | integer | yes | The internal ID of a project's issue | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 4ad6071a0ed..8133251dffe 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -29,22 +29,30 @@ Example response: { "id": 1, "path": "user1", - "kind": "user" + "kind": "user", + "full_path": "user1" }, { "id": 2, "path": "group1", - "kind": "group" + "kind": "group", + "full_path": "group1", + "parent_id": "null", + "members_count_with_descendants": 2 }, { "id": 3, "path": "bar", "kind": "group", "full_path": "foo/bar", + "parent_id": "9", + "members_count_with_descendants": 5 } ] ``` +**Note**: `members_count_with_descendants` are presented only for group masters/owners. + ## Search for namespace Get all namespaces that match a string in their name or path. @@ -72,6 +80,8 @@ Example response: "path": "twitter", "kind": "group", "full_path": "twitter", + "parent_id": "null", + "members_count_with_descendants": 2 } ] ``` diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index 92491de4daa..d74398c6e65 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -119,3 +119,35 @@ Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `snippet_id` (required) - The ID of a project's snippet + +## Get user agent details + +> **Notes:** +> [Introduced][ce-29508] in GitLab 9.4. + + +Available only for admins. + +``` +GET /projects/:id/snippets/:snippet_id/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | Integer | yes | The ID of a snippet | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/snippets/1/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` + +[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 diff --git a/doc/api/projects.md b/doc/api/projects.md index cc1bb3911c8..0d892c74d00 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -173,6 +173,164 @@ Parameters: ] ``` +### List a user's projects + +Get a list of visible projects for the given user. When accessed without authentication, only public projects are returned. + +``` +GET /users/:user_id/projects +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user_id` | string | yes | The ID or username of the user | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | +| `owned` | boolean | no | Limit by projects owned by the current user | +| `membership` | boolean | no | Limit by projects that the current user is a member of | +| `starred` | boolean | no | Limit by projects starred by the current user | +| `statistics` | boolean | no | Include project statistics | +| `with_issues_enabled` | boolean | no | Limit by enabled issues feature | +| `with_merge_requests_enabled` | boolean | no | Limit by enabled merge requests feature | + +```json +[ + { + "id": 4, + "description": null, + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", + "web_url": "http://example.com/diaspora/diaspora-client", + "tag_list": [ + "example", + "disapora client" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Diaspora Client", + "name_with_namespace": "Diaspora / Diaspora Client", + "path": "diaspora-client", + "path_with_namespace": "diaspora/diaspora-client", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 3, + "name": "Diaspora", + "path": "diaspora", + "kind": "group", + "full_path": "diaspora" + }, + "import_status": "none", + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "statistics": { + "commit_count": 37, + "storage_size": 1038090, + "repository_size": 1038090, + "lfs_objects_size": 0, + "job_artifacts_size": 0 + } + }, + { + "id": 6, + "description": null, + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", + "http_url_to_repo": "http://example.com/brightbox/puppet.git", + "web_url": "http://example.com/brightbox/puppet", + "tag_list": [ + "example", + "puppet" + ], + "owner": { + "id": 4, + "name": "Brightbox", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Puppet", + "name_with_namespace": "Brightbox / Puppet", + "path": "puppet", + "path_with_namespace": "brightbox/puppet", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 4, + "name": "Brightbox", + "path": "brightbox", + "kind": "group", + "full_path": "brightbox" + }, + "import_status": "none", + "import_error": null, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": false, + "avatar_url": null, + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "statistics": { + "commit_count": 12, + "storage_size": 2066080, + "repository_size": 2066080, + "lfs_objects_size": 0, + "job_artifacts_size": 0 + } + } +] +``` + ### Get single project Get a specific project. This endpoint can be accessed without authentication if @@ -336,7 +494,7 @@ Parameters: | `snippets_enabled` | boolean | no | Enable snippets for this project | | `container_registry_enabled` | boolean | no | Enable container registry for this project | | `shared_runners_enabled` | boolean | no | Enable shared runners for this project | -| `visibility` | String | no | See [project visibility level](#project-visibility-level) | +| `visibility` | string | no | See [project visibility level](#project-visibility-level) | | `import_url` | string | no | URL to import repository from | | `public_jobs` | boolean | no | If `true`, jobs can be viewed by non-project-members | | `only_allow_merge_if_pipeline_succeeds` | boolean | no | Set whether merge requests can only be merged with successful jobs | @@ -346,6 +504,7 @@ Parameters: | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | +| `ci_config_path` | string | no | The path to CI config file | ### Create project for user @@ -382,6 +541,7 @@ Parameters: | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | +| `ci_config_path` | string | no | The path to CI config file | ### Edit project @@ -416,6 +576,7 @@ Parameters: | `request_access_enabled` | boolean | no | Allow users to request member access | | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | +| `ci_config_path` | string | no | The path to CI config file | ### Fork project diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 18ceb8f779e..1fc577561a0 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -61,7 +61,7 @@ POST /projects/:id/repository/files/:file_path ``` ```bash -curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fprojectrb%2E?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' +curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fprojectrb%2E?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' ``` Example response: @@ -90,7 +90,7 @@ PUT /projects/:id/repository/files/:file_path ``` ```bash -curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' ``` Example response: @@ -129,7 +129,7 @@ DELETE /projects/:id/repository/files/:file_path ``` ```bash -curl --request DELETE --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' +curl --request DELETE --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' ``` Example response: diff --git a/doc/api/snippets.md b/doc/api/snippets.md index efaab712367..fdafbfb5b9e 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -234,3 +234,35 @@ Example response: } ] ``` + +## Get user agent details + +> **Notes:** +> [Introduced][ce-29508] in GitLab 9.4. + + +Available only for admins. + +``` +GET /snippets/:id/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | Integer | yes | The ID of a snippet | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/snippets/1/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` + +[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index be4dea55c20..d3433594eb7 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,4 +1,4 @@ -# Using Docker Images +# Using Docker images GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use [Docker Engine](https://www.docker.com/) to test and build any application. @@ -17,14 +17,16 @@ can also run on your workstation. The added benefit is that you can test all the commands that we will explore later from your shell, rather than having to test them on a dedicated CI server. -## Register docker runner +## Register Docker Runner -To use GitLab Runner with docker you need to register a new runner to use the -`docker` executor: +To use GitLab Runner with Docker you need to [register a new Runner][register] +to use the `docker` executor. + +A one-line example can be seen below: ```bash -gitlab-ci-multi-runner register \ - --url "https://gitlab.com/" \ +sudo gitlab-runner register \ + --url "https://gitlab.example.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ --executor "docker" \ @@ -33,26 +35,26 @@ gitlab-ci-multi-runner register \ --docker-mysql latest ``` -The registered runner will use the `ruby:2.1` docker image and will run two +The registered runner will use the `ruby:2.1` Docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. ## What is an image -The `image` keyword is the name of the docker image the docker executor -will run to perform the CI tasks. +The `image` keyword is the name of the Docker image the Docker executor +will run to perform the CI tasks. -By default the executor will only pull images from [Docker Hub][hub], +By default, the executor will only pull images from [Docker Hub][hub], but this can be configured in the `gitlab-runner/config.toml` by setting -the [docker pull policy][] to allow using local images. +the [Docker pull policy][] to allow using local images. For more information about images and Docker Hub please read the [Docker Fundamentals][] documentation. ## What is a service -The `services` keyword defines just another docker image that is run during -your job and is linked to the docker image that the `image` keyword defines. +The `services` keyword defines just another Docker image that is run during +your job and is linked to the Docker image that the `image` keyword defines. This allows you to access the service image during build time. The service image can run any application, but the most common use case is to @@ -60,6 +62,11 @@ run a database container, eg. `mysql`. It's easier and faster to use an existing image and run it as an additional container than install `mysql` every time the project is built. +You are not limited to have only database services. You can add as many +services you need to `.gitlab-ci.yml` or manually modify `config.toml`. +Any image found at [Docker Hub][hub] or your private Container Registry can be +used as a service. + You can see some widely used services examples in the relevant documentation of [CI services examples](../services/README.md). @@ -73,22 +80,49 @@ then be used to create a container that is linked to the job container. The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host -named `mysql` instead of a socket or `localhost`. +named `mysql` instead of a socket or `localhost`. Read more in [accessing the +services](#accessing-the-services). -## Overwrite image and services +### Accessing the services -See [How to use other images as services](#how-to-use-other-images-as-services). +Let's say that you need a Wordpress instance to test some API integration with +your application. -## How to use other images as services +You can then use for example the [tutum/wordpress][] image in your +`.gitlab-ci.yml`: -You are not limited to have only database services. You can add as many -services you need to `.gitlab-ci.yml` or manually modify `config.toml`. -Any image found at [Docker Hub][hub] can be used as a service. +```yaml +services: +- tutum/wordpress:latest +``` + +If you don't [specify a service alias](#available-settings-for-services-entry), +when the job is run, `tutum/wordpress` will be started and you will have +access to it from your build container under two hostnames to choose from: -## Define image and services from `.gitlab-ci.yml` +- `tutum-wordpress` +- `tutum__wordpress` + +>**Note:** +Hostnames with underscores are not RFC valid and may cause problems in 3rd party +applications. + +The default aliases for the service's hostname are created from its image name +following these rules: + +- Everything after the colon (`:`) is stripped +- Slash (`/`) is replaced with double underscores (`__`) and the primary alias + is created +- Slash (`/`) is replaced with a single dash (`-`) and the secondary alias is + created (requires GitLab Runner v1.1.0 or higher) + +To override the default behavior, you can +[specify a service alias](#available-settings-for-services-entry). + +## Define `image` and `services` from `.gitlab-ci.yml` You can simply define an image that will be used for all jobs and a list of -services that you want to use during build time. +services that you want to use during build time: ```yaml image: ruby:2.2 @@ -125,6 +159,203 @@ test:2.2: - bundle exec rake spec ``` +Or you can pass some [extended configuration options](#extended-docker-configuration-options) +for `image` and `services`: + +```yaml +image: + name: ruby:2.2 + entrypoint: ["/bin/bash"] + +services: +- name: my-postgres:9.4 + alias: db-postgres + entrypoint: ["/usr/local/bin/db-postgres"] + command: ["start"] + +before_script: +- bundle install + +test: + script: + - bundle exec rake spec +``` + +## Extended Docker configuration options + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +When configuring the `image` or `services` entries, you can use a string or a map as +options: + +- when using a string as an option, it must be the full name of the image to use + (including the Registry part if you want to download the image from a Registry + other than Docker Hub) +- when using a map as an option, then it must contain at least the `name` + option, which is the same name of the image as used for the string setting + +For example, the following two definitions are equal: + +1. Using a string as an option to `image` and `services`: + + ```yaml + image: "registry.example.com/my/image:latest" + + services: + - postgresql:9.4 + - redis:latest + ``` + +1. Using a map as an option to `image` and `services`. The use of `image:name` is + required: + + ```yaml + image: + name: "registry.example.com/my/image:latest" + + services: + - name: postgresql:9.4 + - name: redis:latest + ``` + +### Available settings for `image` + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +| Setting | Required | Description | +|------------|----------|-------------| +| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | + +### Available settings for `services` + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +| Setting | Required | Description | +|------------|----------|-------------| +| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | +| `command` | no | Command or script that should be used as the container's command. It will be translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`][cmd] directive, where each shell token is a separate string in the array. | +| `alias` | no | Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | + +### Starting multiple services from the same image + +Before the new extended Docker configuration options, the following configuration +would not work properly: + +```yaml +services: +- mysql:latest +- mysql:latest +``` + +The Runner would start two containers using the `mysql:latest` image, but both +of them would be added to the job's container with the `mysql` alias based on +the [default hostname naming](#accessing-the-services). This would end with one +of the services not being accessible. + +After the new extended Docker configuration options, the above example would +look like: + +```yaml +services: +- name: mysql:latest + alias: mysql-1 +- name: mysql:latest + alias: mysql-2 +``` + +The Runner will still start two containers using the `mysql:latest` image, +but now each of them will also be accessible with the alias configured +in `.gitlab-ci.yml` file. + +### Setting a command for the service + +Let's assume you have a `super/sql:latest` image with some SQL database +inside it and you would like to use it as a service for your job. Let's also +assume that this image doesn't start the database process while starting +the container and the user needs to manually use `/usr/bin/super-sql run` as +a command to start the database. + +Before the new extended Docker configuration options, you would need to create +your own image based on the `super/sql:latest` image, add the default command, +and then use it in job's configuration, like: + +```Dockerfile +# my-super-sql:latest image's Dockerfile + +FROM super/sql:latest +CMD ["/usr/bin/super-sql", "run"] +``` + +```yaml +# .gitlab-ci.yml + +services: +- my-super-sql:latest +``` + +After the new extended Docker configuration options, you can now simply +set a `command` in `.gitlab-ci.yml`, like: + +```yaml +# .gitlab-ci.yml + +services: +- name: super/sql:latest + command: ["/usr/bin/super-sql", "run"] +``` + +As you can see, the syntax of `command` is similar to [Dockerfile's `CMD`][cmd]. + +### Overriding the entrypoint of an image + +Let's assume you have a `super/sql:experimental` image with some SQL database +inside it and you would like to use it as a base image for your job because you +want to execute some tests with this database binary. Let's also assume that +this image is configured with `/usr/bin/super-sql run` as an entrypoint. That +means, that when starting the container without additional options, it will run +the database's process, while Runner expects that the image will have no +entrypoint or at least will start with a shell as its entrypoint. + +Previously we would need to create our own image based on the +`super/sql:experimental` image, set the entrypoint to a shell, and then use +it in job's configuration, e.g.: + +Before the new extended Docker configuration options, you would need to create +your own image based on the `super/sql:experimental` image, set the entrypoint +to a shell and then use it in job's configuration, like: + +```Dockerfile +# my-super-sql:experimental image's Dockerfile + +FROM super/sql:experimental +ENTRYPOINT ["/bin/sh"] +``` + +```yaml +# .gitlab-ci.yml + +image: my-super-sql:experimental +``` + +After the new extended Docker configuration options, you can now simply +set an `entrypoint` in `.gitlab-ci.yml`, like: + +```yaml +# .gitlab-ci.yml + +image: + name: super/sql:experimental + entrypoint: ["/bin/sh"] +``` + +As you can see the syntax of `entrypoint` is similar to +[Dockerfile's `ENTRYPOINT`][entrypoint]. + ## Define image and services in `config.toml` Look for the `[runners.docker]` section: @@ -138,7 +369,7 @@ Look for the `[runners.docker]` section: The image and services defined this way will be added to all job run by that runner. -## Define an image from a private Docker registry +## Define an image from a private Container Registry > **Notes:** - This feature requires GitLab Runner **1.8** or higher @@ -193,44 +424,6 @@ To configure access for `registry.example.com`, follow these steps: You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. -## Accessing the services - -Let's say that you need a Wordpress instance to test some API integration with -your application. - -You can then use for example the [tutum/wordpress][] image in your -`.gitlab-ci.yml`: - -```yaml -services: -- tutum/wordpress:latest -``` - -When the job is run, `tutum/wordpress` will be started and you will have -access to it from your build container under the hostnames `tutum-wordpress` -(requires GitLab Runner v1.1.0 or newer) and `tutum__wordpress`. - -When using a private registry, the image name also includes a hostname and port -of the registry. - -```yaml -services: -- docker.example.com:5000/wordpress:latest -``` - -The service hostname will also include the registry hostname. Service will be -available under hostnames `docker.example.com-wordpress` (requires GitLab Runner v1.1.0 or newer) -and `docker.example.com__wordpress`. - -*Note: hostname with underscores is not RFC valid and may cause problems in 3rd party applications.* - -The alias hostnames for the service are made from the image name following these -rules: - -1. Everything after `:` is stripped -2. Slash (`/`) is replaced with double underscores (`__`) - primary alias -3. Slash (`/`) is replaced with dash (`-`) - secondary alias, requires GitLab Runner v1.1.0 or newer - ## Configuring services Many services accept environment variables which allow you to easily change @@ -257,7 +450,7 @@ See the specific documentation for ## How Docker integration works -Below is a high level overview of the steps performed by docker during job +Below is a high level overview of the steps performed by Docker during job time. 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. @@ -274,7 +467,7 @@ time. ## How to debug a job locally *Note: The following commands are run without root privileges. You should be -able to run docker with your regular user account.* +able to run Docker with your regular user account.* First start with creating a file named `build_script`: @@ -334,3 +527,6 @@ creation. [mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: http://docs.gitlab.com/runner/configuration/advanced-configuration.html#using-a-private-container-registry [secret variable]: ../variables/README.md#secret-variables +[entrypoint]: https://docs.docker.com/engine/reference/builder/#entrypoint +[cmd]: https://docs.docker.com/engine/reference/builder/#cmd +[register]: https://docs.gitlab.com/runner/register/ diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 8b0d8a003fd..b9f0485290e 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -20,12 +20,12 @@ before_script: - php -r "unlink('composer-setup.php');" ``` -This will make sure we have all requirements ready. Next, we want to run `composer update` to fetch all PHP dependencies and `npm install` to load node packages, then run the `npm` script. We need to append them into `before_script` section: +This will make sure we have all requirements ready. Next, we want to run `composer install` to fetch all PHP dependencies and `npm install` to load node packages, then run the `npm` script. We need to append them into `before_script` section: ```yaml before_script: # ... - - php composer.phar update + - php composer.phar install - npm install - npm run deploy ``` @@ -133,7 +133,7 @@ before_script: - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" - php composer-setup.php - php -r "unlink('composer-setup.php');" - - php composer.phar update + - php composer.phar install - npm install - npm run deploy - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 41cae58782d..88e53ff40e8 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -155,7 +155,7 @@ Find more information about different Runners in the [Runners](../runners/README.md) documentation. You can find whether any Runners are assigned to your project by going to -**Settings ➔ CI/CD Pipelines**. Setting up a Runner is easy and straightforward. The +**Settings ➔ Pipelines**. Setting up a Runner is easy and straightforward. The official Runner supported by GitLab is written in Go and its documentation can be found at <https://docs.gitlab.com/runner/>. @@ -168,7 +168,7 @@ Follow the links above to set up your own Runner or use a Shared Runner as described in the next section. Once the Runner has been set up, you should see it on the Runners page of your -project, following **Settings ➔ CI/CD Pipelines**. +project, following **Settings ➔ Pipelines**.  @@ -181,7 +181,7 @@ These are special virtual machines that run on GitLab's infrastructure and can build any project. To enable the **Shared Runners** you have to go to your project's -**Settings ➔ CI/CD Pipelines** and click **Enable shared runners**. +**Settings ➔ Pipelines** and click **Enable shared runners**. [Read more on Shared Runners](../runners/README.md). diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index befaa06e918..cf25a8b618f 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -34,9 +34,9 @@ instructions to [generate an SSH key](../../ssh/README.md). Do not add a passphrase to the SSH key, or the `before_script` will prompt for it. Then, create a new **Secret Variable** in your project settings on GitLab -following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` -and in the **Value** field paste the content of your _private_ key that you -created earlier. +following **Settings > Pipelines** and look for the "Secret Variables" section. +As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the +content of your _private_ key that you created earlier. It is also good practice to check the server's own public key to make sure you are not being targeted by a man-in-the-middle attack. To do this, add another diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index d1f9881e51b..22e7f6879ed 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -9,8 +9,9 @@ and a list of **user-defined variables**. The variables can be overwritten and they take precedence over each other in this order: -1. [Trigger variables][triggers] (take precedence over all) -1. [Secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) +1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) 1. YAML-defined [job-level variables](../yaml/README.md#job-variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) @@ -37,9 +38,10 @@ future GitLab releases.** |-------------------------------- |--------|--------|-------------| | **CI** | all | 0.4 | Mark that job is executed in CI environment | | **CI_COMMIT_REF_NAME** | 9.0 | all | The branch or tag name for which project is built | -| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | +| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | **CI_COMMIT_SHA** | 9.0 | all | The commit revision for which project is built | | **CI_COMMIT_TAG** | 9.0 | 0.5 | The commit tag name. Present only when building tags. | +| **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | | **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job | | **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | @@ -141,25 +143,30 @@ script: >**Notes:** - This feature requires GitLab Runner 0.4.0 or higher. +- Group-level secret variables added in GitLab 9.4. - Be aware that secret variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your project's Pipelines settings. Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. -GitLab CI allows you to define per-project **secret variables** that are set in -the build environment. The secret variables are stored out of the repository -(`.gitlab-ci.yml`) and are securely passed to GitLab Runner making them -available in the build environment. It's the recommended method to use for -storing things like passwords, secret keys and credentials. +GitLab CI allows you to define per-project or per-group **secret variables** +that are set in the build environment. The secret variables are stored out of +the repository (`.gitlab-ci.yml`) and are securely passed to GitLab Runner +making them available in the build environment. It's the recommended method to +use for storing things like passwords, secret keys and credentials. -Secret variables can be added by going to your project's -**Settings ➔ Pipelines**, then finding the section called -**Secret variables**. +Project-level secret variables can be added by going to your project's +**Settings ➔ Pipelines**, then finding the section called **Secret variables**. -Once you set them, they will be available for all subsequent pipelines. +Likewise, group-level secret variables can be added by going to your group's +**Settings ➔ Pipelines**, then finding the section called **Secret variables**. +Any variables of [subgroups] will be inherited recursively. + +Once you set them, they will be available for all subsequent pipelines. You can also +[protect your variables](#protected-secret-variables). -## Protected secret variables +### Protected secret variables >**Notes:** This feature requires GitLab 9.3 or higher. @@ -425,10 +432,12 @@ export CI_REGISTRY_PASSWORD="longalfanumstring" ``` [ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 -[runner]: https://docs.gitlab.com/runner/ -[triggered]: ../triggers/README.md -[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger +[eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" +[envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md [protected tags]: ../../user/project/protected_tags.md +[runner]: https://docs.gitlab.com/runner/ [shellexecutors]: https://docs.gitlab.com/runner/executors/ -[eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" +[triggered]: ../triggers/README.md +[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger +[subgroups]: ../../user/group/subgroups/index.md diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 8a0662db6fd..724843a4d56 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -306,6 +306,53 @@ cache: untracked: true ``` +### cache:policy + +> Introduced in GitLab 9.4. + +The default behaviour of a caching job is to download the files at the start of +execution, and to re-upload them at the end. This allows any changes made by the +job to be persisted for future runs, and is known as the `pull-push` cache +policy. + +If you know the job doesn't alter the cached files, you can skip the upload step +by setting `policy: pull` in the job specification. Typically, this would be +twinned with an ordinary cache job at an earlier stage to ensure the cache +is updated from time to time: + +```yaml +stages: + - setup + - test + +prepare: + stage: setup + cache: + key: gems + paths: + - vendor/bundle + script: + - bundle install --deployment + +rspec: + stage: test + cache: + key: gems + paths: + - vendor/bundle + policy: pull + script: + - bundle exec rspec ... +``` + +This helps to speed up job execution and reduce load on the cache server, +especially when you have a large number of cache-using jobs executing in +parallel. + +Additionally, if you have a job that unconditionally recreates the cache without +reference to its previous contents, you can use `policy: push` in that job to +skip the download step. + ## Jobs `.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job diff --git a/doc/development/README.md b/doc/development/README.md index 9496a87d84d..58993c52dcd 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -54,6 +54,8 @@ - [Polymorphic Associations](polymorphic_associations.md) - [Single Table Inheritance](single_table_inheritance.md) - [Background Migrations](background_migrations.md) +- [Storing SHA1 Hashes As Binary](sha1_as_binary.md) +- [Iterating Tables In Batches](iterating_tables_in_batches.md) ## i18n diff --git a/doc/development/architecture.md b/doc/development/architecture.md index acd5e3c2093..54029e00507 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -194,3 +194,7 @@ bundle exec rake gitlab:check RAILS_ENV=production ``` Note: It is recommended to log into the `git` user using `sudo -i -u git` or `sudo su - git`. While the sudo commands provided by gitlabhq work in Ubuntu they do not always work in RHEL. + +## GitLab.com + +We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/handbook/infrastructure/production-architecture/) but this is probably over the top unless you have millions of users. diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index d2d89517241..ae844fa1051 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -463,20 +463,24 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `destroyed` #### Vue and Boostrap -1. Tooltips: Do not rely on `has-tooltip` class name for vue components +1. Tooltips: Do not rely on `has-tooltip` class name for Vue components ``` javascript // bad - <span class="has-tooltip"> + <span + class="has-tooltip" + title="Some tooltip text"> Text </span> // good - <span data-toggle="tooltip"> + <span + v-tooltip + title="Some tooltip text"> Text </span> ``` -1. Tooltips: When using a tooltip, include the tooltip mixin +1. Tooltips: When using a tooltip, include the tooltip directive, `./app/assets/javascripts/vue_shared/directives/tooltip.js` 1. Don't change `data-original-title`. ```javascript diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 5c6316b9ac6..59e8a087e02 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -3,5 +3,19 @@ Starting from GitLab 9.3 we support feature flags via [Flipper](https://github.com/jnunemaker/flipper/). You should use the `Feature` class (defined in `lib/feature.rb`) in your code to get, set and list feature -flags. During runtime you can set the values for the gates via the -[admin API](../api/features.md). +flags. + +During runtime you can set the values for the gates via the +[features API](../api/features.md) (accessible to admins only). + +## Feature groups + +Starting from GitLab 9.4 we support feature groups via +[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group). + +Feature groups must be defined statically in `lib/feature.rb` (in the +`.register_feature_groups` method), but their implementation can obviously be +dynamic (querying the DB etc.). + +Once defined in `lib/feature.rb`, you will be able to activate a +feature for a given feature group via the [`feature_group` param of the features API](../api/features.md#set-or-create-a-feature) diff --git a/doc/development/iterating_tables_in_batches.md b/doc/development/iterating_tables_in_batches.md new file mode 100644 index 00000000000..590c8cbba2d --- /dev/null +++ b/doc/development/iterating_tables_in_batches.md @@ -0,0 +1,37 @@ +# Iterating Tables In Batches + +Rails provides a method called `in_batches` that can be used to iterate over +rows in batches. For example: + +```ruby +User.in_batches(of: 10) do |relation| + relation.update_all(updated_at: Time.now) +end +``` + +Unfortunately this method is implemented in a way that is not very efficient, +both query and memory usage wise. + +To work around this you can include the `EachBatch` module into your models, +then use the `each_batch` class method. For example: + +```ruby +class User < ActiveRecord::Base + include EachBatch +end + +User.each_batch(of: 10) do |relation| + relation.update_all(updated_at: Time.now) +end +``` + +This will end up producing queries such as: + +``` +User Load (0.7ms) SELECT "users"."id" FROM "users" WHERE ("users"."id" >= 41654) ORDER BY "users"."id" ASC LIMIT 1 OFFSET 1000 + (0.7ms) SELECT COUNT(*) FROM "users" WHERE ("users"."id" >= 41654) AND ("users"."id" < 42687) +``` + +The API of this method is similar to `in_batches`, though it doesn't support +all of the arguments that `in_batches` supports. You should always use +`each_batch` _unless_ you have a specific need for `in_batches`. diff --git a/doc/development/policies.md b/doc/development/policies.md new file mode 100644 index 00000000000..62141356f59 --- /dev/null +++ b/doc/development/policies.md @@ -0,0 +1,116 @@ +# `DeclarativePolicy` framework + +The DeclarativePolicy framework is designed to assist in performance of policy checks, and to enable ease of extension for EE. The DSL code in `app/policies` is what `Ability.allowed?` uses to check whether a particular action is allowed on a subject. + +The policy used is based on the subject's class name - so `Ability.allowed?(user, :some_ability, project)` will create a `ProjectPolicy` and check permissions on that. + +## Managing Permission Rules + +Permissions are broken into two parts: `conditions` and `rules`. Conditions are boolean expressions that can access the database and the environment, while rules are statically configured combinations of expressions and other rules that enable or prevent certain abilities. For an ability to be allowed, it must be enabled by at least one rule, and not prevented by any. + + +### Conditions + +Conditions are defined by the `condition` method, and are given a name and a block. The block will be executed in the context of the policy object - so it can access `@user` and `@subject`, as well as call any methods defined on the policy. Note that `@user` may be nil (in the anonymous case), but `@subject` is guaranteed to be a real instance of the subject class. + +``` ruby +class FooPolicy < BasePolicy + condition(:is_public) do + # @subject guaranteed to be an instance of Foo + @subject.public? + end + + # instance methods can be called from the condition as well + condition(:thing) { check_thing } + + def check_thing + # ... + end +end +``` + +When you define a condition, a predicate method is defined on the policy to check whether that condition passes - so in the above example, an instance of `FooPolicy` will also respond to `#is_public?` and `#thing?`. + +Conditions are cached according to their scope. Scope and ordering will be covered later. + +### Rules + +A `rule` is a logical combination of conditions and other rules, that are configured to enable or prevent certain abilities. It is important to note that the rule configuration is static - a rule's logic cannot touch the database or know about `@user` or `@subject`. This allows us to cache only at the condition level. Rules are specified through the `rule` method, which takes a block of DSL configuration, and returns an object that responds to `#enable` or `#prevent`: + +``` ruby +class FooPolicy < BasePolicy + # ... + + rule { is_public }.enable :read + rule { thing }.prevent :read + + # equivalently, + rule { is_public }.policy do + enable :read + end + + rule { ~thing }.policy do + prevent :read + end +end +``` + +Within the rule DSL, you can use: + +* A regular word mentions a condition by name - a rule that is in effect when that condition is truthy. +* `~` indicates negation +* `&` and `|` are logical combinations, also available as `all?(...)` and `any?(...)` +* `can?(:other_ability)` delegates to the rules that apply to `:other_ability`. Note that this is distinct from the instance method `can?`, which can check dynamically - this only configures a delegation to another ability. + +## Scores, Order, Performance + +To see how the rules get evaluated into a judgment, it is useful in a console to use `policy.debug(:some_ability)`. This will print the rules in the order they are evaluated. + +When a policy is asked whether a particular ability is allowed (`policy.allowed?(:some_ability)`), it does not necessarily have to compute all the conditions on the policy. First, only the rules relevant to that particular ability are selected. Then, the execution model takes advantage of short-circuiting, and attempts to sort rules based on a heuristic of how expensive they will be to calculate. The sorting is dynamic and cache-aware, so that previously calculated conditions will be considered first, before computing other conditions. + +## Scope + +Sometimes, a condition will only use data from `@user` or only from `@subject`. In this case, we want to change the scope of the caching, so that we don't recalculate conditions unnecessarily. For example, given: + +``` ruby +class FooPolicy < BasePolicy + condition(:expensive_condition) { @subject.expensive_query? } + + rule { expensive_condition }.enable :some_ability +end +``` + +Naively, if we call `Ability.can?(user1, :some_ability, foo)` and `Ability.can?(user2, :some_ability, foo)`, we would have to calculate the condition twice - since they are for different users. But if we use the `scope: :subject` option: + +``` ruby + condition(:expensive_condition, scope: :subject) { @subject.expensive_query? } +``` + +then the result of the condition will be cached globally only based on the subject - so it will not be calculated repeatedly for different users. Similarly, `scope: :user` will cache only based on the user. + +**DANGER**: If you use a `:scope` option when the condition actually uses data from +both user and subject (including a simple anonymous check!) your result will be cached at too global of a scope and will result in cache bugs. + +Sometimes we are checking permissions for a lot of users for one subject, or a lot of subjects for one user. In this case, we want to set a *preferred scope* - i.e. tell the system that we prefer rules that can be cached on the repeated parameter. For example, in `Ability.users_that_can_read_project`: + +``` ruby +def users_that_can_read_project(users, project) + DeclarativePolicy.subject_scope do + users.select { |u| allowed?(u, :read_project, project) } + end +end +``` + +This will, for example, prefer checking `project.public?` to checking `user.admin?`. + +## Delegation + +Delegation is the inclusion of rules from another policy, on a different subject. For example, + +``` ruby +class FooPolicy < BasePolicy + delegate { @subject.project } +end +``` + +will include all rules from `ProjectPolicy`. The delegated conditions will be evaluated with the correct delegated subject, and will be sorted along with the regular rules in the policy. Note that only the relevant rules for a particular ability will actually be considered. diff --git a/doc/development/sha1_as_binary.md b/doc/development/sha1_as_binary.md new file mode 100644 index 00000000000..3151cc29bbc --- /dev/null +++ b/doc/development/sha1_as_binary.md @@ -0,0 +1,36 @@ +# Storing SHA1 Hashes As Binary + +Storing SHA1 hashes as strings is not very space efficient. A SHA1 as a string +requires at least 40 bytes, an additional byte to store the encoding, and +perhaps more space depending on the internals of PostgreSQL and MySQL. + +On the other hand, if one were to store a SHA1 as binary one would only need 20 +bytes for the actual SHA1, and 1 or 4 bytes of additional space (again depending +on database internals). This means that in the best case scenario we can reduce +the space usage by 50%. + +To make this easier to work with you can include the concern `ShaAttribute` into +a model and define a SHA attribute using the `sha_attribute` class method. For +example: + +```ruby +class Commit < ActiveRecord::Base + include ShaAttribute + + sha_attribute :sha +end +``` + +This allows you to use the value of the `sha` attribute as if it were a string, +while storing it as binary. This means that you can do something like this, +without having to worry about converting data to the right binary format: + +```ruby +commit = Commit.find_by(sha: '88c60307bd1f215095834f09a1a5cb18701ac8ad') +commit.sha = '971604de4cfa324d91c41650fabc129420c8d1cc' +commit.save +``` + +There is however one requirement: the column used to store the SHA has _must_ be +a binary type. For Rails this means you need to use the `:binary` type instead +of `:text` or `:string`. diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index fe4b6d73771..75bae324585 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -46,6 +46,19 @@ $ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDepre $ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" production ``` +### Secret variables environment scopes + +If you're using this feature and there are variables sharing the same +key, but they have different scopes in a project, then you might want to +revisit the environment scope setting for those variables. + +In CE, environment scopes are completely ignored, therefore you could +accidentally get a variable which you're not expecting for a particular +environment. Make sure that you have the right variables in this case. + +Data is completely preserved, so you could always upgrade back to EE and +restore the behavior if you leave it alone. + ## Downgrade to CE After performing the above mentioned steps, you are now ready to downgrade your diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 9a171d34671..37e9b3101ca 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -43,7 +43,7 @@ mysql> SET GLOBAL innodb_file_per_table=1, innodb_file_format=Barracuda, innodb_ mysql> CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_general_ci`; # Grant the GitLab user necessary permissions on the database -mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES ON `gitlabhq_production`.* TO 'git'@'localhost'; +mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES, TRIGGER ON `gitlabhq_production`.* TO 'git'@'localhost'; # Quit the database session mysql> \q diff --git a/doc/install/installation.md b/doc/install/installation.md index 84af6432889..992ff162efb 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -294,9 +294,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-2-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-3-stable gitlab -**Note:** You can change `9-2-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `9-3-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index b8bc0795f2e..515b2841d08 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -54,6 +54,13 @@ gitlabURL: http://gitlab.your-domain.com/ ## runnerRegistrationToken: "" +## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Provide resource name for a Kubernetes Secret Object in the same namespace, +## this is used to populate the /etc/gitlab-runner/certs directory +## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates +## +#certsSecretName: + ## Configure the maximum number of concurrent jobs ## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section ## @@ -135,6 +142,52 @@ runners: privileged: true ``` +### Providing a custom certificate for accessing GitLab + +You can provide a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) +to the GitLab Runner Helm Chart, which will be used to populate the container's +`/etc/gitlab-runner/certs` directory. + +Each key name in the Secret will be used as a filename in the directory, with the +file content being the value associated with the key. + +More information on how GitLab Runner uses these certificates can be found in the +[Runner Documentation](https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates). + + - The key/file name used should be in the format `<gitlab-hostname>.crt`. For example: `gitlab.your-domain.com.crt`. + - Any intermediate certificates need to be concatenated to your server certificate in the same file. + - The hostname used should be the one the certificate is registered for. + +The GitLab Runner Helm Chart does not create a secret for you. In order to create +the secret, you can prepare your certificate on you local machine, and then run +the `kubectl create secret` command from the directory with the certificate + +```bash +kubectl + --namespace <NAMESPACE> + create secret generic <SECRET_NAME> + --from-file=<CERTFICATE_FILENAME> +``` + +- `<NAMESPACE>` is the Kubernetes namespace where you want to install the GitLab Runner. +- `<SECRET_NAME>` is the Kubernetes Secret resource name. For example: `gitlab-domain-cert` +- `<CERTFICATE_FILENAME>` is the filename for the certificate in your current directory that will be imported into the secret + +You then need to provide the secret's name to the GitLab Runner chart. + +Add the following to your `values.yaml` + +```yaml +## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Provide resource name for a Kubernetes Secret Object in the same namespace, +## this is used to populate the /etc/gitlab-runner/certs directory +## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates +## +certsSecretName: <SECRET NAME> +``` + +- `<SECRET_NAME>` is the Kubernetes Secret resource name. For example: `gitlab-domain-cert` + ## Installing GitLab Runner using the Helm Chart Once you [have configured](#configuration) GitLab Runner in your `values.yml` file, diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 643fe5b686b..a3d676433e6 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -95,15 +95,16 @@ installation (e.g. the number of users, projects, etc). We currently support the following databases: - PostgreSQL (highly recommended) -- MySQL/MariaDB (doesn't support all features) +- MySQL/MariaDB (strongly discouraged, not all GitLab features are supported, no support for [MySQL/MariaDB GTID](https://mariadb.com/kb/en/mariadb/gtid/)) -We **highly recommend** the use of PostgreSQL instead of MySQL/MariaDB as not all +We highly recommend the use of PostgreSQL instead of MySQL/MariaDB as not all features of GitLab work with MySQL/MariaDB: 1. MySQL support for subgroups was [dropped with GitLab 9.3][post]. See [issue #30472][30472] for more information. 1. GitLab Geo does [not support MySQL](https://docs.gitlab.com/ee/gitlab-geo/database.html#mysql-replication). 1. [Zero downtime migrations][zero] do not work with MySQL +1. We expect this list to grow over time. Existing users using GitLab with MySQL/MariaDB are advised to [migrate to PostgreSQL](../update/mysql_to_postgresql.md) instead. diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 265c891cf83..2dd9b33273c 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -8,6 +8,9 @@ you to do the following: issue index of the external tracker - clicking **New issue** on the project dashboard creates a new issue on the external tracker +- you can reference these external issues inside GitLab interface + (merge requests, commits, comments) and they will be automatically converted + into links ## Configuration diff --git a/doc/update/8.9-to-8.10.md b/doc/update/8.9-to-8.10.md index d6b2f11d49a..42132f690d8 100644 --- a/doc/update/8.9-to-8.10.md +++ b/doc/update/8.9-to-8.10.md @@ -156,7 +156,7 @@ See [smtp_settings.rb.sample] as an example. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab - + For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index e7d97fde14e..225a4dcc924 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -70,7 +70,27 @@ curl --location https://yarnpkg.com/install.sh | bash - More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). -### 5. Get latest code +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code ```bash cd /home/git/gitlab @@ -97,7 +117,7 @@ cd /home/git/gitlab sudo -u git -H git checkout 9-2-stable-ee ``` -### 6. Update gitlab-shell +### 7. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -107,11 +127,10 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 7. Update gitlab-workhorse +### 8. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires -[Go 1.8](https://golang.org/dl). Go (at least 1.5) should already be on your system from -GitLab 8.1 and shall be upgraded if necessary. Please note that starting in Gitlab 9.3, only Go 1.8.3 and above will be supported. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). If you are not using Linux you may have to run `gmake` instead of `make` below. @@ -123,7 +142,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 8. Update configuration files +### 9. Update configuration files #### New configuration options for `gitlab.yml` @@ -197,7 +216,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 9. Install libs, migrations, etc. +### 10. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -223,7 +242,7 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 10. Optional: install Gitaly +### 11. Optional: install Gitaly Gitaly is still an optional component of GitLab. If you want to save time during your 9.2 upgrade **you can skip this step**. @@ -240,14 +259,14 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 11. Start application +### 12. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 12. Check application status +### 13. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 0c32e4db53f..097b996ec31 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -72,8 +72,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.3 and higher only supports Go 1.8.3 and dropped support for Go 1.5.x through 1.7.x. Be -sure to upgrade your installation if necessary +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -117,7 +117,7 @@ cd /home/git/gitlab sudo -u git -H git checkout 9-3-stable-ee ``` -### 5. Update gitlab-shell +### 7. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -127,11 +127,10 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 6. Update gitlab-workhorse +### 8. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires -[Go 1.5](https://golang.org/dl) which should already be on your system from -GitLab 8.1. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). If you are not using Linux you may have to run `gmake` instead of `make` below. @@ -143,7 +142,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 7. Update Gitaly +### 9. Update Gitaly If you have not yet set up Gitaly then follow [Gitaly section of the installation guide](../install/installation.md#install-gitaly). @@ -157,7 +156,16 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 10. Update configuration files +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +### 11. Update configuration files #### New configuration options for `gitlab.yml` @@ -231,7 +239,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 11. Install libs, migrations, etc. +### 12. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -257,14 +265,14 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 12. Start application +### 13. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 13. Check application status +### 14. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index a712ce5a8b1..bbb7f4a8d48 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -148,6 +148,8 @@ sudo -u git -H make If you have not yet set up Gitaly then follow [Gitaly section of the installation guide](../install/installation.md#install-gitaly). +As of GitLab 9.4, Gitaly is a mandatory component of GitLab. + #### Check Gitaly configuration Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 9488ce1ef30..f28c034e74c 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -14,6 +14,9 @@ accepted method of authentication when you have Once you have your token, [pass it to the API][usage] using either the `private_token` parameter or the `PRIVATE-TOKEN` header. +The expiration of personal access tokens happens on the date you define, +at midnight UTC. + ## Creating a personal access token You can create as many personal access tokens as you like from your GitLab diff --git a/doc/user/project/img/issue_board.png b/doc/user/project/img/issue_board.png Binary files differindex b636cb294b8..cf7f519f783 100644 --- a/doc/user/project/img/issue_board.png +++ b/doc/user/project/img/issue_board.png diff --git a/doc/user/project/img/issue_board_add_list.png b/doc/user/project/img/issue_board_add_list.png Binary files differindex cdfc466d23f..973d9f7cde4 100644 --- a/doc/user/project/img/issue_board_add_list.png +++ b/doc/user/project/img/issue_board_add_list.png diff --git a/doc/user/project/img/issue_board_move_issue_card_list.png b/doc/user/project/img/issue_board_move_issue_card_list.png Binary files differnew file mode 100644 index 00000000000..c6b17ada40e --- /dev/null +++ b/doc/user/project/img/issue_board_move_issue_card_list.png diff --git a/doc/user/project/img/issue_board_welcome_message.png b/doc/user/project/img/issue_board_welcome_message.png Binary files differindex 5318e6ea4a9..127b9b08cc7 100644 --- a/doc/user/project/img/issue_board_welcome_message.png +++ b/doc/user/project/img/issue_board_welcome_message.png diff --git a/doc/user/project/img/issue_boards_add_issues_modal.png b/doc/user/project/img/issue_boards_add_issues_modal.png Binary files differindex 33049dce74f..bedaf724a15 100644 --- a/doc/user/project/img/issue_boards_add_issues_modal.png +++ b/doc/user/project/img/issue_boards_add_issues_modal.png diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 0b219e84478..6a040516231 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -16,3 +16,14 @@ Once you have configured and enabled Bugzilla: - the **Issues** link on the GitLab project pages takes you to the appropriate Bugzilla product page - clicking **New issue** on the project dashboard takes you to Bugzilla for entering a new issue + +## Referencing issues in Bugzilla + +Issues in Bugzilla can be referenced in two alternative ways: +1. `#<ID>` where `<ID>` is a number (example `#143`) +2. `<PROJECT>-<ID>` where `<PROJECT>` starts with a capital letter which is + then followed by capital letters, numbers or underscores, and `<ID>` is + a number (example `API_32-143`). + +Please note that `<PROJECT>` part is ignored and links always point to the +address specified in `issues_url`. diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index 73fa83d72a8..bfe2672e098 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -55,6 +55,7 @@ GitLab CI build environment: - `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path to a file containing PEM data. - `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. +- `KUBECONFIG` - Path to a file containing kubeconfig for this deployment. CA bundle would be embedded if specified. ## Web terminals diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 89c0312d3c2..8026f1f57bc 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -21,3 +21,14 @@ Once you have configured and enabled Redmine: As an example, below is a configuration for a project named gitlab-ci.  + +## Referencing issues in Redmine + +Issues in Redmine can be referenced in two alternative ways: +1. `#<ID>` where `<ID>` is a number (example `#143`) +2. `<PROJECT>-<ID>` where `<PROJECT>` starts with a capital letter which is + then followed by capital letters, numbers or underscores, and `<ID>` is + a number (example `API_32-143`). + +Please note that `<PROJECT>` part is ignored and links always point to the +address specified in `issues_url`. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 0517ed3ec18..023c6932e41 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -736,7 +736,7 @@ X-Gitlab-Event: Merge Request Hook ### Wiki Page events -Triggered when a wiki page is created, edited or deleted. +Triggered when a wiki page is created, updated or deleted. **Request Header**: diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index ebea7062ecb..e2cc67726e0 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -1,8 +1,7 @@ -# Issue board +# Issue Board ->**Notes:** -- [Introduced][ce-5554] in GitLab 8.11. -- The Backlog column was replaced by the **Add issues** button in GitLab 8.17. +>**Note:** +[Introduced][ce-5554] in [GitLab 8.11](https://about.gitlab.com/2016/08/22/gitlab-8-11-released/#issue-board). The GitLab Issue Board is a software project management tool used to plan, organize, and visualize a workflow for a feature or product release. @@ -15,12 +14,65 @@ Other interesting links: ## Overview -The Issue Board builds on GitLab's existing issue tracking functionality and +The Issue Board builds on GitLab's existing +[issue tracking functionality](issues/index.md#issue-tracker) and leverages the power of [labels] by utilizing them as lists of the scrum board. -With the Issue Board you can have a different view of your issues while also +With the Issue Board you can have a different view of your issues while maintaining the same filtering and sorting abilities you see across the -issue tracker. +issue tracker. An Issue Board is based on its project's label structure, therefore, it +applies the same descriptive labels to indicate placement on the board, keeping +consistency throughout the entire development lifecycle. + +An Issue Board shows you what issues your team is working on, who is assigned to each, +and where in the workflow those issues are. + +You create issues, host code, perform reviews, build, test, +and deploy from one single platform. Issue Boards help you to visualize +and manage the entire process _in_ GitLab. + +With [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards), available +only in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/), +you go even further, as you can not only keep yourself and your project +organized from a broader perspective with one Issue Board per project, +but also allow your team members to organize their own workflow by creating +multiple Issue Boards within the same project. + +## Use cases + +GitLab Workflow allows you to discuss proposals in issues, categorize them +with labels, and from there organize and prioritize them with Issue Boards. + +For example, let's consider this simplified development workflow: + +1. You have a repository hosting your app's codebase +and your team actively contributing to code +1. Your **backend** team starts working a new +implementation, gathers feedback and approval, and pass it over to **frontend** +1. When frontend is complete, the new feature is deployed to **staging** to be tested +1. When successful, it is deployed to **production** + +If we have the labels "**backend**", "**frontend**", "**staging**", and +"**production**", and an Issue Board with a list for each, we can: + +- Visualize the entire flow of implementations since the +beginning of the development lifecycle until deployed to production +- Prioritize the issues in a list by moving them vertically +- Move issues between lists to organize them according to the labels you've set +- Add multiple issues to lists in the board by selecting one or more existing issues + + + +> **Notes:** +> +>- For a broader use case, please check the blog post +[GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/#gitlab-workflow-use-case-scenario). +> +>- For a real use case, please check why +[Codepen decided to adopt Issue Boards](https://about.gitlab.com/2017/01/27/codepen-welcome-to-gitlab/#project-management-everything-in-one-place) +to improve their workflow with [multiple boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards). + +## Issue Board terminology Below is a table of the definitions used for GitLab's Issue Board. @@ -57,7 +109,7 @@ In short, here's a list of actions you can take in an Issue Board: If you are not able to perform one or more of the things above, make sure you have the right [permissions](#permissions). -## First time using the issue board +## First time using the Issue Board The first time you navigate to your Issue Board, you will be presented with a default list (**Done**) and a welcoming message that gives @@ -98,7 +150,7 @@ list view that is removed. You can always add it back later if you need. ## Adding issues to a list You can add issues to a list by clicking the **Add issues** button that is -present in the upper right corner of the issue board. This will open up a modal +present in the upper right corner of the Issue Board. This will open up a modal window where you can see all the issues that do not belong to any list. Select one or more issues by clicking on the cards and then click **Add issues** diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index fe87e6f9495..e55e2aea023 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -1,4 +1,4 @@ -# Issues documentation +# Issues The GitLab Issue Tracker is an advanced and complete tool for tracking the evolution of a new idea or the process diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 99233ed5ae2..1848514e2dd 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -21,14 +21,11 @@ Once you fill in all the details, hit the **Create milestone** button. >**Note:** You need [Master permissions](../../permissions.md) in order to create a milestone. -You can create a milestone for several projects in the same group simultaneously. -On the group's **Issues ➔ Milestones** page, you will be able to see the status -of that milestone across all of the selected projects. To create a new milestone -for selected projects in the group, click the **New milestone** button. The -form is the same as when creating a milestone for a specific project with the -addition of the selection of the projects you want to inherit this milestone. - - +You can create a milestone for a group that will be shared across group projects. +On the group's **Issues ➔ Milestones** page, you will be able to see the state +of that milestone and the issues/merge requests count that it shares across the group projects. To create a new milestone click the **New milestone** button. The form is the same as when creating a milestone for a specific project which you can find in the previous item. + +In addition to that you will be able to filter issues or merge requests by group milestones in all projects that belongs to the milestone group. ## Special milestone filters diff --git a/doc/user/project/pipelines/img/pipeline_schedule_variables.png b/doc/user/project/pipelines/img/pipeline_schedule_variables.png Binary files differnew file mode 100644 index 00000000000..47a0c6f3697 --- /dev/null +++ b/doc/user/project/pipelines/img/pipeline_schedule_variables.png diff --git a/doc/user/project/pipelines/img/pipeline_schedules_new_form.png b/doc/user/project/pipelines/img/pipeline_schedules_new_form.png Binary files differindex ea5394fa8a6..5a0e5965992 100644 --- a/doc/user/project/pipelines/img/pipeline_schedules_new_form.png +++ b/doc/user/project/pipelines/img/pipeline_schedules_new_form.png diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index 17cc21238ff..258b3a2f955 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -31,6 +31,15 @@ is installed on.  +### Making use of scheduled pipeline variables + +> [Introduced][ce-12328] in GitLab 9.4. + +You can pass any number of arbitrary variables and they will be available in +GitLab CI so that they can be used in your `.gitlab-ci.yml` file. + + + ## Using only and except To configure that a job can be executed only when the pipeline has been @@ -79,4 +88,5 @@ don't have admin access to the server, ask your administrator. [ce-10533]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10533 [ce-10853]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10853 +[ce-12328]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12328 [settings]: https://about.gitlab.com/gitlab-com/settings/#cron-jobs diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 1d2eba4f74b..3ff5a08d72c 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -1,7 +1,7 @@ # Pipelines settings To reach the pipelines settings navigate to your project's -**Settings ➔ CI/CD Pipelines**. +**Settings ➔ Pipelines**. The following settings can be configured per project. @@ -27,6 +27,22 @@ The default value is 60 minutes. Decrease the time limit if you want to impose a hard limit on your jobs' running time or increase it otherwise. In any case, if the job surpasses the threshold, it is marked as failed. +## Custom CI config path + +> - [Introduced][ce-12509] in GitLab 9.4. + +By default we look for the `.gitlab-ci.yml` file in the project's root +directory. If you require a different location **within** the repository, +you can set a custom filepath that will be used to lookup the config file, +this filepath should be **relative** to the root. + +Here are some valid examples: + +> * .gitlab-ci.yml +> * .my-custom-file.yml +> * my/path/.gitlab-ci.yml +> * my/path/.my-custom-file.yml + ## Test coverage parsing If you use test coverage in your code, GitLab can capture its output in the @@ -59,8 +75,8 @@ pipelines** checkbox and save the changes. > [Introduced][ce-9362] in GitLab 9.1. -If you want to auto-cancel all pending non-HEAD pipelines on branch, when -new pipeline will be created (after your git push or manually from UI), +If you want to auto-cancel all pending non-HEAD pipelines on branch, when +new pipeline will be created (after your git push or manually from UI), check **Auto-cancel pending pipelines** checkbox and save the changes. ## Badges @@ -115,3 +131,4 @@ into your `README.md`: [var]: ../../../ci/yaml/README.md#git-strategy [coverage report]: #test-coverage-parsing [ce-9362]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9362 +[ce-12509]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12509 diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index e10ccc4fc46..ea28968fbb2 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -300,7 +300,7 @@ If there are no merge conflicts and the feature branches are short lived the ris If there are merge conflicts you merge the master branch into the feature branch and the CI server will rerun the tests. If you have long lived feature branches that last for more than a few days you should make your issues smaller. -## Working wih feature branches +## Working with feature branches  |
