diff options
Diffstat (limited to 'doc')
43 files changed, 872 insertions, 147 deletions
diff --git a/doc/README.md b/doc/README.md index 254394eb63e..dd0eb97489e 100644 --- a/doc/README.md +++ b/doc/README.md @@ -29,9 +29,9 @@ - [Install](install/README.md) Requirements, directory structures and installation from source. - [Restart GitLab](administration/restart_gitlab.md) Learn how to restart GitLab and its components. - [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, Twitter. -- [Issue closing](customization/issue_closing.md) Customize how to close an issue from commit messages. +- [Issue closing pattern](administration/issue_closing_pattern.md) Customize how to close an issue from commit messages. - [Koding](administration/integration/koding.md) Set up Koding to use with GitLab. -- [Libravatar](customization/libravatar.md) Use Libravatar for user avatars. +- [Libravatar](customization/libravatar.md) Use Libravatar instead of Gravatar for user avatars. - [Log system](administration/logs.md) Log system. - [Environment Variables](administration/environment_variables.md) to configure GitLab. - [Operations](operations/README.md) Keeping GitLab up and running. diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md new file mode 100644 index 00000000000..28e1fd4e12e --- /dev/null +++ b/doc/administration/issue_closing_pattern.md @@ -0,0 +1,49 @@ +# Issue closing pattern + +>**Note:** +This is the administration documentation. +There is a separate [user documentation] on issue closing pattern. + +When a commit or merge request resolves one or more issues, it is possible to +automatically have these issues closed when the commit or merge request lands +in the project's default branch. + +## Change the issue closing pattern + +In order to change the pattern you need to have access to the server that GitLab +is installed on. + +The default pattern can be located in [gitlab.yml.example] under the +"Automatic issue closing" section. + +> **Tip:** +You are advised to use http://rubular.com to test the issue closing pattern. +Because Rubular doesn't understand `%{issue_ref}`, you can replace this by +`#\d+` when testing your patterns, which matches only local issue references like `#123`. + +**For Omnibus installations** + +1. Open `/etc/gitlab/gitlab.rb` with your editor. +1. Change the value of `gitlab_rails['issue_closing_pattern']` to a regular + expression of your liking: + + ```ruby + gitlab_rails['issue_closing_pattern'] = "((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` +1. [Reconfigure] GitLab for the changes to take effect. + +**For installations from source** + +1. Open `gitlab.yml` with your editor. +1. Change the value of `issue_closing_pattern`: + + ```yaml + issue_closing_pattern: "((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` + +1. [Restart] GitLab for the changes to take effect. + +[gitlab.yml.example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example +[reconfigure]: restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: restart_gitlab.md#installations-from-source +[user documentation]: ../user/project/issues/automatic_issue_closing.md diff --git a/doc/api/README.md b/doc/api/README.md index 7661e1eea02..8e4f7f12b4b 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -10,6 +10,7 @@ following locations: - [Award Emoji](award_emoji.md) - [Branches](branches.md) +- [Broadcast Messages](broadcast_messages.md) - [Builds](builds.md) - [Build Triggers](build_triggers.md) - [Build Variables](build_variables.md) @@ -55,11 +56,12 @@ The following documentation is for the [internal CI API](ci/README.md): ## Authentication -All API requests require authentication via a token. There are three types of tokens -available: private tokens, OAuth 2 tokens, and personal access tokens. +All API requests require authentication via a session cookie or token. There are +three types of tokens available: private tokens, OAuth 2 tokens, and personal +access tokens. -If a token is invalid or omitted, an error message will be returned with -status code `401`: +If authentication information is invalid or omitted, an error message will be +returned with status code `401`: ```json { @@ -98,6 +100,13 @@ that needs access to the GitLab API. Once you have your token, pass it to the API using either the `private_token` parameter or the `PRIVATE-TOKEN` header. + +### Session cookie + +When signing in to GitLab as an ordinary user, a `_gitlab_session` cookie is +set. The API will use this cookie for authentication if it is present, but using +the API to generate a new session cookie is currently not supported. + ## Basic Usage API requests should be prefixed with `api` and the API version. The API version diff --git a/doc/api/award_emoji.md b/doc/api/award_emoji.md index 72ec99b7c56..c464e3f3f71 100644 --- a/doc/api/award_emoji.md +++ b/doc/api/award_emoji.md @@ -1,12 +1,13 @@ # Award Emoji -> [Introduced][ce-4575] in GitLab 8.9. +> [Introduced][ce-4575] in GitLab 8.9, Snippet support in 8.12 + An awarded emoji tells a thousand words, and can be awarded on issues, merge -requests and notes/comments. Issues, merge requests and notes are further called +requests, snippets, and notes/comments. Issues, merge requests, snippets, and notes are further called `awardables`. -## Issues and merge requests +## Issues, merge requests, and snippets ### List an awardable's award emoji @@ -15,6 +16,7 @@ Gets a list of all award emoji ``` GET /projects/:id/issues/:issue_id/award_emoji GET /projects/:id/merge_requests/:merge_request_id/award_emoji +GET /projects/:id/snippets/:snippet_id/award_emoji ``` Parameters: @@ -69,11 +71,12 @@ Example Response: ### Get single award emoji -Gets a single award emoji from an issue or merge request. +Gets a single award emoji from an issue, snippet, or merge request. ``` GET /projects/:id/issues/:issue_id/award_emoji/:award_id GET /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id +GET /projects/:id/snippets/:snippet_id/award_emoji/:award_id ``` Parameters: @@ -116,6 +119,7 @@ This end point creates an award emoji on the specified resource ``` POST /projects/:id/issues/:issue_id/award_emoji POST /projects/:id/merge_requests/:merge_request_id/award_emoji +POST /projects/:id/snippets/:snippet_id/award_emoji ``` Parameters: @@ -159,6 +163,7 @@ admins or the author of the award. Status code 200 on success, 401 if unauthoriz ``` DELETE /projects/:id/issues/:issue_id/award_emoji/:award_id DELETE /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id +DELETE /projects/:id/snippets/:snippet_id/award_emoji/:award_id ``` Parameters: @@ -197,7 +202,7 @@ Example Response: ## Award Emoji on Notes The endpoints documented above are available for Notes as well. Notes -are a sub-resource of Issues and Merge Requests. The examples below +are a sub-resource of Issues, Merge Requests, or Snippets. The examples below describe working with Award Emoji on notes for an Issue, but can be easily adapted for notes on a Merge Request. diff --git a/doc/api/groups.md b/doc/api/groups.md index a898387eaa2..e81d6f9de4b 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -84,7 +84,8 @@ Parameters: "forks_count": 0, "open_issues_count": 3, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "request_access_enabled": false } ] ``` @@ -118,6 +119,7 @@ Example response: "visibility_level": 20, "avatar_url": null, "web_url": "https://gitlab.example.com/groups/twitter", + "request_access_enabled": false, "projects": [ { "id": 7, @@ -163,7 +165,8 @@ Example response: "forks_count": 0, "open_issues_count": 3, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "request_access_enabled": false }, { "id": 6, @@ -209,7 +212,8 @@ Example response: "forks_count": 0, "open_issues_count": 8, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "request_access_enabled": false } ], "shared_projects": [ @@ -288,6 +292,8 @@ Parameters: - `path` (required) - The path of the group - `description` (optional) - The group's description - `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public. +- `lfs_enabled` (optional) - Enable/disable Large File Storage (LFS) for the projects in this group +- `request_access_enabled` (optional) - Allow users to request member access. ## Transfer project to group @@ -317,6 +323,8 @@ PUT /groups/:id | `path` | string | no | The path of the group | | `description` | string | no | The description of the group | | `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. | +| `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group | +| `request_access_enabled` | boolean | no | Allow users to request member access. | ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental" @@ -334,6 +342,7 @@ Example response: "visibility_level": 10, "avatar_url": null, "web_url": "http://gitlab.example.com/groups/h5bp", + "request_access_enabled": false, "projects": [ { "id": 9, @@ -378,7 +387,8 @@ Example response: "forks_count": 0, "open_issues_count": 3, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "request_access_enabled": false } ] } diff --git a/doc/api/notes.md b/doc/api/notes.md index 85d140d06ac..572844b8b3f 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -78,7 +78,8 @@ Parameters: ### Create new issue note -Creates a new note to a single project issue. +Creates a new note to a single project issue. If you create a note where the body +only contains an Award Emoji, you'll receive this object back. ``` POST /projects/:id/issues/:issue_id/notes @@ -204,6 +205,7 @@ Parameters: ### Create new snippet note Creates a new note for a single snippet. Snippet notes are comments users can post to a snippet. +If you create a note where the body only contains an Award Emoji, you'll receive this object back. ``` POST /projects/:id/snippets/:snippet_id/notes @@ -332,6 +334,8 @@ Parameters: ### Create new merge request note Creates a new note for a single merge request. +If you create a note where the body only contains an Award Emoji, you'll receive +this object back. ``` POST /projects/:id/merge_requests/:merge_request_id/notes diff --git a/doc/api/projects.md b/doc/api/projects.md index fe3c8709d13..750ce1508df 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -85,7 +85,8 @@ Parameters: "runners_token": "b8547b1dc37721d05889db52fa2f02", "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false }, { "id": 6, @@ -146,7 +147,8 @@ Parameters: "runners_token": "b8547b1dc37721d05889db52fa2f02", "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ] ``` @@ -283,7 +285,8 @@ Parameters: "group_access_level": 10 } ], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -453,6 +456,7 @@ Parameters: - `public_builds` (optional) - `only_allow_merge_if_build_succeeds` (optional) - `lfs_enabled` (optional) +- `request_access_enabled` (optional) - Allow users to request member access. ### Create project for user @@ -480,6 +484,7 @@ Parameters: - `public_builds` (optional) - `only_allow_merge_if_build_succeeds` (optional) - `lfs_enabled` (optional) +- `request_access_enabled` (optional) - Allow users to request member access. ### Edit project @@ -508,6 +513,7 @@ Parameters: - `public_builds` (optional) - `only_allow_merge_if_build_succeeds` (optional) - `lfs_enabled` (optional) +- `request_access_enabled` (optional) - Allow users to request member access. On success, method returns 200 with the updated project. If parameters are invalid, 400 is returned. @@ -588,7 +594,8 @@ Example response: "star_count": 1, "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -655,7 +662,8 @@ Example response: "star_count": 0, "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -742,7 +750,8 @@ Example response: "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b", "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -829,7 +838,8 @@ Example response: "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b", "public_builds": true, "shared_with_groups": [], - "only_allow_merge_if_build_succeeds": false + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index fc3af5544de..1bc6a24e914 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -44,7 +44,7 @@ POST /projects/:id/repository/files ``` ```bash -curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&content=some%20content&commit_message=create%20a%20new%20file' +curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' ``` Example response: @@ -61,6 +61,8 @@ Parameters: - `file_path` (required) - Full path to new file. Ex. lib/class.rb - `branch_name` (required) - The name of branch - `encoding` (optional) - 'text' or 'base64'. Text is default. +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `content` (required) - File content - `commit_message` (required) - Commit message @@ -71,7 +73,7 @@ PUT /projects/:id/repository/files ``` ```bash -curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&content=some%20other%20content&commit_message=update%20file' +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' ``` Example response: @@ -88,6 +90,8 @@ Parameters: - `file_path` (required) - Full path to file. Ex. lib/class.rb - `branch_name` (required) - The name of branch - `encoding` (optional) - 'text' or 'base64'. Text is default. +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `content` (required) - New file content - `commit_message` (required) - Commit message @@ -107,7 +111,7 @@ DELETE /projects/:id/repository/files ``` ```bash -curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&commit_message=delete%20file' +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' ``` Example response: @@ -123,4 +127,6 @@ Parameters: - `file_path` (required) - Full path to file. Ex. lib/class.rb - `branch_name` (required) - The name of branch +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `commit_message` (required) - Commit message diff --git a/doc/api/settings.md b/doc/api/settings.md index a76dad0ebd4..aaa2c99642b 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -67,7 +67,7 @@ PUT /application/settings | `default_snippet_visibility` | integer | no | What visibility level new snippets receive. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is `0`.| | `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. | | `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` | -| `domain_blacklist` | array of strings | yes (if `domain_whitelist_enabled` is `true` | People trying to sign-up with emails from this domain will not be allowed to do so. | +| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. | | `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider | | `after_sign_out_path` | string | no | Where to redirect users after logout | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | diff --git a/doc/ci/README.md b/doc/ci/README.md index 10ce4ac8940..341bc85a16a 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -16,5 +16,7 @@ - [Trigger builds through the API](triggers/README.md) - [Build artifacts](../user/project/builds/artifacts.md) - [User permissions](../user/permissions.md#gitlab-ci) +- [Build permissions](../user/permissions.md#build-permissions) - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) +- [**New CI build permissions model**](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 406396deaaa..71670e6247c 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -16,4 +16,4 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -[gitlab-ci-templates][https://gitlab.com/gitlab-org/gitlab-ci-yml] +[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index c835ebc2d44..c40cdd55ea5 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -105,7 +105,8 @@ What is important is that each job is run independently from each other. If you want to check whether your `.gitlab-ci.yml` file is valid, there is a Lint tool under the page `/ci/lint` of your GitLab instance. You can also find -the link under **Settings > CI settings** in your project. +a "CI Lint" button to go to this page under **Pipelines > Pipelines** and +**Pipelines > Builds** in your project. For more information and a complete `.gitlab-ci.yml` syntax, please read [the documentation on .gitlab-ci.yml](../yaml/README.md). diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 6c6767fea0b..b78422f6d0e 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -2,6 +2,10 @@ > [Introduced][ci-229] in GitLab CE 7.14. +> **Note**: +GitLab 8.12 has a completely redesigned build permissions system. +Read all about the [new model and its implications][../../user/project/new_ci_build_permissions_model.md#build-triggers]. + Triggers can be used to force a rebuild of a specific branch, tag or commit, with an API call. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index ff4c8ddc54b..16868554c1f 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -90,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string. ### after_script ->**Note:** -Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 +> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 `after_script` is used to define the command that will be run after for all builds. This has to be an array or a multi-line string. @@ -135,8 +134,7 @@ Alias for [stages](#stages). ### variables ->**Note:** -Introduced in GitLab Runner v0.5.0. +> Introduced in GitLab Runner v0.5.0. GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the build environment. The variables are stored in the Git repository and are meant @@ -158,8 +156,7 @@ Variables can be also defined on [job level](#job-variables). ### cache ->**Note:** -Introduced in GitLab Runner v0.7.0. +> Introduced in GitLab Runner v0.7.0. `cache` is used to specify a list of files and directories which should be cached between builds. @@ -220,8 +217,7 @@ will be always present. For implementation details, please check GitLab Runner. #### cache:key ->**Note:** -Introduced in GitLab Runner v1.0.0. +> Introduced in GitLab Runner v1.0.0. The `key` directive allows you to define the affinity of caching between jobs, allowing to have a single cache for all jobs, @@ -531,8 +527,7 @@ The above script will: #### Manual actions ->**Note:** -Introduced in GitLab 8.10. +> Introduced in GitLab 8.10. Manual actions are a special type of job that are not executed automatically; they need to be explicitly started by a user. Manual actions can be started @@ -543,17 +538,16 @@ An example usage of manual actions is deployment to production. ### environment ->**Note:** -Introduced in GitLab 8.9. +> Introduced in GitLab 8.9. -`environment` is used to define that a job deploys to a specific environment. +`environment` is used to define that a job deploys to a specific [environment]. This allows easy tracking of all deployments to your environments straight from GitLab. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name must contain only letters, digits, '-' and '_'. Common +The `environment` name must contain only letters, digits, '-', '_', '/', '$', '{', '}' and spaces. Common names are `qa`, `staging`, and `production`, but you can use whatever name works with your workflow. @@ -571,6 +565,35 @@ deploy to production: The `deploy to production` job will be marked as doing deployment to `production` environment. +#### dynamic environments + +> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. + +`environment` can also represent a configuration hash with `name` and `url`. +These parameters can use any of the defined CI [variables](#variables) +(including predefined, secure variables and `.gitlab-ci.yml` variables). + +The common use case is to create dynamic environments for branches and use them +as review apps. + +--- + +**Example configurations** + +``` +deploy as review app: + stage: deploy + script: ... + environment: + name: review-apps/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.review.example.com/ +``` + +The `deploy as review app` job will be marked as deployment to dynamically +create the `review-apps/branch-name` environment. + +This environment should be accessible under `https://branch-name.review.example.com/`. + ### artifacts >**Notes:** @@ -638,8 +661,7 @@ be available for download in the GitLab UI. #### artifacts:name ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.0. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.0. The `name` directive allows you to define the name of the created artifacts archive. That way, you can have a unique name for every archive which could be @@ -702,8 +724,7 @@ job: #### artifacts:when ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:when` is used to upload artifacts on build failure or despite the failure. @@ -728,8 +749,7 @@ job: #### artifacts:expire_in ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:expire_in` is used to delete uploaded artifacts after the specified time. By default, artifacts are stored on GitLab forever. `expire_in` allows you @@ -764,8 +784,7 @@ job: ### dependencies ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. This feature should be used in conjunction with [`artifacts`](#artifacts) and allows you to define the artifacts to pass between different builds. @@ -839,9 +858,8 @@ job: ## Git Strategy ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future -releases or be removed completely. +> Introduced in GitLab 8.9 as an experimental feature. May change in future + releases or be removed completely. You can set the `GIT_STRATEGY` used for getting recent application code. `clone` is slower, but makes sure you have a clean directory before every build. `fetch` @@ -863,8 +881,7 @@ variables: ## Shallow cloning ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future +> Introduced in GitLab 8.9 as an experimental feature. May change in future releases or be removed completely. You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows @@ -894,8 +911,7 @@ variables: ## Hidden keys ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. Keys that start with a dot (`.`) will be not processed by GitLab CI. You can use this feature to ignore jobs, or use the @@ -923,8 +939,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya ### Anchors ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. YAML also has a handy feature called 'anchors', which let you easily duplicate content across your document. Anchors can be used to duplicate/inherit @@ -1067,3 +1082,5 @@ Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. [examples]: ../examples/README.md +[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 +[environment]: ../environments.md diff --git a/doc/container_registry/README.md b/doc/container_registry/README.md index 047a0b08406..d7740647a91 100644 --- a/doc/container_registry/README.md +++ b/doc/container_registry/README.md @@ -78,9 +78,9 @@ delete them. > **Note:** This feature requires GitLab 8.8 and GitLab Runner 1.2. -Make sure that your GitLab Runner is configured to allow building docker images. -You have to check the [Using Docker Build documentation](../ci/docker/using_docker_build.md). -Then see the CI documentation on [Using the GitLab Container Registry](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry). +Make sure that your GitLab Runner is configured to allow building Docker images by +following the [Using Docker Build](../ci/docker/using_docker_build.md) +and [Using the GitLab Container Registry documentation](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry). ## Limitations diff --git a/doc/customization/issue_closing.md b/doc/customization/issue_closing.md index 4620bb2dcde..31164ccd465 100644 --- a/doc/customization/issue_closing.md +++ b/doc/customization/issue_closing.md @@ -1,39 +1,4 @@ -# Issue closing pattern +This document was split into: -When a commit or merge request resolves one or more issues, it is possible to automatically have these issues closed when the commit or merge request lands in the project's default branch. - -If a commit message or merge request description contains a sentence matching the regular expression below, all issues referenced from -the matched text will be closed. This happens when the commit is pushed to a project's default branch, or when a commit or merge request is merged into there. - -When not specified, the default `issue_closing_pattern` as shown below will be used: - -```bash -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) -``` - -Here, `%{issue_ref}` is a complex regular expression defined inside GitLab, that matches a reference to a local issue (`#123`), cross-project issue (`group/project#123`) or a link to an issue (`https://gitlab.example.com/group/project/issues/123`). - -For example: - -``` -git commit -m "Awesome commit message (Fix #20, Fixes #21 and Closes group/otherproject#22). This commit is also related to #17 and fixes #18, #19 and https://gitlab.example.com/group/otherproject/issues/23." -``` - -will close `#18`, `#19`, `#20`, and `#21` in the project this commit is pushed to, as well as `#22` and `#23` in group/otherproject. `#17` won't be closed as it does not match the pattern. It also works with multiline commit messages. - -Tip: you can test this closing pattern at [http://rubular.com][1]. Use this site -to test your own patterns. -Because Rubular doesn't understand `%{issue_ref}`, you can replace this by `#\d+` in testing, which matches only local issue references like `#123`. - -## Change the pattern - -For Omnibus installs you can change the default pattern in `/etc/gitlab/gitlab.rb`: - -``` -issue_closing_pattern: '((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)' -``` - -For manual installs you can customize the pattern in [gitlab.yml][0] using the `issue_closing_pattern` key. - -[0]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example -[1]: http://rubular.com/r/Xmbexed1OJ +- [administration/issue_closing_pattern.md](../administration/issue_closing_pattern.md). +- [user/project/issues/automatic_issue_closing](../user/project/issues/automatic_issue_closing.md). diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index b8fab3aaff7..295eae0a88e 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -111,6 +111,28 @@ class MyMigration < ActiveRecord::Migration end ``` + +## Integer column type + +By default, an integer column can hold up to a 4-byte (32-bit) number. That is +a max value of 2,147,483,647. Be aware of this when creating a column that will +hold file sizes in byte units. If you are tracking file size in bytes this +restricts the maximum file size to just over 2GB. + +To allow an integer column to hold up to an 8-byte (64-bit) number, explicitly +set the limit to 8-bytes. This will allow the column to hold a value up to +9,223,372,036,854,775,807. + +Rails migration example: + +``` +add_column_with_default(:projects, :foo, :integer, default: 10, limit: 8) + +# or + +add_column(:projects, :foo, :integer, default: 10, limit: 8) +``` + ## Testing Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct. diff --git a/doc/gitlab-basics/create-issue.md b/doc/gitlab-basics/create-issue.md index 5221d85b661..da9a165b8f5 100644 --- a/doc/gitlab-basics/create-issue.md +++ b/doc/gitlab-basics/create-issue.md @@ -1,6 +1,6 @@ # How to create an Issue in GitLab -The Issue Tracker is a good place to add things that need to be improved or solved in a project. +The Issue Tracker is a good place to add things that need to be improved or solved in a project. To create an Issue, sign in to GitLab. @@ -24,4 +24,4 @@ You may assign the Issue to a user, add a milestone and add labels (they are all  -Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](http://docs.gitlab.com/ce/customization/issue_closing.html). +Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](../user/project/issues/automatic_issue_closing.md). diff --git a/doc/install/installation.md b/doc/install/installation.md index df98655c396..3ac813aa914 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -400,7 +400,7 @@ If you are not using Linux you may have to run `gmake` instead of cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse - sudo -u git -H git checkout v0.8.1 + sudo -u git -H git checkout v0.8.2 sudo -u git -H make ### Initialize Database and Activate Advanced Features diff --git a/doc/intro/README.md b/doc/intro/README.md index 71fef50ceb4..1790b2b761f 100644 --- a/doc/intro/README.md +++ b/doc/intro/README.md @@ -22,7 +22,7 @@ Create merge requests and review code. - [Fork a project and contribute to it](../workflow/forking_workflow.md) - [Create a new merge request](../gitlab-basics/add-merge-request.md) -- [Automatically close issues from merge requests](../customization/issue_closing.md) +- [Automatically close issues from merge requests](../user/project/issues/automatic_issue_closing.md) - [Automatically merge when your builds succeed](../user/project/merge_requests/merge_when_build_succeeds.md) - [Revert any commit](../user/project/merge_requests/revert_changes.md) - [Cherry-pick any commit](../user/project/merge_requests/cherry_pick_changes.md) diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 835af5443a3..3f4056dc440 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -79,6 +79,9 @@ gitlab_rails['backup_upload_connection'] = { 'region' => 'eu-west-1', 'aws_access_key_id' => 'AKIAKIAKI', 'aws_secret_access_key' => 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. 'aws_access_key_id' => '', + # 'use_iam_profile' => 'true' } gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' ``` @@ -95,6 +98,9 @@ For installations from source: region: eu-west-1 aws_access_key_id: AKIAKIAKI aws_secret_access_key: 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. aws_access_key_id: '' + # use_iam_profile: 'true' # The remote 'directory' to store your backups. For S3, this would be the bucket name. remote_directory: 'my.s3.bucket' # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional diff --git a/doc/update/8.11-to-8.12.md b/doc/update/8.11-to-8.12.md index 8017c36587e..076696f565b 100644 --- a/doc/update/8.11-to-8.12.md +++ b/doc/update/8.11-to-8.12.md @@ -70,7 +70,7 @@ sudo -u git -H git checkout 8-12-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch --all --tags -sudo -u git -H git checkout v3.5.0 +sudo -u git -H git checkout v3.6.0 ``` ### 6. Update gitlab-workhorse @@ -82,7 +82,7 @@ GitLab 8.1. ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch --all -sudo -u git -H git checkout v0.8.1 +sudo -u git -H git checkout v0.8.2 sudo -u git -H make ``` diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 1498cb361c8..12d5b8f8744 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -63,7 +63,7 @@ The following table depicts the various user permission levels in a project. | Force push to protected branches [^2] | | | | | | | Remove protected branches [^2] | | | | | | -[^1]: If **Allow guest to access builds** is enabled in CI settings +[^1]: If **Public pipelines** is enabled in **Project Settings > CI/CD Pipelines** [^2]: Not allowed for Guest, Reporter, Developer, Master, or Owner ## Group @@ -138,3 +138,33 @@ instance and project. In addition, all admins can use the admin interface under | Add shared runners | | | | ✓ | | See events in the system | | | | ✓ | | Admin interface | | | | ✓ | + +### Build permissions + +> Changed in GitLab 8.12. + +GitLab 8.12 has a completely redesigned build permissions system. +Read all about the [new model and its implications][new-mod]. + +This table shows granted privileges for builds triggered by specific types of +users: + +| Action | Guest, Reporter | Developer | Master | Admin | +|---------------------------------------------|-----------------|-------------|----------|--------| +| Run CI build | | ✓ | ✓ | ✓ | +| Clone source and LFS from current project | | ✓ | ✓ | ✓ | +| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | +| Clone source and LFS from internal projects | | ✓ [^3] | ✓ [^3] | ✓ | +| Clone source and LFS from private projects | | ✓ [^4] | ✓ [^4] | ✓ [^4] | +| Push source and LFS | | | | | +| Pull container images from current project | | ✓ | ✓ | ✓ | +| Pull container images from public projects | | ✓ | ✓ | ✓ | +| Pull container images from internal projects| | ✓ [^3] | ✓ [^3] | ✓ | +| Pull container images from private projects | | ✓ [^4] | ✓ [^4] | ✓ [^4] | +| Push container images to current project | | ✓ | ✓ | ✓ | +| Push container images to other projects | | | | | + +[^3]: Only if user is not external one. +[^4]: Only if user is a member of the project. +[ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 +[new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/project/builds/artifacts.md b/doc/user/project/builds/artifacts.md index c93ae1c369c..88f1863dddb 100644 --- a/doc/user/project/builds/artifacts.md +++ b/doc/user/project/builds/artifacts.md @@ -101,4 +101,36 @@ inside GitLab that make that possible.  +## Downloading the latest build artifacts + +It is possible to download the latest artifacts of a build via a well known URL +so you can use it for scripting purposes. + +The structure of the URL is the following: + +``` +https://example.com/<namespace>/<project>/builds/artifacts/<ref>/download?job=<job_name> +``` + +For example, to download the latest artifacts of the job named `rspec 6 20` of +the `master` branch of the `gitlab-ce` project that belongs to the `gitlab-org` +namespace, the URL would be: + +``` +https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/download?job=rspec+6+20 +``` + +The latest builds are also exposed in the UI in various places. Specifically, +look for the download button in: + +- the main project's page +- the branches page +- the tags page + +If the latest build has failed to upload the artifacts, you can see that +information in the UI. + + + + [gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository" diff --git a/doc/user/project/builds/img/build_latest_artifacts_browser.png b/doc/user/project/builds/img/build_latest_artifacts_browser.png Binary files differnew file mode 100644 index 00000000000..d8e9071958c --- /dev/null +++ b/doc/user/project/builds/img/build_latest_artifacts_browser.png diff --git a/doc/user/project/cycle_analytics.md b/doc/user/project/cycle_analytics.md new file mode 100644 index 00000000000..abef80e7914 --- /dev/null +++ b/doc/user/project/cycle_analytics.md @@ -0,0 +1,114 @@ +# Cycle Analytics + +> [Introduced][ce-5986] in GitLab 8.12. +> +> **Note:** +This the first iteration of Cycle Analytics, you can follow the following issue +to track the changes that are coming to this feature: [#20975][ce-20975]. + +Cycle Analytics measures the time it takes to go from [an idea to production] for +each project you have. This is achieved by not only indicating the total time it +takes to reach at that point, but the total time is broken down into the +multiple stages an idea has to pass through to be shipped. + +Cycle Analytics is that it is tightly coupled with the [GitLab flow] and +calculates a separate median for each stage. + +## Overview + +You can find the Cycle Analytics page under your project's **Pipelines > Cycle +Analytics** tab. + + + +You can see that there are seven stages in total: + +- **Issue** (Tracker) + - Median time from issue creation until given a milestone or list label + (first assignment, any milestone, milestone date or assignee is not required) +- **Plan** (Board) + - Median time from giving an issue a milestone or label until pushing the + first commit +- **Code** (IDE) + - Median time from the first commit until the merge request is created +- **Test** (CI) + - Median total test time for all commits/merges +- **Review** (Merge Request/MR) + - Median time from merge request creation until the merge request is merged + (closed merge requests won't be taken into account) +- **Staging** (Continuous Deployment) + - Median time from when the merge request got merged until the deploy to + production (production is last stage/environment) +- **Production** (Total) + - Sum of all the above stages excluding the Test (CI) time + +## How the data is measured + +Cycle Analytics records cycle time so only data on the issues that have been +deployed to production are measured. In case you just started a new project and +you have not pushed anything to production, then you will not be able to +properly see the Cycle Analytics of your project. + +Specifically, if your CI is not set up and you have not defined a `production` +[environment], then you will not have any data. + +Below you can see in more detail what the various stages of Cycle Analytics mean. + +| **Stage** | **Description** | +| --------- | --------------- | +| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list][board] created for it. | +| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the repository. To make this change tracked, the pushed commit needs to contain the [issue closing pattern], for example `Closes #xxx`, where `xxx` is the number of the issue related to this commit. If the commit does not contain the issue closing pattern, it is not considered to the measurement time of the stage. | +| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request related to that commit. The key to keep the process tracked is include the [issue closing pattern] to the description of the merge request. | +| Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. `master` is not excluded. It does not attempt to track time for any particular stages. | +| Review | Measures the median time taken to review the merge request, between its creation and until it's merged. | +| Staging | Measures the median time between merging the merge request until the very first deployment to production. It's tracked by the [environment] set to `production` in your GitLab CI configuration. If there isn't a `production` environment, this is not tracked. | +| Production| The sum of all time taken to run the entire process, from issue creation to deploying the code to production. | + +--- + +Here's a little explanation of how this works behind the scenes: + +1. Issues and merge requests are grouped together in pairs, such that for each + `<issue, merge request>` pair, the merge request has `Fixes #xxx` for the + corresponding issue. All other issues and merge requests are **not** considered. + +1. Then the <issue, merge request> pairs are filtered out. Any merge request + that has **not** been deployed to production in the last XX days (specified + by the UI - default is 90 days) prohibits these pairs from being considered. + +1. For the remaining `<issue, merge request>` pairs, we check the information that + we need for the stages, like issue creation date, merge request merge time, + etc. + +To sum up, anything that doesn't follow the [GitLab flow] won't be tracked at all. +So, if a merge request doesn't close an issue or an issue is not labeled with a +label present in the Issue Board or assigned a milestone or a project has no +`production` environment, the Cycle Analytics dashboard won't present any data +at all. + +## Permissions + +The current permissions on the Cycle Analytics dashboard are: + +- Public projects - anyone can access +- Private/internal projects - any member (guest level and above) can access + +You can [read more about permissions][permissions] in general. + +## More resources + +Learn more about Cycle Analytics in the following resources: + +- [Cycle Analytics feature page](https://about.gitlab.com/solutions/cycle-analytics/) +- [Cycle Analytics feature preview](https://about.gitlab.com/2016/09/16/feature-preview-introducing-cycle-analytics/) +- [Cycle Analytics feature highlight](https://about.gitlab.com/2016/09/21/cycle-analytics-feature-highlight/) + + +[ce-5986]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5986 +[ce-20975]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20975 +[GitLab flow]: ../../workflow/gitlab_flow.md +[permissions]: ../permissions.md +[environment]: ../../ci/yaml/README.md#environment +[board]: issue_board.md#creating-a-new-list +[idea to production]: https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab +[issue closing pattern]: issues/automatic_issue_closing.md diff --git a/doc/user/project/img/cycle_analytics_landing_page.png b/doc/user/project/img/cycle_analytics_landing_page.png Binary files differnew file mode 100644 index 00000000000..4fa42c87395 --- /dev/null +++ b/doc/user/project/img/cycle_analytics_landing_page.png diff --git a/doc/user/project/issues/automatic_issue_closing.md b/doc/user/project/issues/automatic_issue_closing.md new file mode 100644 index 00000000000..d6f3a7d5555 --- /dev/null +++ b/doc/user/project/issues/automatic_issue_closing.md @@ -0,0 +1,55 @@ +# Automatic issue closing + +>**Note:** +This is the user docs. In order to change the default issue closing pattern, +follow the steps in the [administration docs]. + +When a commit or merge request resolves one or more issues, it is possible to +automatically have these issues closed when the commit or merge request lands +in the project's default branch. + +If a commit message or merge request description contains a sentence matching +a certain regular expression, all issues referenced from the matched text will +be closed. This happens when the commit is pushed to a project's **default** +branch, or when a commit or merge request is merged into it. + +## Default closing pattern value + +When not specified, the default issue closing pattern as shown below will be +used: + +```bash +((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) +``` + +Note that `%{issue_ref}` is a complex regular expression defined inside GitLab's +source code that can match a reference to 1) a local issue (`#123`), +2) a cross-project issue (`group/project#123`) or 3) a link to an issue +(`https://gitlab.example.com/group/project/issues/123`). + +--- + +This translates to the following keywords: + +- Close, Closes, Closed, Closing, close, closes, closed, closing +- Fix, Fixes, Fixed, Fixing, fix, fixes, fixed, fixing +- Resolve, Resolves, Resolved, Resolving, resolve, resolves, resolved, resolving + +--- + +For example the following commit message: + +``` +Awesome commit message + +Fix #20, Fixes #21 and Closes group/otherproject#22. +This commit is also related to #17 and fixes #18, #19 +and https://gitlab.example.com/group/otherproject/issues/23. +``` + +will close `#18`, `#19`, `#20`, and `#21` in the project this commit is pushed +to, as well as `#22` and `#23` in group/otherproject. `#17` won't be closed as +it does not match the pattern. It works with multi-line commit messages as well +as one-liners when used with `git commit -m`. + +[administration docs]: ../../../administration/issue_closing_pattern.md diff --git a/doc/user/project/merge_requests.md b/doc/user/project/merge_requests.md index f79535d1542..5af9a5d049c 100644 --- a/doc/user/project/merge_requests.md +++ b/doc/user/project/merge_requests.md @@ -93,6 +93,9 @@ A merge request contains all the history from a repository, plus the additional commits added to the branch associated with the merge request. Here's a few tricks to checkout a merge request locally. +Please note that you can checkout a merge request locally even if the source +project is a fork (even a private fork) of the target project. + #### Checkout locally by adding a git alias Add the following alias to your `~/.gitconfig`: diff --git a/doc/user/project/merge_requests/img/versions-compare.png b/doc/user/project/merge_requests/img/versions-compare.png Binary files differnew file mode 100644 index 00000000000..890cae7768c --- /dev/null +++ b/doc/user/project/merge_requests/img/versions-compare.png diff --git a/doc/user/project/merge_requests/img/versions-dropdown.png b/doc/user/project/merge_requests/img/versions-dropdown.png Binary files differnew file mode 100644 index 00000000000..9bab9304e14 --- /dev/null +++ b/doc/user/project/merge_requests/img/versions-dropdown.png diff --git a/doc/user/project/merge_requests/img/versions.png b/doc/user/project/merge_requests/img/versions.png Binary files differindex f279ccd7ce3..6c86f2c68ac 100644 --- a/doc/user/project/merge_requests/img/versions.png +++ b/doc/user/project/merge_requests/img/versions.png diff --git a/doc/user/project/merge_requests/versions.md b/doc/user/project/merge_requests/versions.md index a6aa4b47835..2805fdf635c 100644 --- a/doc/user/project/merge_requests/versions.md +++ b/doc/user/project/merge_requests/versions.md @@ -7,14 +7,18 @@ of merge request diff is created. When you visit a merge request that contains more than one pushes, you can select and compare the versions of those merge request diffs. + + By default, the latest version of changes is shown. However, you can select an older one from version dropdown. - + You can also compare the merge request version with older one to see what is changed since then. + + Please note that comments are disabled while viewing outdated merge versions or comparing to versions other than base. diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md new file mode 100644 index 00000000000..e73f60023b5 --- /dev/null +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -0,0 +1,289 @@ +# New CI build permissions model + +> Introduced in GitLab 8.12. + +GitLab 8.12 has a completely redesigned [build permissions] system. You can find +all discussion and all our concerns when choosing the current approach in issue +[#18994](https://gitlab.com/gitlab-org/gitlab-ce/issues/18994). + +--- + +Builds permissions should be tightly integrated with the permissions of a user +who is triggering a build. + +The reasons to do it like that are: + +- We already have a permissions system in place: group and project membership + of users. +- We already fully know who is triggering a build (using `git push`, using the + web UI, executing triggers). +- We already know what user is allowed to do. +- We use the user permissions for builds that are triggered by the user. +- It opens a lot of possibilities to further enforce user permissions, like + allowing only specific users to access runners or use secure variables and + environments. +- It is simple and convenient that your build can access everything that you + as a user have access to. +- Short living unique tokens are now used, granting access for time of the build + and maximizing security. + +With the new behavior, any build that is triggered by the user, is also marked +with their permissions. When a user does a `git push` or changes files through +the web UI, a new pipeline will be usually created. This pipeline will be marked +as created be the pusher (local push or via the UI) and any build created in this +pipeline will have the permissions of the pusher. + +This allows us to make it really easy to evaluate the access for all projects +that have Git submodules or are using container images that the pusher would +have access too. **The permission is granted only for time that build is running. +The access is revoked after the build is finished.** + +## Types of users + +It is important to note that we have a few types of users: + +- **Administrators**: CI builds created by Administrators will not have access + to all GitLab projects, but only to projects and container images of projects + that the administrator is a member of.That means that if a project is either + public or internal users have access anyway, but if a project is private, the + Administrator will have to be a member of it in order to have access to it + via another project's build. + +- **External users**: CI builds created by [external users][ext] will have + access only to projects to which user has at least reporter access. This + rules out accessing all internal projects by default, + +This allows us to make the CI and permission system more trustworthy. +Let's consider the following scenario: + +1. You are an employee of a company. Your company has a number of internal tools + hosted in private repositories and you have multiple CI builds that make use + of these repositories. + +2. You invite a new [external user][ext]. CI builds created by that user do not + have access to internal repositories, because the user also doesn't have the + access from within GitLab. You as an employee have to grant explicit access + for this user. This allows us to prevent from accidental data leakage. + +## Build token + +A unique build token is generated for each build and it allows the user to +access all projects that would be normally accessible to the user creating that +build. + +We try to make sure that this token doesn't leak by: + +1. Securing all API endpoints to not expose the build token. +1. Masking the build token from build logs. +1. Allowing to use the build token **only** when build is running. + +However, this brings a question about the Runners security. To make sure that +this token doesn't leak, you should also make sure that you configure +your Runners in the most possible secure way, by avoiding the following: + +1. Any usage of Docker's `privileged` mode is risky if the machines are re-used. +1. Using the `shell` executor since builds run on the same machine. + +By using an insecure GitLab Runner configuration, you allow the rogue developers +to steal the tokens of other builds. + +## Debugging problems + +With the new permission model in place, there may be times that your build will +fail. This is most likely because your project tries to access other project's +sources, and you don't have the appropriate permissions. In the build log look +for information about 403 or forbidden access messages + +As an Administrator, you can verify that the user is a member of the group or +project they're trying to have access to, and you can impersonate the user to +retry the failing build in order to verify that everything is correct. + +## Build triggers + +[Build triggers][triggers] do not support the new permission model. +They continue to use the old authentication mechanism where the CI build +can access only its own sources. We plan to remove that limitation in one of +the upcoming releases. + +## Before GitLab 8.12 + +In versions before GitLab 8.12, all CI builds would use the CI Runner's token +to checkout project sources. + +The project's Runner's token was a token that you could find under the +project's **Settings > CI/CD Pipelines** and was limited to access only that +project. +It could be used for registering new specific Runners assigned to the project +and to checkout project sources. +It could also be used with the GitLab Container Registry for that project, +allowing pulling and pushing Docker images from within the CI build. + +--- + +GitLab would create a special checkout URL like: + +``` +https://gitlab-ci-token:<project-runners-token>/gitlab.com/gitlab-org/gitlab-ce.git +``` + +And then the users could also use it in their CI builds all Docker related +commands to interact with GitLab Container Registry. For example: + +``` +docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com +``` + +Using single token had multiple security implications: + +- The token would be readable to anyone who had developer access to a project + that could run CI builds, allowing the developer to register any specific + Runner for that project. +- The token would allow to access only the project's sources, forbidding from + accessing any other projects. +- The token was not expiring and was multi-purpose: used for checking out sources, + for registering specific runners and for accessing a project's container + registry with read-write permissions. + +All the above led to a new permission model for builds that was introduced +with GitLab 8.12. + +## Making use of the new CI build permissions model + +With the new build permission model, there is now an easy way to access all +dependent source code in a project. That way, we can: + +1. Access a project's Git submodules +1. Access private container images +1. Access project's and submodule LFS objects + +Let's see how that works with Git submodules and private Docker images hosted on +the container registry. + +## Git submodules + +> +It often happens that while working on one project, you need to use another +project from within it; perhaps it’s a library that a third party developed or +you’re developing a project separately and are using it in multiple parent +projects. +A common issue arises in these scenarios: you want to be able to treat the two +projects as separate yet still be able to use one from within the other. +> +_Excerpt from the [Git website][git-scm] about submodules._ + +If dealing with submodules, your project will probably have a file named +`.gitmodules`. And this is how it usually looks like: + +``` +[submodule "tools"] + path = tools + url = git@gitlab.com/group/tools.git +``` + +> **Note:** +If you are **not** using GitLab 8.12 or higher, you would need to work your way +around this issue in order to access the sources of `gitlab.com/group/tools` +(e.g., use [SSH keys](../ssh_keys/README.md)). +> +With GitLab 8.12 onward, your permissions are used to evaluate what a CI build +can access. More information about how this system works can be found in the +[Build permissions model](../../user/permissions.md#builds-permissions). + +To make use of the new changes, you have to update your `.gitmodules` file to +use a relative URL. + +Let's consider the following example: + +1. Your project is located at `https://gitlab.com/secret-group/my-project`. +1. To checkout your sources you usually use an SSH address like + `git@gitlab.com:secret-group/my-project.git`. +1. Your project depends on `https://gitlab.com/group/tools`. +1. You have the `.gitmodules` file with above content. + +Since Git allows the usage of relative URLs for your `.gitmodules` configuration, +this easily allows you to use HTTP for cloning all your CI builds and SSH +for all your local checkouts. + +For example, if you change the `url` of your `tools` dependency, from +`git@gitlab.com/group/tools.git` to `../../group/tools.git`, this will instruct +Git to automatically deduce the URL that should be used when cloning sources. +Whether you use HTTP or SSH, Git will use that same channel and it will allow +to make all your CI builds use HTTPS (because GitLab CI uses HTTPS for cloning +your sources), and all your local clones will continue using SSH. + +Given the above explanation, your `.gitmodules` file should eventually look +like this: + +``` +[submodule "tools"] + path = tools + url = ../../group/tools.git +``` + +However, you have to explicitly tell GitLab CI to clone your submodules as this +is not done automatically. You can achieve that by adding a `before_script` +section to your `.gitlab-ci.yml`: + +``` +before_script: + - git submodule update --init --recursive + +test: + script: + - run-my-tests +``` + +This will make GitLab CI initialize (fetch) and update (checkout) all your +submodules recursively. + +In case your environment or your Docker image doesn't have Git installed, +you have to either ask your Administrator or install the missing dependency +yourself: + +``` +# Debian / Ubuntu +before_script: + - apt-get update -y + - apt-get install -y git-core + - git submodule update --init --recursive + +# CentOS / RedHat +before_script: + - yum install git + - git submodule update --init --recursive + +# Alpine +before_script: + - apk add -U git + - git submodule update --init --recursive +``` + +### Container Registry + +With the update permission model we also extended the support for accessing +Container Registries for private projects. + +> **Note:** +As GitLab Runner 1.6 doesn't yet incorporate the introduced changes for +permissions, this makes the `image:` directive to not work with private projects +automatically. The manual configuration by an Administrator is required to use +private images. We plan to remove that limitation in one of the upcoming releases. + +Your builds can access all container images that you would normally have access +to. The only implication is that you can push to the Container Registry of the +project for which the build is triggered. + +This is how an example usage can look like: + +``` +test: + script: + - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY + - docker pull $CI_REGISTRY/group/other-project:latest + - docker run $CI_REGISTRY/group/other-project:latest +``` + +[git-scm]: https://git-scm.com/book/en/v2/Git-Tools-Submodules +[build permissions]: ../permissions.md#builds-permissions +[ext]: ../permissions.md#external-users +[triggers]: ../../ci/triggers/README.md diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 7c041d019bb..993c6bfb7e9 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -172,4 +172,4 @@ you commit the changes you will be taken to a new merge request form.  [ce-2808]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2808 -[issue closing pattern]: ../customization/issue_closing.md +[issue closing pattern]: ../user/project/issues/automatic_issue_closing.md diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 08ff89ce6ae..445c0ee8333 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -3,8 +3,8 @@ >**Notes:** > > - [Introduced][ce-3050] in GitLab 8.9. -> - Importing will not be possible if the import instance version is lower -> than that of the exporter. +> - Importing will not be possible if the import instance version differs from +> that of the exporter. > - For existing installations, the project import option has to be enabled in > application settings (`/admin/application_settings`) under 'Import sources'. > You will have to be an administrator to enable and use the import functionality. @@ -17,6 +17,20 @@ Existing projects running on any GitLab instance or GitLab.com can be exported with all their related data and be moved into a new GitLab instance. +## Version history + +| GitLab version | Import/Export version | +| -------- | -------- | +| 8.12.0 to current | 0.1.4 | +| 8.10.3 | 0.1.3 | +| 8.10.0 | 0.1.2 | +| 8.9.5 | 0.1.1 | +| 8.9.0 | 0.1.0 | + + > The table reflects what GitLab version we updated the Import/Export version at. + > For instance, 8.10.3 and 8.11 will have the same Import/Export version (0.1.3) + > and the exports between them will be compatible. + ## Exported contents The following items will be exported: diff --git a/doc/workflow/README.md b/doc/workflow/README.md index dcb9c32ad58..2d9bfbc0629 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -1,6 +1,8 @@ # Workflow +- [Automatic issue closing](../user/project/issues/automatic_issue_closing.md) - [Change your time zone](timezone.md) +- [Cycle Analytics](../user/project/cycle_analytics.md) - [Description templates](../user/project/description_templates.md) - [Feature branch workflow](workflow.md) - [GitLab Flow](gitlab_flow.md) diff --git a/doc/workflow/importing/img/import_projects_from_github_importer.png b/doc/workflow/importing/img/import_projects_from_github_importer.png Binary files differindex b6ed8dd692a..eadd33c695f 100644 --- a/doc/workflow/importing/img/import_projects_from_github_importer.png +++ b/doc/workflow/importing/img/import_projects_from_github_importer.png diff --git a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png Binary files differindex c8f35a50f48..6e91c430a33 100644 --- a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png +++ b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png diff --git a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png Binary files differnew file mode 100644 index 00000000000..c11863ab10c --- /dev/null +++ b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md index 370d885d366..c36dfdb78ec 100644 --- a/doc/workflow/importing/import_projects_from_github.md +++ b/doc/workflow/importing/import_projects_from_github.md @@ -1,54 +1,118 @@ # Import your project from GitHub to GitLab
+Import your projects from GitHub to GitLab with minimal effort.
+
+## Overview
+
>**Note:**
-In order to enable the GitHub import setting, you may also want to
-enable the [GitHub integration][gh-import] in your GitLab instance. This
-configuration is optional, you will be able import your GitHub repositories
-with a Personal Access Token.
+If you are an administrator you can enable the [GitHub integration][gh-import]
+in your GitLab instance sitewide. This configuration is optional, users will be
+able import their GitHub repositories with a [personal access token][gh-token].
-At its current state, GitHub importer can import:
+- At its current state, GitHub importer can import:
+ - the repository description (GitLab 7.7+)
+ - the Git repository data (GitLab 7.7+)
+ - the issues (GitLab 7.7+)
+ - the pull requests (GitLab 8.4+)
+ - the wiki pages (GitLab 8.4+)
+ - the milestones (GitLab 8.7+)
+ - the labels (GitLab 8.7+)
+ - the release note descriptions (GitLab 8.12+)
+- References to pull requests and issues are preserved (GitLab 8.7+)
+- Repository public access is retained. If a repository is private in GitHub
+ it will be created as private in GitLab as well.
-- the repository description (introduced in GitLab 7.7)
-- the git repository data (introduced in GitLab 7.7)
-- the issues (introduced in GitLab 7.7)
-- the pull requests (introduced in GitLab 8.4)
-- the wiki pages (introduced in GitLab 8.4)
-- the milestones (introduced in GitLab 8.7)
-- the labels (introduced in GitLab 8.7)
-- the release note descriptions (introduced in GitLab 8.12)
+## How it works
-With GitLab 8.7+, references to pull requests and issues are preserved.
+When issues/pull requests are being imported, the GitHub importer tries to find
+the GitHub author/assignee in GitLab's database using the GitHub ID. For this
+to work, the GitHub author/assignee should have signed in beforehand in GitLab
+and [**associated their GitHub account**][social sign-in]. If the user is not
+found in GitLab's database, the project creator (most of the times the current
+user that started the import process) is set as the author, but a reference on
+the issue about the original GitHub author is kept.
-The importer page is visible when you [create a new project][new-project].
-Click on the **GitHub** link and, if you are logged in via the GitHub
-integration, you will be redirected to GitHub for permission to access your
-projects. After accepting, you'll be automatically redirected to the importer.
+The importer will create any new namespaces (groups) if they don't exist or in
+the case the namespace is taken, the repository will be imported under the user's
+namespace that started the import process.
-If you are not using the GitHub integration, you can still perform a one-off
-authorization with GitHub to access your projects.
+## Importing your GitHub repositories
-Alternatively, you can also enter a GitHub Personal Access Token. Once you enter
-your token, you'll be taken to the importer.
+The importer page is visible when you create a new project.
----
+Click on the **GitHub** link and the import authorization process will start.
+There are two ways to authorize access to your GitHub repositories:
+
+1. [Using the GitHub integration][gh-integration] (if it's enabled by your
+ GitLab administrator). This is the preferred way as it's possible to
+ preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works)
+ section.
+1. [Using a personal access token][gh-token] provided by GitHub.
+
+
+
+### Authorize access to your repositories using the GitHub integration
+
+If the [GitHub integration][gh-import] is enabled by your GitLab administrator,
+you can use it instead of the personal access token.
+
+1. First you may want to connect your GitHub account to GitLab in order for
+ the username mapping to be correct. Follow the [social sign-in] documentation
+ on how to do so.
+1. Once you connect GitHub, click the **List your GitHub repositories** button
+ and you will be redirected to GitHub for permission to access your projects.
+1. After accepting, you'll be automatically redirected to the importer.
+
+You can now go on and [select which repositories to import](#select-which-repositories-to-import).
+
+### Authorize access to your repositories using a personal access token
+
+>**Note:**
+For a proper author/assignee mapping for issues and pull requests, the
+[GitHub integration][gh-integration] should be used instead of the
+[personal access token][gh-token]. If the GitHub integration is enabled by your
+GitLab administrator, it should be the preferred method to import your repositories.
+Read more in the [How it works](#how-it-works) section.
+
+If you are not using the GitHub integration, you can still perform a one-off
+authorization with GitHub to grant GitLab access your repositories:
+
+1. Go to <https://github.com/settings/tokens/new>.
+1. Enter a token description.
+1. Check the `repo` scope.
+1. Click **Generate token**.
+1. Copy the token hash.
+1. Go back to GitLab and provide the token to the GitHub importer.
+1. Hit the **List your GitHub repositories** button and wait while GitLab reads
+ your repositories' information. Once done, you'll be taken to the importer
+ page to select the repositories to import.
+
+### Select which repositories to import
+
+After you've authorized access to your GitHub repositories, you will be
+redirected to the GitHub importer page.
+
+From there, you can see the import statuses of your GitHub repositories.
+
+- Those that are being imported will show a _started_ status,
+- those already successfully imported will be green with a _done_ status,
+- whereas those that are not yet imported will have an **Import** button on the
+ right side of the table.
-While at the GitHub importer page, you can see the import statuses of your
-GitHub projects. Those that are being imported will show a _started_ status,
-those already imported will be green, whereas those that are not yet imported
-have an **Import** button on the right side of the table. If you want, you can
-import all your GitHub projects in one go by hitting **Import all projects**
-in the upper left corner.
+If you want, you can import all your GitHub projects in one go by hitting
+**Import all projects** in the upper left corner.
---
-The importer will create any new namespaces if they don't exist or in the
-case the namespace is taken, the project will be imported on the user's
-namespace.
+You can also choose a different name for the project and a different namespace,
+if you have the privileges to do so.
[gh-import]: ../../integration/github.md "GitHub integration"
-[ee-gh]: http://docs.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"
[new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"
+[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration
+[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token
+[social sign-in]: ../../profile/account/social_sign_in.md
diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md index 9dc1e9b47e3..b3c73e947f0 100644 --- a/doc/workflow/lfs/lfs_administration.md +++ b/doc/workflow/lfs/lfs_administration.md @@ -45,5 +45,5 @@ In `config/gitlab.yml`: * Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported * Currently, removing LFS objects from GitLab Git LFS storage is not supported -* LFS authentications via SSH is not supported for the time being -* Only compatible with the GitLFS client versions 1.1.0 or 1.0.2. +* LFS authentications via SSH was added with GitLab 8.12 +* Only compatible with the GitLFS client versions 1.1.0 and up, or 1.0.2. diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 9fe065fa680..1a4f213a792 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -35,6 +35,10 @@ Documentation for GitLab instance administrators is under [LFS administration do credentials store is recommended * Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting) + +>**Note**: With 8.12 GitLab added LFS support to SSH. The Git LFS communication + still goes over HTTP, but now the SSH client passes the correct credentials + to the Git LFS client, so no action is required by the user. ## Using Git LFS @@ -132,6 +136,10 @@ git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs" ### Credentials are always required when pushing an object +>**Note**: With 8.12 GitLab added LFS support to SSH. The Git LFS communication + still goes over HTTP, but now the SSH client passes the correct credentials + to the Git LFS client, so no action is required by the user. + Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing the LFS object on every push for every object, user HTTPS credentials are required. |
