diff options
Diffstat (limited to 'doc')
28 files changed, 520 insertions, 48 deletions
diff --git a/doc/README.md b/doc/README.md index 1a7638b3d7e..2b3b0998fcc 100644 --- a/doc/README.md +++ b/doc/README.md @@ -115,6 +115,7 @@ Manage files and branches from the UI (user interface): - [Project Services](user/project/integrations/project_services.md): Integrate a project with external services, such as CI and chat. - [GitLab Integration](integration/README.md): Integrate with multiple third-party services with GitLab to allow external issue trackers and external authentication. +- [Trello Power-Up](integration/trello_power_up.md): Integrate with GitLab's Trello Power-Up ---- diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index c8987dea5e2..3449f9e15ce 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -228,7 +228,7 @@ Tip: If you want to limit access to the nested members of an Active Directory group you can use the following syntax: ``` -(memberOf:1.2.840.113556.1.4.1941=CN=My Group,DC=Example,DC=com) +(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) ``` Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index afafb6bf1f5..8cb0e5b1562 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -465,23 +465,42 @@ on how to achieve that. ## Disable Container Registry but use GitLab as an auth endpoint -You can disable the embedded Container Registry to use an external one, but -still use GitLab as an auth endpoint. - **Omnibus GitLab** + +You can use GitLab as an auth endpoint and use a non-bundled Container Registry. + 1. Open `/etc/gitlab/gitlab.rb` and set necessary configurations: ```ruby - registry['enable'] = false gitlab_rails['registry_enabled'] = true gitlab_rails['registry_host'] = "registry.gitlab.example.com" gitlab_rails['registry_port'] = "5005" gitlab_rails['registry_api_url'] = "http://localhost:5000" - gitlab_rails['registry_key_path'] = "/var/opt/gitlab/gitlab-rails/certificate.key" gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" ``` +1. A certificate keypair is required for GitLab and the Container Registry to + communicate securely. By default omnibus-gitlab will generate one keypair, + which is saved to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key`. + When using an non-bundled Container Registry, you will need to supply a + custom certificate key. To do that, add the following to + `/etc/gitlab/gitlab.rb` + + ```ruby + gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" + # registry['internal_key'] should contain the contents of the custom key + # file. Line breaks in the key file should be marked using `\n` character + # Example: + registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" + ``` + + **Note:** The file specified at `registry_key_path` gets populated with the + content specified by `internal_key`, each time reconfigure is executed. If + no file is specified, omnibus-gitlab will default it to + `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate + it. + 1. Save the file and [reconfigure GitLab][] for the changes to take effect. **Installations from source** diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 6023112d615..6baae20d16a 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -20,31 +20,45 @@ it, the client IP needs to be [included in a whitelist][whitelist]. Currently the embedded Prometheus server is not automatically configured to collect metrics from this endpoint. We recommend setting up another Prometheus server, because the embedded server configuration is overwritten once every -[reconfigure of GitLab][reconfigure]. In the future this will not be required. +[reconfigure of GitLab][reconfigure]. In the future this will not be required. ## Metrics available In this experimental phase, only a few metrics are available: -| Metric | Type | Description | -| --------------------------------- | --------- | ----------- | -| db_ping_timeout | Gauge | Whether or not the last database ping timed out | -| db_ping_success | Gauge | Whether or not the last database ping succeeded | -| db_ping_latency_seconds | Gauge | Round trip time of the database ping | -| filesystem_access_latency_seconds | Gauge | Latency in accessing a specific filesystem | -| filesystem_accessible | Gauge | Whether or not a specific filesystem is accessible | -| filesystem_write_latency_seconds | Gauge | Write latency of a specific filesystem | -| filesystem_writable | Gauge | Whether or not the filesystem is writable | -| filesystem_read_latency_seconds | Gauge | Read latency of a specific filesystem | -| filesystem_readable | Gauge | Whether or not the filesystem is readable | -| http_requests_total | Counter | Rack request count | -| http_request_duration_seconds | Histogram | HTTP response time from rack middleware | -| pipelines_created_total | Counter | Counter of pipelines created | -| rack_uncaught_errors_total | Counter | Rack connections handling uncaught errors count | -| redis_ping_timeout | Gauge | Whether or not the last redis ping timed out | -| redis_ping_success | Gauge | Whether or not the last redis ping succeeded | -| redis_ping_latency_seconds | Gauge | Round trip time of the redis ping | -| user_session_logins_total | Counter | Counter of how many users have logged in | +| Metric | Type | Since | Description | +|:--------------------------------- |:--------- |:----- |:----------- | +| db_ping_timeout | Gauge | 9.4 | Whether or not the last database ping timed out | +| db_ping_success | Gauge | 9.4 | Whether or not the last database ping succeeded | +| db_ping_latency_seconds | Gauge | 9.4 | Round trip time of the database ping | +| filesystem_access_latency_seconds | Gauge | 9.4 | Latency in accessing a specific filesystem | +| filesystem_accessible | Gauge | 9.4 | Whether or not a specific filesystem is accessible | +| filesystem_write_latency_seconds | Gauge | 9.4 | Write latency of a specific filesystem | +| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable | +| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem | +| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable | +| http_requests_total | Counter | 9.4 | Rack request count | +| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | +| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | +| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | +| redis_ping_timeout | Gauge | 9.4 | Whether or not the last redis ping timed out | +| redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | +| redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | +| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | + +## Metrics shared directory + +GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. +Those files are shared among all instances running under Unicorn server. +The directory needs to be accessible to all running Unicorn's processes otherwise +metrics will not function correctly. + +For best performance its advisable that this directory will be located in `tmpfs`. + +Its location is configured using environment variable `prometheus_multiproc_dir`. + +If GitLab is installed using Omnibus and `tmpfs` is available then metrics +directory will be automatically configured. [← Back to the main Prometheus page](index.md) diff --git a/doc/api/README.md b/doc/api/README.md index 95e7a457848..a888c0ebb4e 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -29,7 +29,8 @@ following locations: - [Keys](keys.md) - [Labels](labels.md) - [Merge Requests](merge_requests.md) -- [Milestones](milestones.md) +- [Project milestones](milestones.md) +- [Group milestones](group_milestones.md) - [Namespaces](namespaces.md) - [Notes](notes.md) (comments) - [Notification settings](notification_settings.md) diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md new file mode 100644 index 00000000000..086fba7e91d --- /dev/null +++ b/doc/api/group_milestones.md @@ -0,0 +1,120 @@ +# Group milestones API + +## List group milestones + +Returns a list of group milestones. + +``` +GET /groups/:id/milestones +GET /groups/:id/milestones?iids=42 +GET /groups/:id/milestones?iids[]=42&iids[]=43 +GET /groups/:id/milestones?state=active +GET /groups/:id/milestones?state=closed +GET /groups/:id/milestones?search=version +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `iids` | Array[integer] | optional | Return only the milestones having the given `iids` | +| `state` | string | optional | Return only `active` or `closed` milestones` | +| `search` | string | optional | Return only milestones with a title or description matching the provided string | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/milestones +``` + +Example Response: + +```json +[ + { + "id": 12, + "iid": 3, + "group_id": 16, + "title": "10.0", + "description": "Version", + "due_date": "2013-11-29", + "start_date": "2013-11-10", + "state": "active", + "updated_at": "2013-10-02T09:24:18Z", + "created_at": "2013-10-02T09:24:18Z" + } +] +``` + + +## Get single milestone + +Gets a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of the group milestone + +## Create new milestone + +Creates a new group milestone. + +``` +POST /groups/:id/milestones +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `title` (required) - The title of an milestone +- `description` (optional) - The description of the milestone +- `due_date` (optional) - The due date of the milestone +- `start_date` (optional) - The start date of the milestone + +## Edit milestone + +Updates an existing group milestone. + +``` +PUT /groups/:id/milestones/:milestone_id +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone +- `title` (optional) - The title of a milestone +- `description` (optional) - The description of a milestone +- `due_date` (optional) - The due date of the milestone +- `start_date` (optional) - The start date of the milestone +- `state_event` (optional) - The state event of the milestone (close|activate) + +## Get all issues assigned to a single milestone + +Gets all issues assigned to a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id/issues +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone + +## Get all merge requests assigned to a single milestone + +Gets all merge requests assigned to a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id/merge_requests +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone diff --git a/doc/api/issues.md b/doc/api/issues.md index a00a63bad4b..0e391c75cd3 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -356,7 +356,13 @@ Example response: "user_notes_count": 1, "due_date": null, "web_url": "http://example.com/example/example/issues/1", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -418,7 +424,13 @@ Example response: "user_notes_count": 0, "due_date": null, "web_url": "http://example.com/example/example/issues/14", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -481,7 +493,13 @@ Example response: "user_notes_count": 0, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/15", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -567,7 +585,13 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -632,7 +656,13 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 61ae89a64c0..d3f8e509612 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -99,7 +99,16 @@ Parameters: "repository_size": 1038090, "lfs_objects_size": 0, "job_artifacts_size": 0 - } + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + }, }, { "id": 6, @@ -168,6 +177,15 @@ Parameters: "repository_size": 2066080, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } } ] @@ -257,6 +275,15 @@ Parameters: "repository_size": 1038090, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } }, { @@ -326,6 +353,15 @@ Parameters: "repository_size": 2066080, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } } ] @@ -427,6 +463,15 @@ Parameters: "repository_size": 1038090, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } } ``` @@ -659,7 +704,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -725,7 +779,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -809,7 +872,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -893,7 +965,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` diff --git a/doc/api/v3_to_v4.md b/doc/api/v3_to_v4.md index 9db8e0351cf..9835fab7c98 100644 --- a/doc/api/v3_to_v4.md +++ b/doc/api/v3_to_v4.md @@ -2,9 +2,11 @@ Since GitLab 9.0, API V4 is the preferred version to be used. -API V3 will be removed in GitLab 9.5, to be released on August 22, 2017. In the -meantime, we advise you to make any necessary changes to applications that use -V3. The V3 API documentation is still [available](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/doc/api/README.md). +API V3 will be unsupported from GitLab 9.5, to be released on August +22, 2017. It will be removed in GitLab 9.5 or later. In the meantime, we advise +you to make any necessary changes to applications that use V3. The V3 API +documentation is still +[available](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/doc/api/README.md). Below are the changes made between V3 and V4. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 724843a4d56..e12ef6e2685 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -395,6 +395,7 @@ job_name: | after_script | no | Override a set of commands that are executed after job | | environment | no | Defines a name of environment to which deployment is done by this job | | coverage | no | Define code coverage settings for a given job | +| retry | no | Define how many times a job can be auto-retried in case of a failure | ### script @@ -1129,9 +1130,33 @@ A simple example: ```yaml job1: + script: rspec coverage: '/Code coverage: \d+\.\d+/' ``` +### retry + +**Notes:** +- [Introduced][ce-3442] in GitLab 9.5. + +`retry` allows you to configure how many times a job is going to be retried in +case of a failure. + +When a job fails, and has `retry` configured it is going to be processed again +up to the amount of times specified by the `retry` keyword. + +If `retry` is set to 2, and a job succeeds in a second run (first retry), it won't be retried +again. `retry` value has to be a positive integer, equal or larger than 0, but +lower or equal to 2 (two retries maximum, three runs in total). + +A simple example: + +```yaml +test: + script: rspec + retry: 2 +``` + ## Git Strategy > Introduced in GitLab 8.9 as an experimental feature. May change or be removed @@ -1506,3 +1531,4 @@ CI with various languages. [variables]: ../variables/README.md [ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 [ce-7447]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7447 +[ce-3442]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3442 diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 72a34aa7de9..e67db9ff142 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -31,6 +31,18 @@ It's also possible for different migrations to be executed at the same time. This means that different background migrations should not migrate data in a way that would cause conflicts. +## Idempotence + +Background migrations are executed in a context of a Sidekiq process. +Usual Sidekiq rules apply, especially the rule that jobs should be small +and idempotent. + +See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices) +for more details. + +Make sure that in case that your migration job is going to be retried data +integrity is guarateed. + ## How It Works Background migrations are simple classes that define a `perform` method. A @@ -212,3 +224,27 @@ end This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. + +## Testing + +It is required to write tests for background migrations' scheduling migration +(either a regular migration or a post deployment migration), background +migration itself and a cleanup migration. You can use the `:migration` RSpec +tag when testing a regular / post deployment migration. +See [README][migrations-readme]. + +When you do that, keep in mind that `before` and `after` RSpec hooks are going +to migrate you database down and up, which can result in other background +migrations being called. That means that using `spy` test doubles with +`have_received` is encouraged, instead of using regular test doubles, because +your expectations defined in a `it` block can conflict with what is being +called in RSpec hooks. See [gitlab-org/gitlab-ce#35351][issue-rspec-hooks] +for more details. + +## Best practices + +1. Make sure that background migration jobs are idempotent. +1. Make sure that tests you write are not false positives. + +[migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md +[issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 4ed89146072..e3f37616757 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -133,6 +133,55 @@ reviewee. tomorrow. When you are not able to find the right balance, ask other people about their opinion. +### GitLab-specific concerns + +GitLab is used in a lot of places. Many users use +our [Omnibus packages](https://about.gitlab.com/installation/), but some use +the [Docker images](https://docs.gitlab.com/omnibus/docker/), some are +[installed from source](https://docs.gitlab.com/ce/install/installation.html), +and there are other installation methods available. GitLab.com itself is a large +Enterprise Edition instance. This has some implications: + +1. **Query changes** should be tested to ensure that they don't result in worse + performance at the scale of GitLab.com: + 1. Generating large quantities of data locally can help. + 2. Asking for query plans from GitLab.com is the most reliable way to validate + these. +2. **Database migrations** must be: + 1. Reversible. + 2. Performant at the scale of GitLab.com - ask a maintainer to test the + migration on the staging environment if you aren't sure. + 3. Categorised correctly: + - Regular migrations run before the new code is running on the instance. + - [Post-deployment migrations](post_deployment_migrations.md) run _after_ + the new code is deployed, when the instance is configured to do that. + - [Background migrations](background_migrations.md) run in Sidekiq, and + should only be done for migrations that would take an extreme amount of + time at GitLab.com scale. +3. **Sidekiq workers** + [cannot change in a backwards-incompatible way](sidekiq_style_guide.md#removing-or-renaming-queues): + 1. Sidekiq queues are not drained before a deploy happens, so there will be + workers in the queue from the previous version of GitLab. + 2. If you need to change a method signature, try to do so across two releases, + and accept both the old and new arguments in the first of those. + 3. Similarly, if you need to remove a worker, stop it from being scheduled in + one release, then remove it in the next. This will allow existing jobs to + execute. + 4. Don't forget, not every instance will upgrade to every intermediate version + (some people may go from X.1.0 to X.10.0, or even try bigger upgrades!), so + try to be liberal in accepting the old format if it is cheap to do so. +4. **Cached values** may persist across releases. If you are changing the type a + cached value returns (say, from a string or nil to an array), change the + cache key at the same time. +5. **Settings** should be added as a + [last resort](https://about.gitlab.com/handbook/product/#convention-over-configuration). + If you're adding a new setting in `gitlab.yml`: + 1. Try to avoid that, and add to `ApplicationSetting` instead. + 2. Ensure that it is also + [added to Omnibus](https://docs.gitlab.com/omnibus/settings/gitlab.yml.html#adding-a-new-setting-to-gitlab-yml). +6. **Filesystem access** can be slow, so try to avoid + [shared files](shared_files.md) when an alternative solution is available. + ### Credits Largely based on the [thoughtbot code review guide]. diff --git a/doc/development/fe_guide/performance.md b/doc/development/fe_guide/performance.md index 2ddcbe13afa..f25313d6cff 100644 --- a/doc/development/fe_guide/performance.md +++ b/doc/development/fe_guide/performance.md @@ -23,6 +23,18 @@ controlled by the server. 1. The backend code will most likely be using etags. You do not and should not check for status `304 Not Modified`. The browser will transform it for you. +### Lazy Loading + +To improve the time to first render we are using lazy loading for images. This works by setting +the actual image source on the `data-src` attribute. After the HTML is rendered and JavaScript is loaded, +the value of `data-src` will be moved to `src` automatically if the image is in the current viewport. + +* Prepare images in HTML for lazy loading by renaming the `src` attribute to `data-src` +* If you are using the Rails `image_tag` helper, all images will be lazy-loaded by default unless `lazy: false` is provided. + +If you are asynchronously adding content which contains lazy images then you need to call the function +`gl.lazyLoader.searchLazyImages()` which will search for lazy images and load them if needed. + ## Reducing Asset Footprint ### Page-specific JavaScript diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index ae844fa1051..149a0159680 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -447,6 +447,7 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `name` 1. `props` 1. `mixins` + 1. `directives` 1. `data` 1. `components` 1. `computedProps` diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index e3a20f29a09..1e9fdbc65e2 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -36,3 +36,10 @@ slow jobs blocking work (even for different jobs) on the shared queue. Each Sidekiq worker must be tested using RSpec, just like any other class. These tests should be placed in `spec/workers`. + +## Removing or renaming queues + +Try to avoid renaming or removing queues in minor and patch releases. +During online update instance can have pending jobs and removing the queue can +lead to those jobs being stuck forever. If you can't write migration for those +Sidekiq jobs, please consider doing rename or remove queue in major release only.
\ No newline at end of file diff --git a/doc/install/installation.md b/doc/install/installation.md index 5e981b0b3e7..8ded607bcab 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -64,7 +64,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake If you want to use Kerberos for user authentication, then install libkrb5-dev: diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 2dd9b33273c..372e1909330 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -4,14 +4,12 @@ GitLab has a great issue tracker but you can also use an external one such as Jira, Redmine, or Bugzilla. Issue trackers are configurable per GitLab project and allow you to do the following: -- the **Issues** link on the GitLab project pages takes you to the appropriate - issue index of the external tracker -- clicking **New issue** on the project dashboard creates a new issue on the - external tracker - you can reference these external issues inside GitLab interface (merge requests, commits, comments) and they will be automatically converted into links +You can have enabled both external and internal GitLab issue trackers in parallel. The **Issues** link always opens the internal issue tracker and in case the internal issue tracker is disabled the link is not visible in the menu. + ## Configuration The configuration is done via a project's **Services**. diff --git a/doc/integration/img/enable_trello_powerup.png b/doc/integration/img/enable_trello_powerup.png Binary files differnew file mode 100644 index 00000000000..65d01f1c38c --- /dev/null +++ b/doc/integration/img/enable_trello_powerup.png diff --git a/doc/integration/img/trello_card_with_gitlab_powerup.png b/doc/integration/img/trello_card_with_gitlab_powerup.png Binary files differnew file mode 100644 index 00000000000..2965dc35855 --- /dev/null +++ b/doc/integration/img/trello_card_with_gitlab_powerup.png diff --git a/doc/integration/trello_power_up.md b/doc/integration/trello_power_up.md new file mode 100644 index 00000000000..d264486a872 --- /dev/null +++ b/doc/integration/trello_power_up.md @@ -0,0 +1,42 @@ +# Trello Power-Up + +GitLab's Trello Power-Up enables you to seamlessly attach +GitLab **merge requests** to Trello cards. + + + +## Configuring the Power-Up + +In order to get started, you will need to configure your Power-Up. + +In Trello: + +1. Go to your Trello board +1. Select `Power-Ups` to see a listing of all the available Power-Ups +1. Look for a row that says `GitLab` and select the `Enable` button +1. Select the `Settings` (gear) icon +1. In the popup menu, select `Authorize Account` + +In this popup, fill in your `API URL` and `Personal Access Token`. After that, you will be able to attach any merge request to any Trello card on your selected Trello board. + +## What is my API URL? + +Your API URL should be your GitLab instance URL with `/api/v4` appended in the end of the URL. +For example, if your GitLab instance URL is `https://gitlab.com`, your API URL would be `https://gitlab.com/api/v4`. +If your instance's URL is `https://example.com`, your API URL will be `https://example.com/api/v4`. + + + +## What is my Personal Access Token? + +Your GitLab's personal access token will enable your GitLab account to be accessed +from Trello. + +> Find it in GitLab by clicking on your avatar (upright corner), from which you access +your user **Settings** > **Access Tokens**. + +Learn more about generating a personal access token in the +[Personal Access Token Documentation][personal-access-token-documentation]. +Don't forget to check the API scope checkbox! + +[personal-access-token-documentation]: ../user/profile/personal_access_tokens.html diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 6a040516231..ba2adc1afda 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -20,10 +20,12 @@ Once you have configured and enabled Bugzilla: ## Referencing issues in Bugzilla Issues in Bugzilla can be referenced in two alternative ways: -1. `#<ID>` where `<ID>` is a number (example `#143`) +1. `#<ID>` where `<ID>` is a number (example `#143`). 2. `<PROJECT>-<ID>` where `<PROJECT>` starts with a capital letter which is then followed by capital letters, numbers or underscores, and `<ID>` is a number (example `API_32-143`). +We suggest using the longer format if you have both internal and external issue trackers enabled. If you use the shorter format and an issue with the same ID exists in the internal issue tracker the internal issue will be linked. + Please note that `<PROJECT>` part is ignored and links always point to the address specified in `issues_url`. diff --git a/doc/user/project/integrations/img/webhook_testing.png b/doc/user/project/integrations/img/webhook_testing.png Binary files differnew file mode 100644 index 00000000000..176dcec9d8a --- /dev/null +++ b/doc/user/project/integrations/img/webhook_testing.png diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md new file mode 100644 index 00000000000..309da610cc0 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -0,0 +1,20 @@ +# Monitoring HA Proxy +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 + +GitLab has support for automatically detecting and monitoring HA Proxy. This is provided by leveraging the [HA Proxy Exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates HA Proxy statistics into a Prometheus readable form. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | + +## Configuring Prometheus to monitor for HA Proxy metrics + +To get started with NGINX monitoring, you should install and configure the [HA Proxy exporter](https://github.com/prometheus/haproxy_exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md index 55146e57370..546e1f51df5 100644 --- a/doc/user/project/integrations/prometheus_library/metrics.md +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -4,6 +4,7 @@ GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). Currently supported exporters are: * [Kubernetes](kubernetes.md) * [NGINX](nginx.md) +* [HA Proxy](haproxy.md) * [Amazon Cloud Watch](cloudwatch.md) We have tried to surface the most important metrics for each exporter, and will be continuing to add support for additional exporters in future releases. If you would like to add support for other official exporters, [contributions](#adding-to-the-library) are welcome. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index fe238e74e36..b3470773996 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -9,7 +9,7 @@ GitLab has support for automatically detecting and monitoring NGINX. This is pro | ---- | ----- | | Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) * 1000 | -| HTTP Error Rate (%) | sum(nginx_responses_total{status_code="5xx", %{environment_filter}}) / sum(nginx_responses_total{server_zone!="*", server_zone!="_", %{environment_filter}}) | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_responses_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_responses_total{%{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for NGINX metrics diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 8026f1f57bc..cf92465da53 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -30,5 +30,7 @@ Issues in Redmine can be referenced in two alternative ways: then followed by capital letters, numbers or underscores, and `<ID>` is a number (example `API_32-143`). +We suggest using the longer format if you have both internal and external issue trackers enabled. If you use the shorter format and an issue with the same ID exists in the internal issue tracker the internal issue will be linked. + Please note that `<PROJECT>` part is ignored and links always point to the address specified in `issues_url`. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 023c6932e41..c03a2df9a72 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1014,6 +1014,13 @@ X-Gitlab-Event: Build Hook } ``` +## Testing webhooks + +You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. +> For example: for triggering `Push Events` your project should have at least one commit. + + + ## Troubleshoot webhooks Gitlab stores each perform of the webhook. @@ -1056,7 +1063,7 @@ Pick an unused port (e.g. 8000) and start the script: `ruby print_http_body.rb 8000`. Then add your server as a webhook receiver in GitLab as `http://my.host:8000/`. -When you press 'Test Hook' in GitLab, you should see something like this in the +When you press 'Test' in GitLab, you should see something like this in the console: ``` diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 19b51c83222..ce4dd4e99d5 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -37,3 +37,4 @@ do. | `/target_branch <Branch Name>` | Set target branch for current merge request | | `/award :emoji:` | Toggle award for :emoji: | | `/board_move ~column` | Move issue to column on the board | +| `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | |
