Add latest changes from gitlab-org/gitlab@master

8 files changed, 142 insertions, 70 deletions

diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
index 3b98ac7316d..3c2812d8b52 100644
--- a/doc/administration/gitaly/praefect.md
+++ b/doc/administration/gitaly/praefect.md
@@ -322,28 +322,6 @@ application server, or a Gitaly node.
    }
    ```
 
-1. Enable automatic failover by editing `/etc/gitlab/gitlab.rb`:
-
-   ```ruby
-   praefect['failover_enabled'] = true
-   praefect['failover_election_strategy'] = 'sql'
-   ```
-
-   When automatic failover is enabled, Praefect checks the health of internal
-   Gitaly nodes. If the primary has a certain amount of health checks fail, it
-   will promote one of the secondaries to be primary, and demote the primary to
-   be a secondary.
-
-   NOTE: **Note:** Database leader election will be [enabled by default in the
-   future](https://gitlab.com/gitlab-org/gitaly/-/issues/2682).
-
-   Caution, **automatic failover** favors availability over consistency and will
-   cause data loss if changes have not been replicated to the newly elected
-   primary. In the next release, leader election will [prefer to promote up to
-   date replicas](https://gitlab.com/gitlab-org/gitaly/-/issues/2642), and it
-   will be an option to favor consistency by marking [out-of-date repositories
-   read-only](https://gitlab.com/gitlab-org/gitaly/-/issues/2630).
-
 1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure
    Praefect](../restart_gitlab.md#omnibus-gitlab-reconfigure):
 
@@ -738,7 +716,7 @@ Praefect regularly checks the health of each backend Gitaly node. This
 information can be used to automatically failover to a new primary node if the
 current primary node is found to be unhealthy.
 
-- **PostgreSQL (recommended):** Enabled by setting
+- **PostgreSQL (recommended):** Enabled by default, and equivalent to:
   `praefect['failover_election_strategy'] = sql`. This configuration
   option will allow multiple Praefect nodes to coordinate via the
   PostgreSQL database to elect a primary Gitaly node. This configuration
@@ -749,18 +727,13 @@ current primary node is found to be unhealthy.
   reconfigured in `/etc/gitlab/gitlab.rb` on the Praefect node. Modify the
   `praefect['virtual_storages']` field by moving the `primary = true` to promote
   a different Gitaly node to primary. In the steps above, `gitaly-1` was set to
-  the primary.
-- **Memory:** Enabled by setting `praefect['failover_enabled'] = true` in
-  `/etc/gitlab/gitlab.rb` on the Praefect node. If a sufficient number of health
+  the primary. Requires `praefect['failover_enabled'] = false` in the configuration.
+- **Memory:** Enabled by setting `praefect['failover_election_strategy'] = 'local'`
+  in `/etc/gitlab/gitlab.rb` on the Praefect node. If a sufficient number of health
   checks fail for the current primary backend Gitaly node, and new primary will
   be elected. **Do not use with multiple Praefect nodes!** Using with multiple
   Praefect nodes is likely to result in a split brain.
 
-NOTE: **Note:**: Praefect does not yet account for replication lag on
-the secondaries during the election process, so data loss can occur
-during a failover. Follow issue
-[#2642](https://gitlab.com/gitlab-org/gitaly/-/issues/2642) for updates.
-
 It is likely that we will implement support for Consul, and a cloud native
 strategy in the future.
 
diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md
index 39819ccd79b..1dea2de73f6 100644
--- a/doc/administration/object_storage.md
+++ b/doc/administration/object_storage.md
@@ -141,10 +141,88 @@ Using the default GitLab settings, some object storage back-ends such as
 and [Alibaba](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1564)
 might generate `ETag mismatch` errors.
 
+If you are seeing this ETag mismatch error with Amazon Web Services S3,
+it's likely this is due to [encryption settings on your bucket](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html).
+See the section on [using Amazon instance profiles](#using-amazon-instance-profiles) on how to fix this issue.
+
 When using GitLab direct upload, the
 [workaround for MinIO](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1564#note_244497658)
 is to use the `--compat` parameter on the server.
 
-We are working on a fix to GitLab component Workhorse, and also
-a workaround, in the mean time, to
-[allow ETag verification to be disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18175).
+We are working on a fix to the [GitLab Workhorse
+component](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/222).
+
+### Using Amazon instance profiles
+
+Instead of supplying AWS access and secret keys in object storage
+configuration, GitLab can be configured to use IAM roles to set up an
+[Amazon instance profile](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html).
+When this is used, GitLab will fetch temporary credentials each time an
+S3 bucket is accessed, so no hard-coded values are needed in the
+configuration.
+
+#### Encrypted S3 buckets
+
+> Introduced in [GitLab 13.1](https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/466) only for instance profiles.
+
+When configured to use an instance profile, GitLab Workhorse
+will properly upload files to S3 buckets that have [SSE-S3 or SSE-KMS
+encryption enabled by default](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html).
+Note that customer master keys (CMKs) and SSE-C encryption are not yet
+supported since this requires supplying keys to the GitLab
+configuration.
+
+Without instance profiles enabled (or prior to GitLab 13.1), GitLab
+Workhorse will upload files to S3 using pre-signed URLs that do not have
+a `Content-MD5` HTTP header computed for them. To ensure data is not
+corrupted, Workhorse checks that the MD5 hash of the data sent equals
+the ETag header returned from the S3 server. When encryption is enabled,
+this is not the case, which causes Workhorse to report an `ETag
+mismatch` error during an upload.
+
+With instance profiles enabled, GitLab Workhorse uses an AWS S3 client
+that properly computes and sends the `Content-MD5` header to the server,
+which eliminates the need for comparing ETag headers. If the data is
+corrupted in transit, the S3 server will reject the file.
+
+#### IAM Permissions
+
+To set up an instance profile, create an Amazon Identity Access and
+Management (IAM) role with the necessary permissions. The following
+example is a role for an S3 bucket named `test-bucket`:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": [
+                "s3:PutObject",
+                "s3:GetObject",
+                "s3:AbortMultipartUpload",
+                "s3:DeleteObject"
+            ],
+            "Resource": "arn:aws:s3:::test-bucket/*"
+        }
+    ]
+}
+```
+
+Associate this role with your GitLab instance, and then configure GitLab
+to use it via the `use_iam_profile` configuration option. For example,
+when configuring uploads to use object storage, see the `AWS IAM profiles`
+section in [S3 compatible connection settings](uploads.md#s3-compatible-connection-settings).
+
+#### Disabling the feature
+
+The Workhorse S3 client is only enabled when the `use_iam_profile`
+configuration flag is `true`.
+
+To disable this feature, ask a GitLab administrator with [Rails console access](feature_flags.md#how-to-enable-and-disable-features-behind-flags) to run the
+following command:
+
+```ruby
+Feature.disable(:use_workhorse_s3_client)
+```
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index 7e330682f8c..33af356b37d 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -380,39 +380,6 @@ user = User.find_by_username ''
 user.skip_reconfirmation!
 ```
 
-### Get an admin token
-
-```ruby
-# Get the first admin's first access token (no longer works on 11.9+. see: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22743)
-User.where(admin:true).first.personal_access_tokens.first.token
-
-# Get the first admin's private token (no longer works on 10.2+)
-User.where(admin:true).private_token
-```
-
-### Create personal access token
-
-```ruby
-personal_access_token = User.find(123).personal_access_tokens.create(
-  name: 'apitoken',
-  impersonation: false,
-  scopes: [:api]
-)
-
-puts personal_access_token.token
-```
-
-You might also want to manually set the token string:
-
-```ruby
-User.find(123).personal_access_tokens.create(
-  name: 'apitoken',
-  token_digest: Gitlab::CryptoHelper.sha256('some-token-string-here'),
-  impersonation: false,
-  scopes: [:api]
-)
-```
-
 ### Active users & Historical users
 
 ```ruby
diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md
index e963bdd6848..bdd372e90ed 100644
--- a/doc/development/i18n/externalization.md
+++ b/doc/development/i18n/externalization.md
@@ -263,7 +263,7 @@ n_("%{project_name}", "%d projects selected", count) % { project_name: 'GitLab'
 A namespace is a way to group translations that belong together. They provide context to our translators by adding a prefix followed by the bar symbol (`|`). For example:
 
 ```ruby
-_('Namespace|Translated string')
+'Namespace|Translated string'
 ```
 
 A namespace provide the following benefits:
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md
index c7e0aa13adb..441ef545141 100644
--- a/doc/topics/autodevops/customize.md
+++ b/doc/topics/autodevops/customize.md
@@ -310,6 +310,7 @@ applications.
 | `AUTO_DEVOPS_CHART_REPOSITORY_NAME`     | From GitLab 11.11, used to set the name of the Helm repository. Defaults to `gitlab`. |
 | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From GitLab 11.11, used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. |
 | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From GitLab 11.11, used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. |
+| `AUTO_DEVOPS_DEPLOY_DEBUG`              | From GitLab 13.1, if this variable is present, Helm will output debug logs. |
 | `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` | From GitLab 12.5, used in combination with [ModSecurity feature flag](../../user/clusters/applications.md#web-application-firewall-modsecurity) to toggle [ModSecurity's `SecRuleEngine`](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRuleEngine) behavior. Defaults to `DetectionOnly`. |
 | `BUILDPACK_URL`                         | Buildpack's full URL. Can point to either [a Git repository URL or a tarball URL](#custom-buildpacks). |
 | `CANARY_ENABLED`                        | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments-premium). |
diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md
index 87c1fe4007a..377b6cd393b 100644
--- a/doc/user/profile/personal_access_tokens.md
+++ b/doc/user/profile/personal_access_tokens.md
@@ -56,6 +56,58 @@ the following table.
 | `read_repository`  | [GitLab 10.7](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17894)  | Allows read-only access (pull) to the repository through `git clone`. |
 | `write_repository` | [GitLab 11.11](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26021) | Allows read-write access (pull, push) to the repository through `git clone`. Required for accessing Git repositories over HTTP when 2FA is enabled. |
 
+## Programmatically creating a personal access token
+
+You can programmatically create a predetermined personal access token for use in
+automation or tests. You will need sufficient access to run a
+[Rails console session](../../administration/troubleshooting/debug.md#starting-a-rails-console-session)
+for your GitLab instance.
+
+To create a token belonging to a user with username `automation-bot`, run the
+following in the Rails console (`sudo gitlab-rails console`):
+
+```ruby
+user = User.find_by_username('automation-bot')
+token = user.personal_access_tokens.create(scopes: [:read_user, :read_repository], name: 'Automation token')
+token.set_token('token-string-here123')
+token.save!
+```
+
+This can be shortened into a single-line shell command using the
+[GitLab Rails Runner](../../administration/troubleshooting/debug.md#using-the-rails-runner):
+
+```shell
+sudo gitlab-rails runner "token = User.find_by_username('automation-bot').personal_access_tokens.create(scopes: [:read_user, :read_repository], name: 'Automation token'); token.set_token('token-string-here123'); token.save!"
+```
+
+NOTE: **Note:**
+The token string must be 20 characters in length, or it will not be
+recognized as a personal access token.
+
+The list of valid scopes and what they do can be found
+[in the source code](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/auth.rb).
+
+## Programmatically revoking a personal access token
+
+You can programmatically revoke a personal access token. You will need
+sufficient access to run a [Rails console session](../../administration/troubleshooting/debug.md#starting-a-rails-console-session)
+for your GitLab instance.
+
+To revoke a known token `token-string-here123`, run the following in the Rails
+console (`sudo gitlab-rails console`):
+
+```ruby
+token = PersonalAccessToken.find_by_token('token-string-here123')
+token.revoke!
+```
+
+This can be shorted into a single-line shell command using the
+[GitLab Rails Runner](../../administration/troubleshooting/debug.md#using-the-rails-runner):
+
+```shell
+sudo gitlab-rails runner "PersonalAccessToken.find_by_token('token-string-here123').revoke!"
+```
+
 <!-- ## Troubleshooting
 
 Include any troubleshooting steps that you can foresee. If you know beforehand what issues
diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md
index fff3cf546b3..0b8807bb9b3 100644
--- a/doc/user/project/import/jira.md
+++ b/doc/user/project/import/jira.md
@@ -49,6 +49,7 @@ Importing large projects may take several minutes depending on the size of the i
 
 1. On the **{issues}** **Issues** page, click the **Import Issues** (**{import}**) button.
 1. Select **Import from Jira**.
+   This option is only visible if you have the [correct permissions](#permissions).
 
    ![Import issues from Jira button](img/jira/import_issues_from_jira_button_v12_10.png)
 
diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md
index fbecaf9ef72..ffb1f6a1407 100644
--- a/doc/user/project/service_desk.md
+++ b/doc/user/project/service_desk.md
@@ -173,12 +173,12 @@ As a result, a new Service Desk issue is created from this email in the `mygroup
 
 #### Enable custom email address
 
-This feature comes with the `service_desk_email` feature flag disabled by default.
+This feature comes with the `service_desk_custom_address` feature flag disabled by default.
 To turn on the feature, ask a GitLab administrator with Rails console access to run the following
 command:
 
 ```ruby
-Feature.enable(:service_desk_email)
+Feature.enable(:service_desk_custom_address)
 ```
 
 The configuration options are the same as for configuring