Add latest changes from gitlab-org/gitlab@master

1 files changed, 110 insertions, 97 deletions

diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index e98df17d944..78a0933ea15 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -1,6 +1,6 @@
 # Backing up and restoring GitLab
 
-![backup banner](backup_hrz.png)
+GitLab provides Rake tasks for backing up and restoring GitLab instances.
 
 An application data backup creates an archive file that contains the database,
 all repositories and all attachments.
@@ -14,31 +14,26 @@ from one server to another is through backup restore.
 In order to be able to backup and restore, you need two essential tools
 installed on your system.
 
-### Rsync
+- **Rsync**: If you installed GitLab:
+  - Using the Omnibus package, you're all set.
+  - From source, make sure `rsync` is installed. For example:
 
-If you installed GitLab:
+    ```shell
+    # Debian/Ubuntu
+    sudo apt-get install rsync
 
-- Using the Omnibus package, you're all set.
-- From source, make sure `rsync` is installed:
+    # RHEL/CentOS
+    sudo yum install rsync
+    ```
 
-  ```shell
-  # Debian/Ubuntu
-  sudo apt-get install rsync
+- **Tar**: Backup and restore tasks use `tar` under the hood to create and extract
+  archives. Ensure you have version 1.30 or above of `tar` available in your
+  system. To check the version, run:
 
-  # RHEL/CentOS
-  sudo yum install rsync
+  ```shell
+  tar --version
   ```
 
-### Tar
-
-Backup and restore tasks use `tar` under the hood to create and extract
-archives. Ensure you have version 1.30 or above of `tar` available in your
-system. To check the version, run:
-
-```shell
-tar --version
-```
-
 ## Backup timestamp
 
 NOTE: **Note:**
@@ -56,9 +51,9 @@ available.
 For example, if the backup name is `1493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar`,
 then the timestamp is `1493107454_2018_04_25_10.6.4-ce`.
 
-## Creating a backup of the GitLab system
+## Back up GitLab
 
-GitLab provides a simple command line interface to backup your whole instance.
+GitLab provides a simple command line interface to back up your whole instance.
 It backs up your:
 
 - Database
@@ -142,12 +137,12 @@ Deleting tmp directories...[DONE]
 Deleting old backups... [SKIPPING]
 ```
 
-## Storing configuration files
+### Storing configuration files
 
-A backup performed by the [raketask GitLab provides](#creating-a-backup-of-the-gitlab-system)
+The [backup Rake task](#back-up-gitlab) GitLab provides
 does **not** store your configuration files. The primary reason for this is that your
 database contains encrypted information for two-factor authentication, the CI/CD
-'secure variables', etc. Storing encrypted information along with its key in the
+'secure variables', and so on. Storing encrypted information along with its key in the
 same place defeats the purpose of using encryption in the first place.
 
 CAUTION: **Warning:**
@@ -182,11 +177,11 @@ If you use Omnibus GitLab, see some additional information
 In the unlikely event that the secrets file is lost, see the
 [troubleshooting section](#when-the-secrets-file-is-lost).
 
-## Backup options
+### Backup options
 
 The command line tool GitLab provides to backup your instance can take more options.
 
-### Backup strategy option
+#### Backup strategy option
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8728) in GitLab 8.17.
 
@@ -214,7 +209,11 @@ sudo gitlab-backup create STRATEGY=copy
 NOTE: **Note**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
-### Backup filename
+#### Backup filename
+
+CAUTION: **Warning:**
+If you use a custom backup filename, you will not be able to
+[limit the lifetime of the backups](#limit-backup-lifetime-for-local-files-prune-old-backups).
 
 By default a backup file is created according to the specification in [the Backup timestamp](#backup-timestamp) section above. You can however override the `[TIMESTAMP]` part of the filename by setting the `BACKUP` environment variable. For example:
 
@@ -227,7 +226,7 @@ For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 The resulting file will then be `dump_gitlab_backup.tar`. This is useful for systems that make use of rsync and incremental backups, and will result in considerably faster transfer speeds.
 
-### Rsyncable
+#### Rsyncable
 
 To make sure the generated archive is intelligently transferable by rsync, the `GZIP_RSYNCABLE=yes` option can be set. This will set the `--rsyncable` option to `gzip`. This is only useful in combination with setting [the Backup filename option](#backup-filename).
 
@@ -240,7 +239,7 @@ sudo gitlab-backup create BACKUP=dump GZIP_RSYNCABLE=yes
 NOTE: **Note**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
-### Excluding specific directories from the backup
+#### Excluding specific directories from the backup
 
 You can choose what should be exempt from the backup up by adding the environment variable `SKIP`.
 The available options are:
@@ -274,7 +273,7 @@ For installations from source:
 sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=production
 ```
 
-### Skipping tar creation
+#### Skipping tar creation
 
 The last part of creating a backup is generation of a `.tar` file containing
 all the parts. In some cases (for example, if the backup is picked up by other
@@ -299,19 +298,19 @@ For installations from source:
 sudo -u git -H bundle exec rake gitlab:backup:create SKIP=tar RAILS_ENV=production
 ```
 
-### Uploading backups to a remote (cloud) storage
+#### Uploading backups to a remote (cloud) storage
 
 Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates.
 It uses the [Fog library](http://fog.io/) to perform the upload.
 In the example below we use Amazon S3 for storage, but Fog also lets you use
 [other storage providers](http://fog.io/storage/). GitLab
 [imports cloud drivers](https://gitlab.com/gitlab-org/gitlab/blob/30f5b9a5b711b46f1065baf755e413ceced5646b/Gemfile#L88)
-for AWS, Google, OpenStack Swift, Rackspace and Aliyun as well. A local driver is
+for AWS, Google, OpenStack Swift, Rackspace, and Aliyun as well. A local driver is
 [also available](#uploading-to-locally-mounted-shares).
 
 [Read more about using object storage with GitLab](../administration/object_storage.md).
 
-#### Using Amazon S3
+##### Using Amazon S3
 
 For Omnibus GitLab packages:
 
@@ -329,9 +328,9 @@ For Omnibus GitLab packages:
    gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect
 
-#### Digital Ocean Spaces
+##### Digital Ocean Spaces
 
 This example can be used for a bucket in Amsterdam (AMS3).
 
@@ -348,7 +347,7 @@ This example can be used for a bucket in Amsterdam (AMS3).
    gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect
 
 NOTE: **Note:**
 If you see `400 Bad Request` by using Digital Ocean Spaces, the cause may be the
@@ -356,7 +355,7 @@ usage of backup encryption. Remove or comment the line that
 contains `gitlab_rails['backup_encryption']` since Digital Ocean Spaces
 doesn't support encryption.
 
-#### Other S3 Providers
+##### Other S3 Providers
 
 Not all S3 providers are fully-compatible with the Fog library. For example,
 if you see `411 Length Required` errors after attempting to upload, you may
@@ -393,7 +392,7 @@ For installations from source:
          # storage_class: 'STANDARD'
    ```
 
-1. [Restart GitLab] for the changes to take effect
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect
 
 If you are uploading your backups to S3 you will probably want to create a new
 IAM user with restricted access rights. To give the upload user access only for
@@ -446,7 +445,7 @@ with the name of your bucket:
 }
 ```
 
-#### Using Google Cloud Storage
+##### Using Google Cloud Storage
 
 If you want to use Google Cloud Storage to save backups, you'll have to create
 an access key from the Google console first:
@@ -472,7 +471,7 @@ For Omnibus GitLab packages:
    gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket'
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect
 
 For installations from source:
 
@@ -488,9 +487,9 @@ For installations from source:
          remote_directory: 'my.google.bucket'
    ```
 
-1. [Restart GitLab] for the changes to take effect
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect
 
-#### Specifying a custom directory for backups
+##### Specifying a custom directory for backups
 
 Note: This option only works for remote storage. If you want to group your backups
 you can pass a `DIRECTORY` environment variable:
@@ -503,9 +502,9 @@ sudo gitlab-backup create DIRECTORY=weekly
 NOTE: **Note**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
-### Uploading to locally mounted shares
+#### Uploading to locally mounted shares
 
-You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by
+You may also send backups to a mounted share (for example, `NFS`,`CIFS`, or `SMB`) by
 using the Fog [`Local`](https://github.com/fog/fog-local#usage) storage provider.
 The directory pointed to by the `local_root` key **must** be owned by the `git`
 user **when mounted** (mounting with the `uid=` of the `git` user for `CIFS` and
@@ -535,7 +534,7 @@ For Omnibus GitLab packages:
    gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups'
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect.
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
 
 For installations from source:
 
@@ -553,9 +552,9 @@ For installations from source:
        remote_directory: 'gitlab_backups'
    ```
 
-1. [Restart GitLab] for the changes to take effect.
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect.
 
-### Backup archive permissions
+#### Backup archive permissions
 
 The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`)
 will have owner/group `git`/`git` and 0600 permissions by default.
@@ -570,7 +569,7 @@ For Omnibus GitLab packages:
    gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect.
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
 
 For installations from source:
 
@@ -581,54 +580,83 @@ For installations from source:
      archive_permissions: 0644 # Makes the backup archives world-readable
    ```
 
-1. [Restart GitLab] for the changes to take effect.
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect.
 
-### Configuring cron to make daily backups
+#### Configuring cron to make daily backups
 
-NOTE: **Note:**
+CAUTION: **Warning:**
 The following cron jobs do not [backup your GitLab configuration files](#storing-configuration-files)
 or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079).
 
+You can schedule a cron job that backs up your repositories and GitLab metadata.
+
 For Omnibus GitLab packages:
 
-1. Edit `/etc/gitlab/gitlab.rb`:
+1. Edit the crontab for the `root` user:
 
-   ```ruby
-   ## Limit backup lifetime to 7 days - 604800 seconds
-   gitlab_rails['backup_keep_time'] = 604800
+   ```shell
+   sudo su -
+   crontab -e
    ```
 
-1. [Reconfigure GitLab] for the changes to take effect.
+1. There, add the following line to schedule the backup for everyday at 2 AM:
 
-Note that the `backup_keep_time` configuration option only manages local
-files. GitLab does not automatically prune old files stored in a third-party
-object storage (e.g., AWS S3) because the user may not have permission to list
-and delete files. We recommend that you configure the appropriate retention
-policy for your object storage. For example, you can configure [the S3 backup
-policy as described here](https://stackoverflow.com/questions/37553070/gitlab-omnibus-delete-backup-from-amazon-s3).
+   ```plaintext
+   0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1
+   ```
 
-To schedule a cron job that backs up your repositories and GitLab metadata, use the root user:
+   NOTE: **Note**
+   For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
-```shell
-sudo su -
-crontab -e
-```
+For installations from source:
 
-There, add the following line to schedule the backup for everyday at 2 AM:
+1. Edit the crontab for the `git` user:
 
-```plaintext
-0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1
-```
+   ```shell
+   sudo -u git crontab -e
+   ```
 
-NOTE: **Note**
-For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
+1. Add the following lines at the bottom:
+
+   ```plaintext
+   # Create a full backup of the GitLab repositories and SQL database every day at 2am
+   0 2 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1
+   ```
+
+NOTE: **Note:**
+The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors.
+This is recommended to reduce cron spam.
+
+### Limit backup lifetime for local files (prune old backups)
+
+CAUTION: **Warning:**
+This will not work if you have used a [custom filename](#backup-filename)
+for your backups.
+
+NOTE: **Note:**
+This configuration option only manages local files. GitLab does not automatically
+prune old files stored in a third-party [object storage](#uploading-backups-to-a-remote-cloud-storage)
+because the user may not have permission to list and delete files. It is
+recommended that you configure the appropriate retention policy for your object
+storage (for example, [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html)).
 
-You may also want to set a limited lifetime for backups to prevent regular
+You may want to set a limited lifetime for backups to prevent regular
 backups using all your disk space.
 
+For Omnibus GitLab packages:
+
+1. Edit `/etc/gitlab/gitlab.rb`:
+
+   ```ruby
+   ## Limit backup lifetime to 7 days - 604800 seconds
+   gitlab_rails['backup_keep_time'] = 604800
+   ```
+
+1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
+
 For installations from source:
 
-1. Edit `home/git/gitlab/config/gitlab.yml`:
+1. Edit `/home/git/gitlab/config/gitlab.yml`:
 
    ```yaml
    backup:
@@ -636,23 +664,9 @@ For installations from source:
      keep_time: 604800
    ```
 
-1. [Restart GitLab] for the changes to take effect.
-
-```shell
-sudo -u git crontab -e # Edit the crontab for the git user
-```
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect.
 
-Add the following lines at the bottom:
-
-```plaintext
-# Create a full backup of the GitLab repositories and SQL database every day at 4am
-0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1
-```
-
-The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors.
-This is recommended to reduce cron spam.
-
-## Restore
+## Restore GitLab
 
 GitLab provides a simple command line interface to restore your whole installation,
 and is flexible enough to fit your needs.
@@ -883,6 +897,8 @@ will have all your repositories, but not any other data.
 
 ## Troubleshooting
 
+The following are possible problems you might encounter with possible solutions.
+
 ### Restoring database backup using Omnibus packages outputs warnings
 
 If you are using backup restore procedures you might encounter the following warnings:
@@ -1060,12 +1076,9 @@ If you have changed the default filesystem location for the registry, you will
 want to run the `chown` against your custom location instead of
 `/var/opt/gitlab/gitlab-rails/shared/registry/docker`.
 
-[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
-[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source
-
 ### Backup fails to complete with Gzip error
 
-While running the backup, you may receive a gzip error:
+While running the backup, you may receive a Gzip error:
 
 ```shell
 sudo /opt/gitlab/bin/gitlab-backup create
@@ -1079,5 +1092,5 @@ Backup failed
 
 If this happens, check the following:
 
-1. Confirm there is sufficient disk space for the gzip operation.
+1. Confirm there is sufficient disk space for the Gzip operation.
 1. If NFS is being used, check if the mount option `timeout` is set. The default is `600`, and changing this to smaller values have resulted in this error.