diff options
Diffstat (limited to 'doc/administration')
| -rw-r--r-- | doc/administration/auth/crowd.md | 12 | ||||
| -rw-r--r-- | doc/administration/auth/img/crowd_application_authorisation.png | bin | 0 -> 126994 bytes | |||
| -rw-r--r-- | doc/administration/auth/okta.md | 2 | ||||
| -rw-r--r-- | doc/administration/container_registry.md | 2 | ||||
| -rw-r--r-- | doc/administration/high_availability/nfs.md | 4 | ||||
| -rw-r--r-- | doc/administration/high_availability/redis.md | 2 | ||||
| -rw-r--r-- | doc/administration/operations/fast_ssh_key_lookup.md | 176 | ||||
| -rw-r--r-- | doc/administration/operations/img/write_to_authorized_keys_setting.png | bin | 0 -> 94218 bytes | |||
| -rw-r--r-- | doc/administration/operations/index.md | 3 | ||||
| -rw-r--r-- | doc/administration/operations/speed_up_ssh.md | 1 | ||||
| -rw-r--r-- | doc/administration/raketasks/check.md | 46 |
11 files changed, 241 insertions, 7 deletions
diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index 2c289c67a6d..6db74958d5a 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -66,3 +66,15 @@ On the sign in page there should now be a Crowd tab in the sign in form. [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure [restart]: ../restart_gitlab.md#installations-from-source + +## Troubleshooting + +If you see an error message like the one below when you sign in after Crowd authentication is configured, you may want to consult the Crowd administrator for the Crowd log file to know the exact cause: + +``` +could not authorize you from Crowd because invalid credentials +``` + +Please make sure the Crowd users who need to login to GitLab are authorized to [the application](#configure-a-new-crowd-application) in the step of **Authorisation**. This could be verified by try "Authentication test" for Crowd as of 2.11. + +
\ No newline at end of file diff --git a/doc/administration/auth/img/crowd_application_authorisation.png b/doc/administration/auth/img/crowd_application_authorisation.png Binary files differnew file mode 100644 index 00000000000..70339891b34 --- /dev/null +++ b/doc/administration/auth/img/crowd_application_authorisation.png diff --git a/doc/administration/auth/okta.md b/doc/administration/auth/okta.md index cb42b7743c5..664657650d4 100644 --- a/doc/administration/auth/okta.md +++ b/doc/administration/auth/okta.md @@ -144,7 +144,7 @@ Now that the Okta app is configured, it's time to enable it in GitLab. 1. [Reconfigure][reconf] or [restart] GitLab for Omnibus and installations from source respectively for the changes to take effect. -You might want to try this out on a incognito browser window. +You might want to try this out on an incognito browser window. ## Configuring groups diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index 57e54815b68..2441ff85783 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -483,7 +483,7 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. 1. A certificate keypair is required for GitLab and the Container Registry to communicate securely. By default omnibus-gitlab will generate one keypair, which is saved to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key`. - When using an non-bundled Container Registry, you will need to supply a + When using a non-bundled Container Registry, you will need to supply a custom certificate key. To do that, add the following to `/etc/gitlab/gitlab.rb` diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index e09ccaba08c..d8928a7fe4c 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -32,7 +32,9 @@ options: ## AWS Elastic File System -GitLab does not recommend using AWS Elastic File System (EFS). +GitLab strongly recommends against using AWS Elastic File System (EFS). +Our support team will not be able to assist on performance issues related to +file system access. Customers and users have reported that AWS EFS does not perform well for GitLab's use-case. There are several issues that can cause problems. For these reasons diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 0e92f7c5a34..fd2677996b1 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -154,7 +154,7 @@ who will take all the decisions to restore the service availability by: - Reconfigure the old **Master** and demote to **Slave** when it comes back online You must have at least `3` Redis Sentinel servers, and they need to -be each in a independent machine (that are believed to fail independently), +be each in an independent machine (that are believed to fail independently), ideally in different geographical areas. You can configure them in the same machines where you've configured the other diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md new file mode 100644 index 00000000000..9d1589d84aa --- /dev/null +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -0,0 +1,176 @@ +# Fast lookup of authorized SSH keys in the database + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/1631) in +> [GitLab Enterprise Edition Standard](https://about.gitlab.com/gitlab-ee) 9.3. +> +> [Available in](https://gitlab.com/gitlab-org/gitlab-ee/issues/3953) GitLab +> Community Edition 10.4. + +Regular SSH operations become slow as the number of users grows because OpenSSH +searches for a key to authorize a user via a linear search. In the worst case, +such as when the user is not authorized to access GitLab, OpenSSH will scan the +entire file to search for a key. This can take significant time and disk I/O, +which will delay users attempting to push or pull to a repository. Making +matters worse, if users add or remove keys frequently, the operating system may +not be able to cache the `authorized_keys` file, which causes the disk to be +accessed repeatedly. + +GitLab Shell solves this by providing a way to authorize SSH users via a fast, +indexed lookup in the GitLab database. This page describes how to enable the fast +lookup of authorized SSH keys. + +> **Warning:** OpenSSH version 6.9+ is required because +`AuthorizedKeysCommand` must be able to accept a fingerprint. These +instructions will break installations using older versions of OpenSSH, such as +those included with CentOS 6 as of September 2017. If you want to use this +feature for CentOS 6, follow [the instructions on how to build and install a custom OpenSSH package](#compiling-a-custom-version-of-openssh-for-centos-6) before continuing. + +## Setting up fast lookup via GitLab Shell + +GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup +to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to +check whether the user is authorized to access GitLab. + +Add the following to your `sshd_config` file. This is usuaully located at +`/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using +Omnibus Docker: + +``` +AuthorizedKeysCommand /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k +AuthorizedKeysCommandUser git +``` + +Reload OpenSSH: + +```bash +# Debian or Ubuntu installations +sudo service ssh reload + +# CentOS installations +sudo service sshd reload +``` + +Confirm that SSH is working by removing your user's SSH key in the UI, adding a +new one, and attempting to pull a repo. + +> **Warning:** Do not disable writes until SSH is confirmed to be working +perfectly, because the file will quickly become out-of-date. + +In the case of lookup failures (which are not uncommon), the `authorized_keys` +file will still be scanned. So git SSH performance will still be slow for many +users as long as a large file exists. + +You can disable any more writes to the `authorized_keys` file by unchecking +`Write to "authorized_keys" file` in the Application Settings of your GitLab +installation. + + + +Again, confirm that SSH is working by removing your user's SSH key in the UI, +adding a new one, and attempting to pull a repo. + +Then you can backup and delete your `authorized_keys` file for best performance. + +## How to go back to using the `authorized_keys` file + +This is a brief overview. Please refer to the above instructions for more context. + +1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file) +1. Enable writes to the `authorized_keys` file in Application Settings +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. +1. Reload sshd: `sudo service sshd reload` +1. Remove the `/opt/gitlab-shell/authorized_keys` file + +## Compiling a custom version of OpenSSH for CentOS 6 + +Building a custom version of OpenSSH is not necessary for Ubuntu 16.04 users, +since Ubuntu 16.04 ships with OpenSSH 7.2. + +It is also unnecessary for CentOS 7.4 users, as that version ships with +OpenSSH 7.4. If you are using CentOS 7.0 - 7.3, we strongly recommend that you +upgrade to CentOS 7.4 instead of following this procedure. This should be as +simple as running `yum update`. + +CentOS 6 users must build their own OpenSSH package to enable SSH lookups via +the database. The following instructions can be used to build OpenSSH 7.5: + +1. First, download the package and install the required packages: + + ``` + sudo su - + cd /tmp + curl --remote-name https://mirrors.evowise.com/pub/OpenBSD/OpenSSH/portable/openssh-7.5p1.tar.gz + tar xzvf openssh-7.5p1.tar.gz + yum install rpm-build gcc make wget openssl-devel krb5-devel pam-devel libX11-devel xmkmf libXt-devel + ``` + +3. Prepare the build by copying files to the right place: + + ``` + mkdir -p /root/rpmbuild/{SOURCES,SPECS} + cp ./openssh-7.5p1/contrib/redhat/openssh.spec /root/rpmbuild/SPECS/ + cp openssh-7.5p1.tar.gz /root/rpmbuild/SOURCES/ + cd /root/rpmbuild/SPECS + ``` + +3. Next, set the spec settings properly: + + ``` + sed -i -e "s/%define no_gnome_askpass 0/%define no_gnome_askpass 1/g" openssh.spec + sed -i -e "s/%define no_x11_askpass 0/%define no_x11_askpass 1/g" openssh.spec + sed -i -e "s/BuildPreReq/BuildRequires/g" openssh.spec + ``` + +3. Build the RPMs: + + ``` + rpmbuild -bb openssh.spec + ``` + +4. Ensure the RPMs were built: + + ``` + ls -al /root/rpmbuild/RPMS/x86_64/ + ``` + + You should see something as the following: + + ``` + total 1324 + drwxr-xr-x. 2 root root 4096 Jun 20 19:37 . + drwxr-xr-x. 3 root root 19 Jun 20 19:37 .. + -rw-r--r--. 1 root root 470828 Jun 20 19:37 openssh-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 490716 Jun 20 19:37 openssh-clients-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 17020 Jun 20 19:37 openssh-debuginfo-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 367516 Jun 20 19:37 openssh-server-7.5p1-1.x86_64.rpm + ``` + +5. Install the packages. OpenSSH packages will replace `/etc/pam.d/sshd` + with its own version, which may prevent users from logging in, so be sure + that the file is backed up and restored after installation: + + ``` + timestamp=$(date +%s) + cp /etc/pam.d/sshd pam-ssh-conf-$timestamp + rpm -Uvh /root/rpmbuild/RPMS/x86_64/*.rpm + yes | cp pam-ssh-conf-$timestamp /etc/pam.d/sshd + ``` + +6. Verify the installed version. In another window, attempt to login to the server: + + ``` + ssh -v <your-centos-machine> + ``` + + You should see a line that reads: "debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5" + + If not, you may need to restart sshd (e.g. `systemctl restart sshd.service`). + +7. *IMPORTANT!* Open a new SSH session to your server before exiting to make + sure everything is working! If you need to downgrade, simple install the + older package: + + ``` + # Only run this if you run into a problem logging in + yum downgrade openssh-server openssh openssh-clients + ``` diff --git a/doc/administration/operations/img/write_to_authorized_keys_setting.png b/doc/administration/operations/img/write_to_authorized_keys_setting.png Binary files differnew file mode 100644 index 00000000000..232765f1917 --- /dev/null +++ b/doc/administration/operations/img/write_to_authorized_keys_setting.png diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index 320d71a9527..5655b7efec6 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -13,4 +13,5 @@ by GitLab to another file system or another server. that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. -- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer.
\ No newline at end of file +- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. +- [Speed up SSH operations](fast_ssh_key_lookup.md): Authorize SSH users via a fast, indexed lookup to the GitLab database. diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md new file mode 100644 index 00000000000..89265b3018b --- /dev/null +++ b/doc/administration/operations/speed_up_ssh.md @@ -0,0 +1 @@ +This document was moved to [another location](fast_ssh_key_lookup.md). diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index c8b5434c068..d1ed152b58c 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -28,19 +28,25 @@ exactly which repositories are causing the trouble. ### Check all GitLab repositories +>**Note:** +> +> - `gitlab:repo:check` has been deprecated in favor of `gitlab:git:fsck` +> - [Deprecated][ce-15931] in GitLab 10.4. +> - `gitlab:repo:check` will be removed in the future. [Removal issue][ce-41699] + This task loops through all repositories on the GitLab server and runs the 3 integrity checks described previously. **Omnibus Installation** ``` -sudo gitlab-rake gitlab:repo:check +sudo gitlab-rake gitlab:git:fsck ``` **Source Installation** ```bash -sudo -u git -H bundle exec rake gitlab:repo:check RAILS_ENV=production +sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production ``` ### Check repositories for a specific user @@ -70,9 +76,45 @@ Example output:  +## Uploaded Files Integrity + +The uploads check Rake task will loop through all uploads in the database +and run two checks to determine the integrity of each file: + +1. Check if the file exist on the file system. +1. Check if the checksum of the file on the file system matches the checksum in the database. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:uploads:check +``` + +**Source Installation** + +```bash +sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production +``` + +This task also accepts some environment variables which you can use to override +certain values: + +Variable | Type | Description +-------- | ---- | ----------- +`BATCH` | integer | Specifies the size of the batch. Defaults to 200. +`ID_FROM` | integer | Specifies the ID to start from, inclusive of the value. +`ID_TO` | integer | Specifies the ID value to end at, inclusive of the value. + +```bash +sudo gitlab-rake gitlab:uploads:check BATCH=100 ID_FROM=50 ID_TO=250 +``` + ## LDAP Check The LDAP check Rake task will test the bind_dn and password credentials (if configured) and will list a sample of LDAP users. This task is also executed as part of the `gitlab:check` task, but can run independently. See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. + +[ce-15931]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15931 +[ce-41699]: https://gitlab.com/gitlab-org/gitlab-ce/issues/41699 |
