Merge branch 'master' into sh-headless-chrome-support

5 files changed, 118 insertions, 6 deletions

diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md
index a7395e03d1c..425c924cdf2 100644
--- a/doc/administration/auth/ldap.md
+++ b/doc/administration/auth/ldap.md
@@ -96,7 +96,7 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server
   #
   #   Example: '/etc/ca.pem'
   #
-  ca_cert: ''
+  ca_file: ''
 
   # Specifies the SSL version for OpenSSL to use, if the OpenSSL default
   # is not appropriate.
@@ -259,9 +259,9 @@ group you can use the following syntax:
 (memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com)
 ```
 
-Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at 
+Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at
 https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx. Support for
-nested members in the user filter should not be confused with 
+nested members in the user filter should not be confused with
 [group sync nested groups support (EE only)](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#supported-ldap-group-types-attributes).
 
 Please note that GitLab does not support the custom filter syntax used by
diff --git a/doc/administration/img/failing_storage.png b/doc/administration/img/failing_storage.png
new file mode 100644
index 00000000000..82b393a58b2
--- /dev/null
+++ b/doc/administration/img/failing_storage.png
diff --git a/doc/administration/logs.md b/doc/administration/logs.md
index 4b8d5c5cc87..76e071dc673 100644
--- a/doc/administration/logs.md
+++ b/doc/administration/logs.md
@@ -9,6 +9,33 @@ documentation](http://docs.gitlab.com/ee/administration/audit_events.html)
 System log files are typically plain text in a standard log file format.
 This guide talks about how to read and use these system log files.
 
+## `production_json.log`
+
+This file lives in `/var/log/gitlab/gitlab-rails/production_json.log` for
+Omnibus GitLab packages or in `/home/git/gitlab/log/production_json.log` for
+installations from source. (When Gitlab is running in an environment
+other than production, the corresponding logfile is shown here.)
+
+It contains a structured log for Rails controller requests received from
+GitLab, thanks to [Lograge](https://github.com/roidrage/lograge/). Note that
+requests from the API [are not yet logged to this
+file](https://gitlab.com/gitlab-org/gitlab-ce/issues/36189).
+
+Each line contains a JSON line that can be ingested by Elasticsearch, Splunk, etc. For example:
+
+```json
+{"method":"GET","path":"/gitlab/gitlab-ce/issues/1234","format":"html","controller":"Projects::IssuesController","action":"show","status":200,"duration":229.03,"view":174.07,"db":13.24,"time":"2017-08-08T20:15:54.821Z","params":{"namespace_id":"gitlab","project_id":"gitlab-ce","id":"1234"},"remote_ip":"18.245.0.1","user_id":1,"username":"admin"}
+```
+
+In this example, you can see this was a GET request for a specific issue. Notice each line also contains performance data:
+
+1. `duration`: the total time taken to retrieve the request
+2. `view`: total time taken inside the Rails views
+3. `db`: total time to retrieve data from the database
+
+In addition, the log contains the IP address from which the request originated
+(`remote_ip`) as well as the user's ID (`user_id`), and username (`username`).
+
 ## `production.log`
 
 This file lives in `/var/log/gitlab/gitlab-rails/production.log` for
diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md
index 3b8c716eff5..a1bb3851951 100644
--- a/doc/administration/reply_by_email_postfix_setup.md
+++ b/doc/administration/reply_by_email_postfix_setup.md
@@ -177,6 +177,20 @@ Courier, which we will install later to add IMAP authentication, requires mailbo
     ```sh
     sudo apt-get install courier-imap
     ```
+    
+    And start `imapd`:
+    ```sh
+    imapd start
+    ```
+    
+1. The courier-authdaemon isn't started after installation. Without it, imap authentication will fail:
+    ```sh
+    sudo service courier-authdaemon start
+    ```
+    You can also configure courier-authdaemon to start on boot:
+    ```sh
+    sudo systemctl enable courier-authdaemon
+    ```
 
 ## Configure Postfix to receive email from the internet
 
diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md
index 55a45119525..624a908b3a3 100644
--- a/doc/administration/repository_storage_paths.md
+++ b/doc/administration/repository_storage_paths.md
@@ -60,7 +60,7 @@ respectively.
           path: /mnt/cephfs/repositories
     ```
 
-1. [Restart GitLab] for the changes to take effect.
+1. [Restart GitLab][restart-gitlab] for the changes to take effect.
 
 >**Note:**
 The [`gitlab_shell: repos_path` entry][repospath] in `gitlab.yml` will be
@@ -97,9 +97,80 @@ be stored via the **Application Settings** in the Admin area.
 Beginning with GitLab 8.13.4, multiple paths can be chosen. New projects will be
 randomly placed on one of the selected paths.
 
+## Handling failing repository storage
+
+> [Introduced][ce-11449] in GitLab 9.5.
+
+When GitLab detects access to the repositories storage fails repeatedly, it can
+gracefully prevent attempts to access the storage. This might be useful when
+the repositories are stored somewhere on the network.
+
+The configuration could look as follows:
+
+**For Omnibus installations**
+
+1. Edit `/etc/gitlab/gitlab.rb`:
+
+    ```ruby
+    git_data_dirs({
+      "default" => {
+        "path" => "/mnt/nfs-01/git-data",
+        "failure_count_threshold" => 10,
+        "failure_wait_time" => 30,
+        "failure_reset_time" => 1800,
+        "storage_timeout" => 5
+       }
+    })
+    ```
+
+1. Save the file and [reconfigure GitLab][reconfigure-gitlab] for the changes to take effect.
+
+---
+
+**For installations from source**
+
+1. Edit `config/gitlab.yml`:
+
+    ```yaml
+    repositories:
+      storages: # You must have at least a `default` storage path.
+        default:
+          path: /home/git/repositories/
+          failure_count_threshold: 10 # number of failures before stopping attempts
+          failure_wait_time: 30 # Seconds after last access failure before trying again
+          failure_reset_time: 1800 # Time in seconds to expire failures
+          storage_timeout: 5 # Time in seconds to wait before aborting a storage access attempt
+    ```
+
+1. Save the file and [restart GitLab][restart-gitlab] for the changes to take effect.
+
+
+**`failure_count_threshold`:** The number of failures of after which GitLab will
+completely prevent access to the storage. The number of failures can be reset in
+the admin interface: `https://gitlab.example.com/admin/health_check` or using the
+[api](../api/repository_storage_health.md) to allow access to the storage again.
+
+**`failure_wait_time`:** When access to a storage fails. GitLab will prevent
+access to the storage for the time specified here. This allows the filesystem to
+recover without.
+
+**`failure_reset_time`:** The time in seconds GitLab will keep failure
+information. When no failures occur during this time, information about the
+mount is reset.
+
+**`storage_timeout`:** The time in seconds GitLab will try to access storage.
+After this time a timeout error will be raised.
+
+When storage failures occur, this will be visible in the admin interface like this:
+
+![failing storage](img/failing_storage.png)
+
+To allow access to all storages, click the `Reset git storage health information` button.
+
 [ce-4578]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4578
-[restart gitlab]: restart_gitlab.md#installations-from-source
-[reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure
+[restart-gitlab]: restart_gitlab.md#installations-from-source
+[reconfigure-gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure
 [backups]: ../raketasks/backup_restore.md
 [raketask]: https://gitlab.com/gitlab-org/gitlab-ce/blob/033e5423a2594e08a7ebcd2379bd2331f4c39032/lib/backup/repository.rb#L54-56
 [repospath]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-9-stable/config/gitlab.yml.example#L457
+[ce-11449]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11449