Merge branch 'usage-ping-port' into 'master'

Usage ping port

Closes #27750

See merge request !10481

5 files changed, 190 insertions, 0 deletions

diff --git a/doc/README.md b/doc/README.md
index 7703c64e152..9e6a5b4ed44 100644
--- a/doc/README.md
+++ b/doc/README.md
@@ -69,6 +69,7 @@ All technical content published by GitLab lives in the documentation, including:
 - [Sidekiq Troubleshooting](administration/troubleshooting/sidekiq.md) Debug when Sidekiq appears hung and is not processing jobs.
 - [System hooks](system_hooks/system_hooks.md) Notifications when users, projects and keys are changed.
 - [Update](update/README.md) Update guides to upgrade your installation.
+- [User cohorts](user/admin_area/user_cohorts.md) View user activity over time.
 - [Web terminals](administration/integration/terminal.md) Provide terminal access to environments from within GitLab.
 - [Welcome message](customization/welcome_message.md) Add a custom welcome message to the sign-in page.
 
diff --git a/doc/api/users.md b/doc/api/users.md
index 2ada4d09c84..a79d31d19fa 100644
--- a/doc/api/users.md
+++ b/doc/api/users.md
@@ -72,6 +72,7 @@ GET /users
     "organization": "",
     "last_sign_in_at": "2012-06-01T11:41:01Z",
     "confirmed_at": "2012-05-23T09:05:22Z",
+    "last_activity_on": "2012-05-23",
     "color_scheme_id": 2,
     "projects_limit": 100,
     "current_sign_in_at": "2012-06-02T06:36:55Z",
@@ -104,6 +105,7 @@ GET /users
     "organization": "",
     "last_sign_in_at": null,
     "confirmed_at": "2012-05-30T16:53:06.148Z",
+    "last_activity_on": "2012-05-23",
     "color_scheme_id": 3,
     "projects_limit": 100,
     "current_sign_in_at": "2014-03-19T17:54:13Z",
@@ -196,6 +198,7 @@ Parameters:
   "organization": "",
   "last_sign_in_at": "2012-06-01T11:41:01Z",
   "confirmed_at": "2012-05-23T09:05:22Z",
+  "last_activity_on": "2012-05-23",
   "color_scheme_id": 2,
   "projects_limit": 100,
   "current_sign_in_at": "2012-06-02T06:36:55Z",
@@ -320,6 +323,7 @@ GET /user
   "organization": "",
   "last_sign_in_at": "2012-06-01T11:41:01Z",
   "confirmed_at": "2012-05-23T09:05:22Z",
+  "last_activity_on": "2012-05-23",
   "color_scheme_id": 2,
   "projects_limit": 100,
   "current_sign_in_at": "2012-06-02T06:36:55Z",
@@ -365,6 +369,7 @@ GET /user
   "organization": "",
   "last_sign_in_at": "2012-06-01T11:41:01Z",
   "confirmed_at": "2012-05-23T09:05:22Z",
+  "last_activity_on": "2012-05-23",
   "color_scheme_id": 2,
   "projects_limit": 100,
   "current_sign_in_at": "2012-06-02T06:36:55Z",
@@ -986,3 +991,55 @@ Parameters:
 | --------- | ---- | -------- | ----------- |
 | `user_id` | integer | yes | The ID of the user |
 | `impersonation_token_id` | integer | yes | The ID of the impersonation token |
+
+### Get user activities (admin only)
+
+>**Note:** This API endpoint is only available on 8.15 (EE) and 9.1 (CE) and above.
+
+Get the last activity date for all users, sorted from oldest to newest.
+
+The activities that update the timestamp are:
+
+  - Git HTTP/SSH activities (such as clone, push)
+  - User logging in into GitLab
+
+By default, it shows the activity for all users in the last 6 months, but this can be
+amended by using the `from` parameter.
+
+```
+GET /user/activities
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `from` | string | no | Date string in the format YEAR-MONTH-DAY, e.g. `2016-03-11`. Defaults to 6 months ago. |
+
+```bash
+curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/user/activities
+```
+
+Example response:
+
+```json
+[
+  {
+    "username": "user1",
+    "last_activity_on": "2015-12-14",
+    "last_activity_at": "2015-12-14"
+  },
+  {
+    "username": "user2",
+    "last_activity_on": "2015-12-15",
+    "last_activity_at": "2015-12-15"
+  },
+  {
+    "username": "user3",
+    "last_activity_on": "2015-12-16",
+    "last_activity_at": "2015-12-16"
+  }
+]
+```
+
+Please note that `last_activity_at` is deprecated, please use `last_activity_on`.
diff --git a/doc/user/admin_area/img/cohorts.png b/doc/user/admin_area/img/cohorts.png
new file mode 100644
index 00000000000..8bae7faff07
--- /dev/null
+++ b/doc/user/admin_area/img/cohorts.png
diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md
new file mode 100644
index 00000000000..c3f3179d99e
--- /dev/null
+++ b/doc/user/admin_area/settings/usage_statistics.md
@@ -0,0 +1,102 @@
+# Usage statistics
+
+GitLab Inc. will periodically collect information about your instance in order
+to perform various actions.
+
+All statistics are opt-in and you can always disable them from the admin panel.
+
+## Version check
+
+GitLab can inform you when an update is available and the importance of it.
+
+No information other than the GitLab version and the instance's domain name
+are collected.
+
+In the **Overview** tab you can see if your GitLab version is up to date. There
+are three cases: 1) you are up to date (green), 2) there is an update available
+(yellow) and 3) your version is vulnerable and a security fix is released (red).
+
+In any case, you will see a message informing you of the state and the
+importance of the update.
+
+If enabled, the version status will also be shown in the help page (`/help`)
+for all signed in users.
+
+## Usage ping
+
+> [Introduced][ee-557] in GitLab Enterprise Edition 8.10. More statistics
+[were added][ee-735] in GitLab Enterprise Edition
+8.12. [Moved to GitLab Community Edition][ce-23361] in 9.1.
+
+GitLab Inc. can collect non-sensitive information about how GitLab users
+use their GitLab instance upon the activation of a ping feature
+located in the admin panel (`/admin/application_settings`).
+
+You can see the **exact** JSON payload that your instance sends to GitLab
+in the "Usage statistics" section of the admin panel.
+
+Nothing qualitative is collected. Only quantitative. That means no project
+names, author names, comment bodies, names of labels, etc.
+
+The usage ping is sent in order for GitLab Inc. to have a better understanding
+of how our users use our product, and to be more data-driven when creating or
+changing features.
+
+The total number of the following is sent back to GitLab Inc.:
+
+- Comments
+- Groups
+- Users
+- Projects
+- Issues
+- Labels
+- CI builds
+- Snippets
+- Milestones
+- Todos
+- Pushes
+- Merge requests
+- Environments
+- Triggers
+- Deploy keys
+- Pages
+- Project Services
+- Projects using the Prometheus service
+- Issue Boards
+- CI Runners
+- Deployments
+- Geo Nodes
+- LDAP Groups
+- LDAP Keys
+- LDAP Users
+- LFS objects
+- Protected branches
+- Releases
+- Remote mirrors
+- Uploads
+- Web hooks
+
+Also, we track if you've installed Mattermost with GitLab.
+For example: `"mattermost_enabled":true"`.
+
+More data will be added over time. The goal of this ping is to be as light as
+possible, so it won't have any performance impact on your installation when
+the calculation is made.
+
+### Deactivate the usage ping
+
+By default, usage ping is opt-out. If you want to deactivate this feature, go to
+the Settings page of your administration panel and uncheck the Usage ping
+checkbox.
+
+## Privacy policy
+
+GitLab Inc. does **not** collect any sensitive information, like project names
+or the content of the comments. GitLab Inc. does not disclose or otherwise make
+available any of the data collected on a customer specific basis.
+
+Read more about this in the [Privacy policy](https://about.gitlab.com/privacy).
+
+[ee-557]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/557
+[ee-735]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/735
+[ce-23361]: https://gitlab.com/gitlab-org/gitlab-ce/issues/23361
diff --git a/doc/user/admin_area/user_cohorts.md b/doc/user/admin_area/user_cohorts.md
new file mode 100644
index 00000000000..1671487bc8c
--- /dev/null
+++ b/doc/user/admin_area/user_cohorts.md
@@ -0,0 +1,30 @@
+# Cohorts
+
+> **Notes:**
+- [Introduced][ce-23361] in GitLab 9.1.
+
+As a benefit of having the [usage ping active](settings/usage_statistics.md),
+GitLab lets you analyze the users' activities of your GitLab installation.
+Under `/admin/cohorts`, when the usage ping is active, GitLab will show the
+monthly cohorts of new users and their activities over time.
+
+How do we read the user cohorts table? Let's take an example with the following
+user cohorts.
+
+![User cohort example](img/cohorts.png)
+
+For the cohort of June 2016, 163 users have been created on this server. One
+month later, in July 2016, 155 users (or 95% of the June cohort) are still
+active. Two months later, 139 users (or 85%) are still active. 9 months later,
+we can see that only 6% of this cohort are still active.
+
+How do we measure the activity of users? GitLab considers a user active if:
+* the user signs in
+* the user has Git activity (whether push or pull).
+
+### Setup
+
+1. Activate the usage ping as defined [in the documentation](settings/usage_statistics.md)
+2. Go to `/admin/cohorts` to see the user cohorts of the server
+
+[ce-23361]: https://gitlab.com/gitlab-org/gitlab-ce/issues/23361