From db66b12b29ab2e210644ae5584402fe58c41bfa3 Mon Sep 17 00:00:00 2001 From: Patricio Cano Date: Fri, 15 Jan 2016 17:02:25 -0500 Subject: Added advanced SAML troubleshooting --- doc/integration/saml.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) (limited to 'doc/integration') diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 1632e42f701..8841dbdb7c6 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -78,6 +78,18 @@ On the sign in page there should now be a SAML button below the regular sign in ## Troubleshooting -If you see a "500 error" in GitLab when you are redirected back from the SAML sign in page, this likely indicates that GitLab could not get the email address for the SAML user. +If you see a "500 error" in GitLab when you are redirected back from the SAML sign in page, +this likely indicates that GitLab could not get the email address for the SAML user. -Make sure the IdP provides a claim containing the user's email address, using claim name 'email' or 'mail'. The email will be used to automatically generate the GitLab username. \ No newline at end of file +Make sure the IdP provides a claim containing the user's email address, using claim name +'email' or 'mail'. The email will be used to automatically generate the GitLab username. + +If after signing in into your SAML server you are redirected back to the sign in page and +no error is displayed, check your `production.log` file. It will most likely contain the +message `Can't verify CSRF token authenticity`. This means that there is an error during +the SAML request, but this error never reaches GitLab due to the CSRF check. + +To bypass this you can add `skip_before_action :verify_authenticity_token` to the +`omniauth_callbacks_controller.rb` file. This will allow the error to hit GitLab, +where it can then be seen in the usual logs, or as a flash message in the login +screen. \ No newline at end of file -- cgit v1.2.1 From d4c05766c7a058cbf62186723216d754eebad92e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 19 Jan 2016 22:18:51 +0100 Subject: Refactor GitLab Metrics docs [ci skip] --- doc/integration/README.md | 1 + doc/integration/metrics/gitlab_configuration.md | 39 +++++ .../img/metrics_gitlab_configuration_settings.png | Bin 0 -> 45148 bytes doc/integration/metrics/influxdb_configuration.md | 192 +++++++++++++++++++++ doc/integration/metrics/influxdb_schema.md | 87 ++++++++++ doc/integration/metrics/introduction.md | 64 +++++++ 6 files changed, 383 insertions(+) create mode 100644 doc/integration/metrics/gitlab_configuration.md create mode 100644 doc/integration/metrics/img/metrics_gitlab_configuration_settings.png create mode 100644 doc/integration/metrics/influxdb_configuration.md create mode 100644 doc/integration/metrics/influxdb_schema.md create mode 100644 doc/integration/metrics/introduction.md (limited to 'doc/integration') diff --git a/doc/integration/README.md b/doc/integration/README.md index 5edac746c7b..5ba0d0fb245 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -15,6 +15,7 @@ See the documentation below for details on how to configure these services. - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users +- [GitLab Metrics](metrics/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. diff --git a/doc/integration/metrics/gitlab_configuration.md b/doc/integration/metrics/gitlab_configuration.md new file mode 100644 index 00000000000..019db1ad02c --- /dev/null +++ b/doc/integration/metrics/gitlab_configuration.md @@ -0,0 +1,39 @@ +# GitLab Configuration + +GitLab Metrics is disabled by default. To enable it and change any of its +settings, navigate to the Admin area in **Settings > Metrics** +(`/admin/application_settings`). + +The minimum required settings you need to set are the InfluxDB host and port. +Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the +changes. + +--- + +![GitLab Metrics Admin Settings](img/metrics_gitlab_configuration_settings.png) + +--- + +Finally, a restart of all GitLab processes is required for the changes to take +effect: + +```bash +# For Omnibus installations +sudo gitlab-ctl restart + +# For installations from source +sudo service gitlab restart +``` + +## Pending Migrations + +When any migrations are pending, the metrics are disabled until the migrations +have been performed. + +--- + +Read more on: + +- [Introduction to GitLab Metrics](introduction.md) +- [InfluxDB Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) diff --git a/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png b/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png new file mode 100644 index 00000000000..14d82b6ac98 Binary files /dev/null and b/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png differ diff --git a/doc/integration/metrics/influxdb_configuration.md b/doc/integration/metrics/influxdb_configuration.md new file mode 100644 index 00000000000..13227f1e889 --- /dev/null +++ b/doc/integration/metrics/influxdb_configuration.md @@ -0,0 +1,192 @@ +# InfluxDB Configuration + +The default settings provided by [InfluxDB] are not sufficient for a high traffic +GitLab environment. The settings discussed in this document are based on the +settings GitLab uses for GitLab.com, depending on your own needs you may need to +further adjust them. + +If you are intending to run InfluxDB on the same server as GitLab, make sure +you have plenty of RAM since InfluxDB can use quite a bit depending on traffic. + +Unless you are going with a budget setup, it's advised to run it separately. + +## Requirements + +- InfluxDB 0.9.5 or newer +- A fairly modern version of Linux +- At least 4GB of RAM +- At least 10GB of storage for InfluxDB data + +Note that the RAM and storage requirements can differ greatly depending on the +amount of data received/stored. To limit the amount of stored data users can +look into [InfluxDB Retention Policies][influxdb-retention]. + +## Installation + +Installing InfluxDB is out of the scope of this document. Please refer to the +[InfluxDB documentation]. + +## InfluxDB Server Settings + +Since InfluxDB has many settings that users may wish to customize themselves +(e.g. what port to run InfluxDB on), we'll only cover the essentials. + +The configuration file in question is usually located at +`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file, +InfluxDB needs to be restarted. + +### Storage Engine + +InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new +storage engine is available, called [TSM Tree]. All users **must** use the new +`tsm1` storage engine as this [will be the default engine][tsm1-commit] in +upcoming InfluxDB releases. + +Make sure you have the following in your configuration file: + +``` +[data] + dir = "/var/lib/influxdb/data" + engine = "tsm1" +``` + +### Admin Panel + +Production environments should have the InfluxDB admin panel **disabled**. This +feature can be disabled by adding the following to your InfluxDB configuration +file: + +``` +[admin] + enabled = false +``` + +### HTTP + +HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana, +thus it should be enabled. When enabling make sure to _also_ enable +authentication: + +``` +[http] + enabled = true + auth-enabled = true +``` + +_**Note:** Before you enable authentication, you might want to [create an +admin user](#create-a-new-admin-user)._ + +### UDP + +GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling +UDP can be done using the following settings: + +``` +[[udp]] + enabled = true + bind-address = ":8089" + database = "gitlab" + batch-size = 1000 + batch-pending = 5 + batch-timeout = "1s" + read-buffer = 209715200 +``` + +This does the following: + +1. Enable UDP and bind it to port 8089 for all addresses. +2. Store any data received in the "gitlab" database. +3. Define a batch of points to be 1000 points in size and allow a maximum of + 5 batches _or_ flush them automatically after 1 second. +4. Define a UDP read buffer size of 200 MB. + +One of the most important settings here is the UDP read buffer size as if this +value is set too low, packets will be dropped. You must also make sure the OS +buffer size is set to the same value, the default value is almost never enough. + +To set the OS buffer size to 200 MB, on Linux you can run the following command: + +```bash +sysctl -w net.core.rmem_max=209715200 +``` + +To make this permanent, add the following to `/etc/sysctl.conf` and restart the +server: + +```bash +net.core.rmem_max=209715200 +``` + +It is **very important** to make sure the buffer sizes are large enough to +handle all data sent to InfluxDB as otherwise you _will_ lose data. The above +buffer sizes are based on the traffic for GitLab.com. Depending on the amount of +traffic, users may be able to use a smaller buffer size, but we highly recommend +using _at least_ 100 MB. + +When enabling UDP, users should take care to not expose the port to the public, +as doing so will allow anybody to write data into your InfluxDB database (as +[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either +whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only +allowing traffic from members of said VLAN. + +## Create a new admin user + +If you want to [enable authentication](#http), you might want to [create an +admin user][influx-admin]: + +``` +influx -execute "CREATE USER thedude WITH PASSWORD '1234' WITH ALL PRIVILEGES" +``` + +## Create the `gitlab` database + +Once you get InfluxDB up and running, you need to create a database for GitLab. +Make sure you have changed the [storage engine](#storage-engine) to `tsm1` +before creating a database. + +_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled +[HTTP authentication](#http), remember to append the username (`-username thedude`) +and password (`-password 1234`) to the commands below._ + +Run the following command to create a database named `gitlab`: + +```bash +influx -execute 'CREATE DATABASE gitlab' +``` + +The name **must** be `gitlab`, do not use any other name. + +Next, make sure that the database was successfully created: + +```bash +influx -execute 'SHOW DATABASES' +``` + +The output should be similar to: + +``` +name: databases +--------------- +name +_internal +gitlab +``` + +That's it! Now your GitLab instance should send data to InfluxDB. + +--- + +Read more on: + +- [Introduction to GitLab Metrics](introduction.md) +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) + +[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management +[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/ +[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/ +[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ +[influxdb]: https://influxdata.com/time-series-platform/influxdb/ +[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/ +[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d +[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user diff --git a/doc/integration/metrics/influxdb_schema.md b/doc/integration/metrics/influxdb_schema.md new file mode 100644 index 00000000000..a9ef0b446c8 --- /dev/null +++ b/doc/integration/metrics/influxdb_schema.md @@ -0,0 +1,87 @@ +# InfluxDB Schema + +The following measurements are currently stored in InfluxDB: + +- `PROCESS_file_descriptors` +- `PROCESS_gc_statistics` +- `PROCESS_memory_usage` +- `PROCESS_method_calls` +- `PROCESS_object_counts` +- `PROCESS_transactions` +- `PROCESS_views` + +Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the +process type. In all series, any form of duration is stored in milliseconds. + +## PROCESS_file_descriptors + +This measurement contains the number of open file descriptors over time. The +value field `value` contains the number of descriptors. + +## PROCESS_gc_statistics + +This measurement contains Ruby garbage collection statistics such as the amount +of minor/major GC runs (relative to the last sampling interval), the time spent +in garbage collection cycles, and all fields/values returned by `GC.stat`. + +## PROCESS_memory_usage + +This measurement contains the process' memory usage (in bytes) over time. The +value field `value` contains the number of bytes. + +## PROCESS_method_calls + +This measurement contains the methods called during a transaction along with +their duration, and a name of the transaction action that invoked the method (if +available). The method call duration is stored in the value field `duration`, +while the method name is stored in the tag `method`. The tag `action` contains +the full name of the transaction action. Both the `method` and `action` fields +are in the following format: + +``` +ClassName#method_name +``` + +For example, a method called by the `show` method in the `UsersController` class +would have `action` set to `UsersController#show`. + +## PROCESS_object_counts + +This measurement is used to store retained Ruby objects (per class) and the +amount of retained objects. The number of objects is stored in the `count` value +field while the class name is stored in the `type` tag. + +## PROCESS_transactions + +This measurement is used to store basic transaction details such as the time it +took to complete a transaction, how much time was spent in SQL queries, etc. The +following value fields are available: + +| Value | Description | +| ----- | ----------- | +| `duration` | The total duration of the transaction | +| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers | +| `method_duration` | The total time spent in method calls | +| `sql_duration` | The total time spent in SQL queries | +| `view_duration` | The total time spent in views | + +## PROCESS_views + +This measurement is used to store view rendering timings for a transaction. The +following value fields are available: + +| Value | Description | +| ----- | ----------- | +| `duration` | The rendering time of the view | +| `view` | The path of the view, relative to the application's root directory | + +The `action` tag contains the action name of the transaction that rendered the +view. + +--- + +Read more on: + +- [Introduction to GitLab Metrics](introduction.md) +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Configuration](influxdb_configuration.md) diff --git a/doc/integration/metrics/introduction.md b/doc/integration/metrics/introduction.md new file mode 100644 index 00000000000..bf388aa2bba --- /dev/null +++ b/doc/integration/metrics/introduction.md @@ -0,0 +1,64 @@ +# GitLab Metrics + +GitLab comes with its own application performance measuring system as of GitLab +8.4, simply called "GitLab Metrics". GitLab Metrics is available in both the +Community and Enterprise editions. + +Apart from this introduction, you are advised to read through the following +documents in order to understand and properly configure GitLab Metrics: + +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) + +## Introduction to GitLab Metrics + +GitLab Metrics makes it possible to measure a wide variety of statistics +including (but not limited to): + +- The time it took to complete a transaction (a web request or Sidekiq job). +- The time spent in running SQL queries and rendering HAML views. +- The time spent executing (instrumented) Ruby methods. +- Ruby object allocations, and retained objects in particular. +- System statistics such as the process' memory usage and open file descriptors. +- Ruby garbage collection statistics. + +Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored +data can be visualized using [Grafana][grafana] or any other application that +supports reading data from InfluxDB. Alternatively data can be queried using the +InfluxDB CLI. + +## Metric Types + +Two types of metrics are collected: + +1. Transaction specific metrics. +1. Sampled metrics, collected at a certain interval in a separate thread. + +### Transaction Metrics + +Transaction metrics are metrics that can be associated with a single +transaction. This includes statistics such as the transaction duration, timings +of any executed SQL queries, time spent rendering HAML views, etc. These metrics +are collected for every Rack request and Sidekiq job processed. + +### Sampled Metrics + +Sampled metrics are metrics that can't be associated with a single transaction. +Examples include garbage collection statistics and retained Ruby objects. These +metrics are collected at a regular interval. This interval is made up out of two +parts: + +1. A user defined interval. +1. A randomly generated offset added on top of the interval, the same offset + can't be used twice in a row. + +The actual interval can be anywhere between a half of the defined interval and a +half above the interval. For example, for a user defined interval of 15 seconds +the actual interval can be anywhere between 7.5 and 22.5. The interval is +re-generated for every sampling run instead of being generated once and re-used +for the duration of the process' lifetime. + +[influxdb]: https://influxdata.com/time-series-platform/influxdb/ +[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ +[grafana]: http://grafana.org/ -- cgit v1.2.1 From 6f32459f2c0ebc0aa10c728e5562375db9cf4c52 Mon Sep 17 00:00:00 2001 From: Yorick Peterse Date: Wed, 20 Jan 2016 12:05:03 +0100 Subject: Renamed "Metrics" to "Performance Monitoring" --- doc/integration/README.md | 2 +- doc/integration/metrics/gitlab_configuration.md | 6 +++--- doc/integration/metrics/influxdb_configuration.md | 2 +- doc/integration/metrics/influxdb_schema.md | 2 +- doc/integration/metrics/introduction.md | 10 +++++----- 5 files changed, 11 insertions(+), 11 deletions(-) (limited to 'doc/integration') diff --git a/doc/integration/README.md b/doc/integration/README.md index 5ba0d0fb245..297e4f6e7bf 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -15,7 +15,7 @@ See the documentation below for details on how to configure these services. - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users -- [GitLab Metrics](metrics/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics +- [GitLab Performance Monitoring](metrics/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. diff --git a/doc/integration/metrics/gitlab_configuration.md b/doc/integration/metrics/gitlab_configuration.md index 019db1ad02c..b856e7935a3 100644 --- a/doc/integration/metrics/gitlab_configuration.md +++ b/doc/integration/metrics/gitlab_configuration.md @@ -1,6 +1,6 @@ # GitLab Configuration -GitLab Metrics is disabled by default. To enable it and change any of its +GitLab Performance Monitoring is disabled by default. To enable it and change any of its settings, navigate to the Admin area in **Settings > Metrics** (`/admin/application_settings`). @@ -10,7 +10,7 @@ changes. --- -![GitLab Metrics Admin Settings](img/metrics_gitlab_configuration_settings.png) +![GitLab Performance Monitoring Admin Settings](img/metrics_gitlab_configuration_settings.png) --- @@ -34,6 +34,6 @@ have been performed. Read more on: -- [Introduction to GitLab Metrics](introduction.md) +- [Introduction to GitLab Performance Monitoring](introduction.md) - [InfluxDB Configuration](influxdb_configuration.md) - [InfluxDB Schema](influxdb_schema.md) diff --git a/doc/integration/metrics/influxdb_configuration.md b/doc/integration/metrics/influxdb_configuration.md index 13227f1e889..df5e4af7fad 100644 --- a/doc/integration/metrics/influxdb_configuration.md +++ b/doc/integration/metrics/influxdb_configuration.md @@ -178,7 +178,7 @@ That's it! Now your GitLab instance should send data to InfluxDB. Read more on: -- [Introduction to GitLab Metrics](introduction.md) +- [Introduction to GitLab Performance Monitoring](introduction.md) - [GitLab Configuration](gitlab_configuration.md) - [InfluxDB Schema](influxdb_schema.md) diff --git a/doc/integration/metrics/influxdb_schema.md b/doc/integration/metrics/influxdb_schema.md index a9ef0b446c8..a5a8aebd2d1 100644 --- a/doc/integration/metrics/influxdb_schema.md +++ b/doc/integration/metrics/influxdb_schema.md @@ -82,6 +82,6 @@ view. Read more on: -- [Introduction to GitLab Metrics](introduction.md) +- [Introduction to GitLab Performance Monitoring](introduction.md) - [GitLab Configuration](gitlab_configuration.md) - [InfluxDB Configuration](influxdb_configuration.md) diff --git a/doc/integration/metrics/introduction.md b/doc/integration/metrics/introduction.md index bf388aa2bba..f2460d31302 100644 --- a/doc/integration/metrics/introduction.md +++ b/doc/integration/metrics/introduction.md @@ -1,19 +1,19 @@ -# GitLab Metrics +# GitLab Performance Monitoring GitLab comes with its own application performance measuring system as of GitLab -8.4, simply called "GitLab Metrics". GitLab Metrics is available in both the +8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the Community and Enterprise editions. Apart from this introduction, you are advised to read through the following -documents in order to understand and properly configure GitLab Metrics: +documents in order to understand and properly configure GitLab Performance Monitoring: - [GitLab Configuration](gitlab_configuration.md) - [InfluxDB Configuration](influxdb_configuration.md) - [InfluxDB Schema](influxdb_schema.md) -## Introduction to GitLab Metrics +## Introduction to GitLab Performance Monitoring -GitLab Metrics makes it possible to measure a wide variety of statistics +GitLab Performance Monitoring makes it possible to measure a wide variety of statistics including (but not limited to): - The time it took to complete a transaction (a web request or Sidekiq job). -- cgit v1.2.1 From 82bc7679e782059568402a282905d3d607dbb97d Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 20 Jan 2016 23:02:24 +0100 Subject: Move integration/metrics to monitoring/performance [ci skip] --- doc/integration/metrics/gitlab_configuration.md | 39 ----- .../img/metrics_gitlab_configuration_settings.png | Bin 45148 -> 0 bytes doc/integration/metrics/influxdb_configuration.md | 192 --------------------- doc/integration/metrics/influxdb_schema.md | 87 ---------- doc/integration/metrics/introduction.md | 64 ------- 5 files changed, 382 deletions(-) delete mode 100644 doc/integration/metrics/gitlab_configuration.md delete mode 100644 doc/integration/metrics/img/metrics_gitlab_configuration_settings.png delete mode 100644 doc/integration/metrics/influxdb_configuration.md delete mode 100644 doc/integration/metrics/influxdb_schema.md delete mode 100644 doc/integration/metrics/introduction.md (limited to 'doc/integration') diff --git a/doc/integration/metrics/gitlab_configuration.md b/doc/integration/metrics/gitlab_configuration.md deleted file mode 100644 index b856e7935a3..00000000000 --- a/doc/integration/metrics/gitlab_configuration.md +++ /dev/null @@ -1,39 +0,0 @@ -# GitLab Configuration - -GitLab Performance Monitoring is disabled by default. To enable it and change any of its -settings, navigate to the Admin area in **Settings > Metrics** -(`/admin/application_settings`). - -The minimum required settings you need to set are the InfluxDB host and port. -Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the -changes. - ---- - -![GitLab Performance Monitoring Admin Settings](img/metrics_gitlab_configuration_settings.png) - ---- - -Finally, a restart of all GitLab processes is required for the changes to take -effect: - -```bash -# For Omnibus installations -sudo gitlab-ctl restart - -# For installations from source -sudo service gitlab restart -``` - -## Pending Migrations - -When any migrations are pending, the metrics are disabled until the migrations -have been performed. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [InfluxDB Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) diff --git a/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png b/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png deleted file mode 100644 index 14d82b6ac98..00000000000 Binary files a/doc/integration/metrics/img/metrics_gitlab_configuration_settings.png and /dev/null differ diff --git a/doc/integration/metrics/influxdb_configuration.md b/doc/integration/metrics/influxdb_configuration.md deleted file mode 100644 index df5e4af7fad..00000000000 --- a/doc/integration/metrics/influxdb_configuration.md +++ /dev/null @@ -1,192 +0,0 @@ -# InfluxDB Configuration - -The default settings provided by [InfluxDB] are not sufficient for a high traffic -GitLab environment. The settings discussed in this document are based on the -settings GitLab uses for GitLab.com, depending on your own needs you may need to -further adjust them. - -If you are intending to run InfluxDB on the same server as GitLab, make sure -you have plenty of RAM since InfluxDB can use quite a bit depending on traffic. - -Unless you are going with a budget setup, it's advised to run it separately. - -## Requirements - -- InfluxDB 0.9.5 or newer -- A fairly modern version of Linux -- At least 4GB of RAM -- At least 10GB of storage for InfluxDB data - -Note that the RAM and storage requirements can differ greatly depending on the -amount of data received/stored. To limit the amount of stored data users can -look into [InfluxDB Retention Policies][influxdb-retention]. - -## Installation - -Installing InfluxDB is out of the scope of this document. Please refer to the -[InfluxDB documentation]. - -## InfluxDB Server Settings - -Since InfluxDB has many settings that users may wish to customize themselves -(e.g. what port to run InfluxDB on), we'll only cover the essentials. - -The configuration file in question is usually located at -`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file, -InfluxDB needs to be restarted. - -### Storage Engine - -InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new -storage engine is available, called [TSM Tree]. All users **must** use the new -`tsm1` storage engine as this [will be the default engine][tsm1-commit] in -upcoming InfluxDB releases. - -Make sure you have the following in your configuration file: - -``` -[data] - dir = "/var/lib/influxdb/data" - engine = "tsm1" -``` - -### Admin Panel - -Production environments should have the InfluxDB admin panel **disabled**. This -feature can be disabled by adding the following to your InfluxDB configuration -file: - -``` -[admin] - enabled = false -``` - -### HTTP - -HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana, -thus it should be enabled. When enabling make sure to _also_ enable -authentication: - -``` -[http] - enabled = true - auth-enabled = true -``` - -_**Note:** Before you enable authentication, you might want to [create an -admin user](#create-a-new-admin-user)._ - -### UDP - -GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling -UDP can be done using the following settings: - -``` -[[udp]] - enabled = true - bind-address = ":8089" - database = "gitlab" - batch-size = 1000 - batch-pending = 5 - batch-timeout = "1s" - read-buffer = 209715200 -``` - -This does the following: - -1. Enable UDP and bind it to port 8089 for all addresses. -2. Store any data received in the "gitlab" database. -3. Define a batch of points to be 1000 points in size and allow a maximum of - 5 batches _or_ flush them automatically after 1 second. -4. Define a UDP read buffer size of 200 MB. - -One of the most important settings here is the UDP read buffer size as if this -value is set too low, packets will be dropped. You must also make sure the OS -buffer size is set to the same value, the default value is almost never enough. - -To set the OS buffer size to 200 MB, on Linux you can run the following command: - -```bash -sysctl -w net.core.rmem_max=209715200 -``` - -To make this permanent, add the following to `/etc/sysctl.conf` and restart the -server: - -```bash -net.core.rmem_max=209715200 -``` - -It is **very important** to make sure the buffer sizes are large enough to -handle all data sent to InfluxDB as otherwise you _will_ lose data. The above -buffer sizes are based on the traffic for GitLab.com. Depending on the amount of -traffic, users may be able to use a smaller buffer size, but we highly recommend -using _at least_ 100 MB. - -When enabling UDP, users should take care to not expose the port to the public, -as doing so will allow anybody to write data into your InfluxDB database (as -[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either -whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only -allowing traffic from members of said VLAN. - -## Create a new admin user - -If you want to [enable authentication](#http), you might want to [create an -admin user][influx-admin]: - -``` -influx -execute "CREATE USER thedude WITH PASSWORD '1234' WITH ALL PRIVILEGES" -``` - -## Create the `gitlab` database - -Once you get InfluxDB up and running, you need to create a database for GitLab. -Make sure you have changed the [storage engine](#storage-engine) to `tsm1` -before creating a database. - -_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled -[HTTP authentication](#http), remember to append the username (`-username thedude`) -and password (`-password 1234`) to the commands below._ - -Run the following command to create a database named `gitlab`: - -```bash -influx -execute 'CREATE DATABASE gitlab' -``` - -The name **must** be `gitlab`, do not use any other name. - -Next, make sure that the database was successfully created: - -```bash -influx -execute 'SHOW DATABASES' -``` - -The output should be similar to: - -``` -name: databases ---------------- -name -_internal -gitlab -``` - -That's it! Now your GitLab instance should send data to InfluxDB. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) - -[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management -[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/ -[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/ -[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ -[influxdb]: https://influxdata.com/time-series-platform/influxdb/ -[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/ -[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d -[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user diff --git a/doc/integration/metrics/influxdb_schema.md b/doc/integration/metrics/influxdb_schema.md deleted file mode 100644 index a5a8aebd2d1..00000000000 --- a/doc/integration/metrics/influxdb_schema.md +++ /dev/null @@ -1,87 +0,0 @@ -# InfluxDB Schema - -The following measurements are currently stored in InfluxDB: - -- `PROCESS_file_descriptors` -- `PROCESS_gc_statistics` -- `PROCESS_memory_usage` -- `PROCESS_method_calls` -- `PROCESS_object_counts` -- `PROCESS_transactions` -- `PROCESS_views` - -Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the -process type. In all series, any form of duration is stored in milliseconds. - -## PROCESS_file_descriptors - -This measurement contains the number of open file descriptors over time. The -value field `value` contains the number of descriptors. - -## PROCESS_gc_statistics - -This measurement contains Ruby garbage collection statistics such as the amount -of minor/major GC runs (relative to the last sampling interval), the time spent -in garbage collection cycles, and all fields/values returned by `GC.stat`. - -## PROCESS_memory_usage - -This measurement contains the process' memory usage (in bytes) over time. The -value field `value` contains the number of bytes. - -## PROCESS_method_calls - -This measurement contains the methods called during a transaction along with -their duration, and a name of the transaction action that invoked the method (if -available). The method call duration is stored in the value field `duration`, -while the method name is stored in the tag `method`. The tag `action` contains -the full name of the transaction action. Both the `method` and `action` fields -are in the following format: - -``` -ClassName#method_name -``` - -For example, a method called by the `show` method in the `UsersController` class -would have `action` set to `UsersController#show`. - -## PROCESS_object_counts - -This measurement is used to store retained Ruby objects (per class) and the -amount of retained objects. The number of objects is stored in the `count` value -field while the class name is stored in the `type` tag. - -## PROCESS_transactions - -This measurement is used to store basic transaction details such as the time it -took to complete a transaction, how much time was spent in SQL queries, etc. The -following value fields are available: - -| Value | Description | -| ----- | ----------- | -| `duration` | The total duration of the transaction | -| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers | -| `method_duration` | The total time spent in method calls | -| `sql_duration` | The total time spent in SQL queries | -| `view_duration` | The total time spent in views | - -## PROCESS_views - -This measurement is used to store view rendering timings for a transaction. The -following value fields are available: - -| Value | Description | -| ----- | ----------- | -| `duration` | The rendering time of the view | -| `view` | The path of the view, relative to the application's root directory | - -The `action` tag contains the action name of the transaction that rendered the -view. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Configuration](influxdb_configuration.md) diff --git a/doc/integration/metrics/introduction.md b/doc/integration/metrics/introduction.md deleted file mode 100644 index f2460d31302..00000000000 --- a/doc/integration/metrics/introduction.md +++ /dev/null @@ -1,64 +0,0 @@ -# GitLab Performance Monitoring - -GitLab comes with its own application performance measuring system as of GitLab -8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the -Community and Enterprise editions. - -Apart from this introduction, you are advised to read through the following -documents in order to understand and properly configure GitLab Performance Monitoring: - -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) - -## Introduction to GitLab Performance Monitoring - -GitLab Performance Monitoring makes it possible to measure a wide variety of statistics -including (but not limited to): - -- The time it took to complete a transaction (a web request or Sidekiq job). -- The time spent in running SQL queries and rendering HAML views. -- The time spent executing (instrumented) Ruby methods. -- Ruby object allocations, and retained objects in particular. -- System statistics such as the process' memory usage and open file descriptors. -- Ruby garbage collection statistics. - -Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored -data can be visualized using [Grafana][grafana] or any other application that -supports reading data from InfluxDB. Alternatively data can be queried using the -InfluxDB CLI. - -## Metric Types - -Two types of metrics are collected: - -1. Transaction specific metrics. -1. Sampled metrics, collected at a certain interval in a separate thread. - -### Transaction Metrics - -Transaction metrics are metrics that can be associated with a single -transaction. This includes statistics such as the transaction duration, timings -of any executed SQL queries, time spent rendering HAML views, etc. These metrics -are collected for every Rack request and Sidekiq job processed. - -### Sampled Metrics - -Sampled metrics are metrics that can't be associated with a single transaction. -Examples include garbage collection statistics and retained Ruby objects. These -metrics are collected at a regular interval. This interval is made up out of two -parts: - -1. A user defined interval. -1. A randomly generated offset added on top of the interval, the same offset - can't be used twice in a row. - -The actual interval can be anywhere between a half of the defined interval and a -half above the interval. For example, for a user defined interval of 15 seconds -the actual interval can be anywhere between 7.5 and 22.5. The interval is -re-generated for every sampling run instead of being generated once and re-used -for the duration of the process' lifetime. - -[influxdb]: https://influxdata.com/time-series-platform/influxdb/ -[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ -[grafana]: http://grafana.org/ -- cgit v1.2.1 From 3a6fac4a852c96391ba9f1eea57a873f1a13e12f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 20 Jan 2016 18:00:29 -0500 Subject: Remove Performance Monitoring from Integration docs [ci skip] --- doc/integration/README.md | 1 - 1 file changed, 1 deletion(-) (limited to 'doc/integration') diff --git a/doc/integration/README.md b/doc/integration/README.md index 297e4f6e7bf..5edac746c7b 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -15,7 +15,6 @@ See the documentation below for details on how to configure these services. - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users -- [GitLab Performance Monitoring](metrics/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. -- cgit v1.2.1 From fd00646b5b82b6a890c5cf0a825fa420abd29799 Mon Sep 17 00:00:00 2001 From: Sytse Sijbrandij Date: Fri, 22 Jan 2016 08:57:29 -0800 Subject: Name more OmniAuth providers in the general readme. --- doc/integration/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/integration') diff --git a/doc/integration/README.md b/doc/integration/README.md index 5edac746c7b..846526f4e80 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -8,7 +8,7 @@ See the documentation below for details on how to configure these services. - [Jira](jira.md) Integrate with the JIRA issue tracker - [External issue tracker](external-issue-tracker.md) Redmine, JIRA, etc. - [LDAP](ldap.md) Set up sign in via LDAP -- [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab, and Google via OAuth. +- [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd and Azure - [SAML](saml.md) Configure GitLab as a SAML 2.0 Service Provider - [CAS](cas.md) Configure GitLab to sign in using CAS - [Slack](slack.md) Integrate with the Slack chat service -- cgit v1.2.1 From 6f615500a98c3375d80f5d5aff5b76a8216db0ca Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Sun, 24 Jan 2016 21:22:19 +0100 Subject: Move all integration docs images in a single directory [ci skip] --- doc/integration/facebook.md | 6 +++--- doc/integration/facebook_api_keys.png | Bin 125921 -> 0 bytes doc/integration/facebook_app_settings.png | Bin 134387 -> 0 bytes doc/integration/facebook_website_url.png | Bin 42292 -> 0 bytes doc/integration/github.md | 2 +- doc/integration/github_app.png | Bin 75297 -> 0 bytes doc/integration/gitlab.md | 2 +- doc/integration/gitlab_app.png | Bin 55325 -> 0 bytes doc/integration/gmail_action_buttons_for_gitlab.md | 2 +- doc/integration/gmail_actions_button.png | Bin 17321 -> 0 bytes doc/integration/google.md | 2 +- doc/integration/google_app.png | Bin 52669 -> 0 bytes doc/integration/img/facebook_api_keys.png | Bin 0 -> 125921 bytes doc/integration/img/facebook_app_settings.png | Bin 0 -> 134387 bytes doc/integration/img/facebook_website_url.png | Bin 0 -> 42292 bytes doc/integration/img/github_app.png | Bin 0 -> 75297 bytes doc/integration/img/gitlab_app.png | Bin 0 -> 55325 bytes .../img/gmail_action_buttons_for_gitlab.png | Bin 0 -> 17321 bytes doc/integration/img/google_app.png | Bin 0 -> 52669 bytes .../img/oauth_provider_admin_application.png | Bin 0 -> 55533 bytes .../img/oauth_provider_application_form.png | Bin 0 -> 25075 bytes .../img/oauth_provider_authorized_application.png | Bin 0 -> 17260 bytes .../img/oauth_provider_user_wide_applications.png | Bin 0 -> 46238 bytes doc/integration/img/twitter_app_api_keys.png | Bin 0 -> 72200 bytes doc/integration/img/twitter_app_details.png | Bin 0 -> 121621 bytes doc/integration/jira-integration-points.png | Bin 67854 -> 0 bytes doc/integration/oauth_provider.md | 8 ++++---- doc/integration/oauth_provider/admin_application.png | Bin 55533 -> 0 bytes doc/integration/oauth_provider/application_form.png | Bin 25075 -> 0 bytes .../oauth_provider/authorized_application.png | Bin 17260 -> 0 bytes .../oauth_provider/user_wide_applications.png | Bin 46238 -> 0 bytes doc/integration/twitter.md | 6 +++--- doc/integration/twitter_app_api_keys.png | Bin 72200 -> 0 bytes doc/integration/twitter_app_details.png | Bin 121621 -> 0 bytes 34 files changed, 14 insertions(+), 14 deletions(-) delete mode 100644 doc/integration/facebook_api_keys.png delete mode 100644 doc/integration/facebook_app_settings.png delete mode 100644 doc/integration/facebook_website_url.png delete mode 100644 doc/integration/github_app.png delete mode 100644 doc/integration/gitlab_app.png delete mode 100644 doc/integration/gmail_actions_button.png delete mode 100644 doc/integration/google_app.png create mode 100644 doc/integration/img/facebook_api_keys.png create mode 100644 doc/integration/img/facebook_app_settings.png create mode 100644 doc/integration/img/facebook_website_url.png create mode 100644 doc/integration/img/github_app.png create mode 100644 doc/integration/img/gitlab_app.png create mode 100644 doc/integration/img/gmail_action_buttons_for_gitlab.png create mode 100644 doc/integration/img/google_app.png create mode 100644 doc/integration/img/oauth_provider_admin_application.png create mode 100644 doc/integration/img/oauth_provider_application_form.png create mode 100644 doc/integration/img/oauth_provider_authorized_application.png create mode 100644 doc/integration/img/oauth_provider_user_wide_applications.png create mode 100644 doc/integration/img/twitter_app_api_keys.png create mode 100644 doc/integration/img/twitter_app_details.png delete mode 100644 doc/integration/jira-integration-points.png delete mode 100644 doc/integration/oauth_provider/admin_application.png delete mode 100644 doc/integration/oauth_provider/application_form.png delete mode 100644 doc/integration/oauth_provider/authorized_application.png delete mode 100644 doc/integration/oauth_provider/user_wide_applications.png delete mode 100644 doc/integration/twitter_app_api_keys.png delete mode 100644 doc/integration/twitter_app_details.png (limited to 'doc/integration') diff --git a/doc/integration/facebook.md b/doc/integration/facebook.md index bc1f1673086..77bb75cbfca 100644 --- a/doc/integration/facebook.md +++ b/doc/integration/facebook.md @@ -19,7 +19,7 @@ something else descriptive. 1. Enter the address of your GitLab installation at the bottom of the package - ![Facebook Website URL](facebook_website_url.png) + ![Facebook Website URL](img/facebook_website_url.png) 1. Choose "Next" @@ -29,7 +29,7 @@ something else descriptive. 1. Fill in a contact email for your app - ![Facebook App Settings](facebook_app_settings.png) + ![Facebook App Settings](img/facebook_app_settings.png) 1. Choose "Save Changes" @@ -45,7 +45,7 @@ something else descriptive. 1. You should now see an app key and app secret (see screenshot). Keep this page open as you continue configuration. - ![Facebook API Keys](facebook_api_keys.png) + ![Facebook API Keys](img/facebook_api_keys.png) 1. On your GitLab server, open the configuration file. diff --git a/doc/integration/facebook_api_keys.png b/doc/integration/facebook_api_keys.png deleted file mode 100644 index d6c44ac0f11..00000000000 Binary files a/doc/integration/facebook_api_keys.png and /dev/null differ diff --git a/doc/integration/facebook_app_settings.png b/doc/integration/facebook_app_settings.png deleted file mode 100644 index 30dd21e198a..00000000000 Binary files a/doc/integration/facebook_app_settings.png and /dev/null differ diff --git a/doc/integration/facebook_website_url.png b/doc/integration/facebook_website_url.png deleted file mode 100644 index dc3088bb2fa..00000000000 Binary files a/doc/integration/facebook_website_url.png and /dev/null differ diff --git a/doc/integration/github.md b/doc/integration/github.md index a789d2c814f..886784a27c9 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -22,7 +22,7 @@ GitHub will generate an application ID and secret key for you to use. 1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). Keep this page open as you continue configuration. - ![GitHub app](github_app.png) + ![GitHub app](img/github_app.png) 1. On your GitLab server, open the configuration file. diff --git a/doc/integration/github_app.png b/doc/integration/github_app.png deleted file mode 100644 index d890345ced9..00000000000 Binary files a/doc/integration/github_app.png and /dev/null differ diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index 80e3c0142a0..b215cc7c609 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -28,7 +28,7 @@ GitLab.com will generate an application ID and secret key for you to use. 1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). Keep this page open as you continue configuration. - ![GitLab app](gitlab_app.png) + ![GitLab app](img/gitlab_app.png) 1. On your GitLab server, open the configuration file. diff --git a/doc/integration/gitlab_app.png b/doc/integration/gitlab_app.png deleted file mode 100644 index 3f9391a821b..00000000000 Binary files a/doc/integration/gitlab_app.png and /dev/null differ diff --git a/doc/integration/gmail_action_buttons_for_gitlab.md b/doc/integration/gmail_action_buttons_for_gitlab.md index de45f25ad62..05a91d9bef9 100644 --- a/doc/integration/gmail_action_buttons_for_gitlab.md +++ b/doc/integration/gmail_action_buttons_for_gitlab.md @@ -4,7 +4,7 @@ GitLab supports [Google actions in email](https://developers.google.com/gmail/ma If correctly setup, emails that require an action will be marked in Gmail. -![gmail_actions_button.png](gmail_actions_button.png) +![gmail_actions_button.png](img/gmail_action_buttons_for_gitlab.png) To get this functioning, you need to be registered with Google. [See how to register with Google in this document.](https://developers.google.com/gmail/markup/registering-with-google) diff --git a/doc/integration/gmail_actions_button.png b/doc/integration/gmail_actions_button.png deleted file mode 100644 index b08f54d137b..00000000000 Binary files a/doc/integration/gmail_actions_button.png and /dev/null differ diff --git a/doc/integration/google.md b/doc/integration/google.md index 91e9b2495cc..f9a20dd840d 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -25,7 +25,7 @@ To enable the Google OAuth2 OmniAuth provider you must register your application - Application type: "Web Application" - Authorized JavaScript origins: This isn't really used by GitLab but go ahead and put 'https://gitlab.example.com' here. - Authorized redirect URI: 'https://gitlab.example.com/users/auth/google_oauth2/callback' -1. Under the heading "Client ID for web application" you should see a Client ID and Client secret (see screenshot). Keep this page open as you continue configuration. ![Google app](google_app.png) +1. Under the heading "Client ID for web application" you should see a Client ID and Client secret (see screenshot). Keep this page open as you continue configuration. ![Google app](img/google_app.png) 1. On your GitLab server, open the configuration file. diff --git a/doc/integration/google_app.png b/doc/integration/google_app.png deleted file mode 100644 index 5a62ad35009..00000000000 Binary files a/doc/integration/google_app.png and /dev/null differ diff --git a/doc/integration/img/facebook_api_keys.png b/doc/integration/img/facebook_api_keys.png new file mode 100644 index 00000000000..d6c44ac0f11 Binary files /dev/null and b/doc/integration/img/facebook_api_keys.png differ diff --git a/doc/integration/img/facebook_app_settings.png b/doc/integration/img/facebook_app_settings.png new file mode 100644 index 00000000000..30dd21e198a Binary files /dev/null and b/doc/integration/img/facebook_app_settings.png differ diff --git a/doc/integration/img/facebook_website_url.png b/doc/integration/img/facebook_website_url.png new file mode 100644 index 00000000000..dc3088bb2fa Binary files /dev/null and b/doc/integration/img/facebook_website_url.png differ diff --git a/doc/integration/img/github_app.png b/doc/integration/img/github_app.png new file mode 100644 index 00000000000..d890345ced9 Binary files /dev/null and b/doc/integration/img/github_app.png differ diff --git a/doc/integration/img/gitlab_app.png b/doc/integration/img/gitlab_app.png new file mode 100644 index 00000000000..3f9391a821b Binary files /dev/null and b/doc/integration/img/gitlab_app.png differ diff --git a/doc/integration/img/gmail_action_buttons_for_gitlab.png b/doc/integration/img/gmail_action_buttons_for_gitlab.png new file mode 100644 index 00000000000..b08f54d137b Binary files /dev/null and b/doc/integration/img/gmail_action_buttons_for_gitlab.png differ diff --git a/doc/integration/img/google_app.png b/doc/integration/img/google_app.png new file mode 100644 index 00000000000..5a62ad35009 Binary files /dev/null and b/doc/integration/img/google_app.png differ diff --git a/doc/integration/img/oauth_provider_admin_application.png b/doc/integration/img/oauth_provider_admin_application.png new file mode 100644 index 00000000000..a5f34512aa8 Binary files /dev/null and b/doc/integration/img/oauth_provider_admin_application.png differ diff --git a/doc/integration/img/oauth_provider_application_form.png b/doc/integration/img/oauth_provider_application_form.png new file mode 100644 index 00000000000..ae135db2627 Binary files /dev/null and b/doc/integration/img/oauth_provider_application_form.png differ diff --git a/doc/integration/img/oauth_provider_authorized_application.png b/doc/integration/img/oauth_provider_authorized_application.png new file mode 100644 index 00000000000..d3ce05be9cc Binary files /dev/null and b/doc/integration/img/oauth_provider_authorized_application.png differ diff --git a/doc/integration/img/oauth_provider_user_wide_applications.png b/doc/integration/img/oauth_provider_user_wide_applications.png new file mode 100644 index 00000000000..719e1974068 Binary files /dev/null and b/doc/integration/img/oauth_provider_user_wide_applications.png differ diff --git a/doc/integration/img/twitter_app_api_keys.png b/doc/integration/img/twitter_app_api_keys.png new file mode 100644 index 00000000000..1076337172a Binary files /dev/null and b/doc/integration/img/twitter_app_api_keys.png differ diff --git a/doc/integration/img/twitter_app_details.png b/doc/integration/img/twitter_app_details.png new file mode 100644 index 00000000000..b95e8af8a74 Binary files /dev/null and b/doc/integration/img/twitter_app_details.png differ diff --git a/doc/integration/jira-integration-points.png b/doc/integration/jira-integration-points.png deleted file mode 100644 index 0692a7b458a..00000000000 Binary files a/doc/integration/jira-integration-points.png and /dev/null differ diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md index 192c321f712..dbe5a175c82 100644 --- a/doc/integration/oauth_provider.md +++ b/doc/integration/oauth_provider.md @@ -15,16 +15,16 @@ GitLab has two ways to add new OAuth2 application to an instance, you can add ap ### Adding application through profile Go to your profile section 'Application' and press button 'New Application' -![applications](oauth_provider/user_wide_applications.png) +![applications](img/oauth_provider_user_wide_applications.png) After this you will see application form, where "Name" is arbitrary name, "Redirect URI" is URL in your app where users will be sent after authorization on GitLab.com. -![application_form](oauth_provider/application_form.png) +![application_form](img/oauth_provider_application_form.png) ### Authorized application Every application you authorized will be shown in your "Authorized application" sections. -![authorized_application](oauth_provider/authorized_application.png) +![authorized_application](img/oauth_provider_authorized_application.png) At any time you can revoke access just clicking button "Revoke" @@ -32,4 +32,4 @@ At any time you can revoke access just clicking button "Revoke" If you want to create application that does not belong to certain user you can create it from admin area -![admin_application](oauth_provider/admin_application.png) \ No newline at end of file +![admin_application](img/oauth_provider_admin_application.png) diff --git a/doc/integration/oauth_provider/admin_application.png b/doc/integration/oauth_provider/admin_application.png deleted file mode 100644 index a5f34512aa8..00000000000 Binary files a/doc/integration/oauth_provider/admin_application.png and /dev/null differ diff --git a/doc/integration/oauth_provider/application_form.png b/doc/integration/oauth_provider/application_form.png deleted file mode 100644 index ae135db2627..00000000000 Binary files a/doc/integration/oauth_provider/application_form.png and /dev/null differ diff --git a/doc/integration/oauth_provider/authorized_application.png b/doc/integration/oauth_provider/authorized_application.png deleted file mode 100644 index d3ce05be9cc..00000000000 Binary files a/doc/integration/oauth_provider/authorized_application.png and /dev/null differ diff --git a/doc/integration/oauth_provider/user_wide_applications.png b/doc/integration/oauth_provider/user_wide_applications.png deleted file mode 100644 index 719e1974068..00000000000 Binary files a/doc/integration/oauth_provider/user_wide_applications.png and /dev/null differ diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index 52ed4a22339..4769f26b259 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -14,7 +14,7 @@ To enable the Twitter OmniAuth provider you must register your application with - Callback URL: 'https://gitlab.example.com/users/auth/twitter/callback' - Agree to the "Developer Agreement". - ![Twitter App Details](twitter_app_details.png) + ![Twitter App Details](img/twitter_app_details.png) 1. Select "Create your Twitter application." 1. Select the "Settings" tab. @@ -27,7 +27,7 @@ To enable the Twitter OmniAuth provider you must register your application with 1. You should now see an API key and API secret (see screenshot). Keep this page open as you continue configuration. - ![Twitter app](twitter_app_api_keys.png) + ![Twitter app](img/twitter_app_api_keys.png) 1. On your GitLab server, open the configuration file. @@ -76,4 +76,4 @@ To enable the Twitter OmniAuth provider you must register your application with 1. Restart GitLab for the changes to take effect. -On the sign in page there should now be a Twitter icon below the regular sign in form. Click the icon to begin the authentication process. Twitter will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. \ No newline at end of file +On the sign in page there should now be a Twitter icon below the regular sign in form. Click the icon to begin the authentication process. Twitter will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. diff --git a/doc/integration/twitter_app_api_keys.png b/doc/integration/twitter_app_api_keys.png deleted file mode 100644 index 1076337172a..00000000000 Binary files a/doc/integration/twitter_app_api_keys.png and /dev/null differ diff --git a/doc/integration/twitter_app_details.png b/doc/integration/twitter_app_details.png deleted file mode 100644 index b95e8af8a74..00000000000 Binary files a/doc/integration/twitter_app_details.png and /dev/null differ -- cgit v1.2.1 From 040fb46dc777995155d6d2a234365508b9b3124b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Sun, 24 Jan 2016 21:36:49 +0100 Subject: Move supported omniauth providers to the top [ci skip] --- doc/integration/omniauth.md | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) (limited to 'doc/integration') diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index e9e17eb4165..8e6627b2be5 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -9,6 +9,23 @@ Configuring OmniAuth does not prevent standard GitLab authentication or LDAP (if - [Enable OmniAuth for an Existing User](#enable-omniauth-for-an-existing-user) - [OmniAuth configuration sample when using Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master#omniauth-google-twitter-github-login) +## Supported Providers + +This is a list of the current supported OmniAuth providers. Before proceeding +on each provider's documentation, make sure to first read this document as it +contains some settings that are common for all providers. + +- [GitHub](github.md) +- [Bitbucket](bitbucket.md) +- [GitLab.com](gitlab.md) +- [Google](google.md) +- [Facebook](facebook.md) +- [Twitter](twitter.md) +- [Shibboleth](shibboleth.md) +- [SAML](saml.md) +- [Crowd](crowd.md) +- [Azure](azure.md) + ## Initial OmniAuth Configuration Before configuring individual OmniAuth providers there are a few global settings that are in common for all providers that we need to consider. @@ -67,19 +84,6 @@ If you want to change these settings: Now we can choose one or more of the Supported Providers below to continue configuration. -## Supported Providers - -- [GitHub](github.md) -- [Bitbucket](bitbucket.md) -- [GitLab.com](gitlab.md) -- [Google](google.md) -- [Facebook](facebook.md) -- [Twitter](twitter.md) -- [Shibboleth](shibboleth.md) -- [SAML](saml.md) -- [Crowd](crowd.md) -- [Azure](azure.md) - ## Enable OmniAuth for an Existing User Existing users can enable OmniAuth for specific providers after the account is created. For example, if the user originally signed in with LDAP an OmniAuth provider such as Twitter can be enabled. Follow the steps below to enable an OmniAuth provider for an existing user. -- cgit v1.2.1 From 3629984b96553cced12855fd4bc096d5268abd6d Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 25 Jan 2016 10:08:11 +0100 Subject: Move JIRA from integration to project_services --- doc/integration/README.md | 2 +- doc/integration/external-issue-tracker.md | 2 +- doc/integration/img/jira_issue_reference.png | Bin 39942 -> 0 bytes doc/integration/img/jira_merge_request_close.png | Bin 111150 -> 0 bytes doc/integration/img/jira_project_name.png | Bin 60598 -> 0 bytes doc/integration/img/jira_service.png | Bin 59082 -> 0 bytes doc/integration/img/jira_service_close_issue.png | Bin 88433 -> 0 bytes doc/integration/img/jira_service_page.png | Bin 35496 -> 0 bytes doc/integration/img/jira_workflow_screenshot.png | Bin 121534 -> 0 bytes doc/integration/jira.md | 150 +---------------------- 10 files changed, 4 insertions(+), 150 deletions(-) delete mode 100644 doc/integration/img/jira_issue_reference.png delete mode 100644 doc/integration/img/jira_merge_request_close.png delete mode 100644 doc/integration/img/jira_project_name.png delete mode 100644 doc/integration/img/jira_service.png delete mode 100644 doc/integration/img/jira_service_close_issue.png delete mode 100644 doc/integration/img/jira_service_page.png delete mode 100644 doc/integration/img/jira_workflow_screenshot.png (limited to 'doc/integration') diff --git a/doc/integration/README.md b/doc/integration/README.md index 846526f4e80..83116bc8370 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -5,7 +5,7 @@ trackers and external authentication. See the documentation below for details on how to configure these services. -- [Jira](jira.md) Integrate with the JIRA issue tracker +- [Jira](../project_services/jira.md) Integrate with the JIRA issue tracker - [External issue tracker](external-issue-tracker.md) Redmine, JIRA, etc. - [LDAP](ldap.md) Set up sign in via LDAP - [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd and Azure diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 3543a67dd49..e25af546527 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -19,7 +19,7 @@ To enable an external issue tracker you must configure the appropriate **Service Visit the links below for details: - [Redmine](../project_services/redmine.md) -- [Jira](jira.md) +- [Jira](../proect_services/jira.md) ### Service Template diff --git a/doc/integration/img/jira_issue_reference.png b/doc/integration/img/jira_issue_reference.png deleted file mode 100644 index 15739a22dc7..00000000000 Binary files a/doc/integration/img/jira_issue_reference.png and /dev/null differ diff --git a/doc/integration/img/jira_merge_request_close.png b/doc/integration/img/jira_merge_request_close.png deleted file mode 100644 index 1e78daf105f..00000000000 Binary files a/doc/integration/img/jira_merge_request_close.png and /dev/null differ diff --git a/doc/integration/img/jira_project_name.png b/doc/integration/img/jira_project_name.png deleted file mode 100644 index 5986fdb63fb..00000000000 Binary files a/doc/integration/img/jira_project_name.png and /dev/null differ diff --git a/doc/integration/img/jira_service.png b/doc/integration/img/jira_service.png deleted file mode 100644 index 1f6628c4371..00000000000 Binary files a/doc/integration/img/jira_service.png and /dev/null differ diff --git a/doc/integration/img/jira_service_close_issue.png b/doc/integration/img/jira_service_close_issue.png deleted file mode 100644 index 67dfc6144c4..00000000000 Binary files a/doc/integration/img/jira_service_close_issue.png and /dev/null differ diff --git a/doc/integration/img/jira_service_page.png b/doc/integration/img/jira_service_page.png deleted file mode 100644 index 2b37eda3520..00000000000 Binary files a/doc/integration/img/jira_service_page.png and /dev/null differ diff --git a/doc/integration/img/jira_workflow_screenshot.png b/doc/integration/img/jira_workflow_screenshot.png deleted file mode 100644 index 8635a32eb68..00000000000 Binary files a/doc/integration/img/jira_workflow_screenshot.png and /dev/null differ diff --git a/doc/integration/jira.md b/doc/integration/jira.md index de574d53410..78aa6634116 100644 --- a/doc/integration/jira.md +++ b/doc/integration/jira.md @@ -1,149 +1,3 @@ -# GitLab Jira integration +# GitLab JIRA integration -GitLab can be configured to interact with Jira. Configuration happens via -username and password. Connecting to a Jira server via CAS is not possible. - -Each project can be configured to connect to a different Jira instance, see the -[configuration](#configuration) section. If you have one Jira instance you can -pre-fill the settings page with a default template. To configure the template -see the [Services Templates][services-templates] document. - -Once the project is connected to Jira, you can reference and close the issues -in Jira directly from GitLab. - -## Table of Contents - -* [Referencing Jira Issues from GitLab](#referencing-jira-issues) -* [Closing Jira Issues from GitLab](#closing-jira-issues) -* [Configuration](#configuration) - -### Referencing Jira Issues - -When GitLab project has Jira issue tracker configured and enabled, mentioning -Jira issue in GitLab will automatically add a comment in Jira issue with the -link back to GitLab. This means that in comments in merge requests and commits -referencing an issue, eg. `PROJECT-7`, will add a comment in Jira issue in the -format: - -``` - USER mentioned this issue in LINK_TO_THE_MENTION -``` - -* `USER` A user that mentioned the issue. This is the link to the user profile in GitLab. -* `LINK_TO_THE_MENTION` Link to the origin of mention with a name of the entity where Jira issue was mentioned. -Can be commit or merge request. - -![example of mentioning or closing the Jira issue](img/jira_issue_reference.png) - ---- - -### Closing Jira Issues - -Jira issues can be closed directly from GitLab by using trigger words, eg. -`Resolves PROJECT-1`, `Closes PROJECT-1` or `Fixes PROJECT-1`, in commits and -merge requests. When a commit which contains the trigger word in the commit -message is pushed, GitLab will add a comment in the mentioned Jira issue. - -For example, for project named `PROJECT` in Jira, we implemented a new feature -and created a merge request in GitLab. - -This feature was requested in Jira issue `PROJECT-7`. Merge request in GitLab -contains the improvement and in merge request description we say that this -merge request `Closes PROJECT-7` issue. - -Once this merge request is merged, the Jira issue will be automatically closed -with a link to the commit that resolved the issue. - -![A Git commit that causes the Jira issue to be closed](img/jira_merge_request_close.png) - ---- - -![The GitLab integration user leaves a comment on Jira](img/jira_service_close_issue.png) - ---- - -## Configuration - -### Configuring JIRA - -We need to create a user in JIRA which will have access to all projects that -need to integrate with GitLab. Login to your JIRA instance as admin and under -Administration go to User Management and create a new user. - -As an example, we'll create a user named `gitlab` and add it to `jira-developers` -group. - -**It is important that the user `gitlab` has write-access to projects in JIRA** - -### Configuring GitLab - -JIRA configuration in GitLab is done via a project's **Services**. - -#### GitLab 7.8 and up with JIRA v6.x - -See next section. - -#### GitLab 7.8 and up - -_The currently supported JIRA versions are v6.x and v7.x._ - -To enable JIRA integration in a project, navigate to the project's -**Settings > Services > JIRA**. - -Fill in the required details on the page as described in the table below. - -| Field | Description | -| ----- | ----------- | -| `description` | A name for the issue tracker (to differentiate between instances, for instance). | -| `project url` | The URL to the JIRA project which is being linked to this GitLab project. | -| `issues url` | The URL to the JIRA project issues overview for the project that is linked to this GitLab project. | -| `new issue url` | This is the URL to create a new issue in JIRA for the project linked to this GitLab project. | -| `api url` | The base URL of the JIRA API. It may be omitted, in which case GitLab will automatically use API version `2` based on the `project url`, i.e. `https://jira.example.com/rest/api/2`. | -| `username` | The username of the user created in [configuring JIRA step](#configuring-jira). | -| `password` |The password of the user created in [configuring JIRA step](#configuring-jira). | -| `Jira issue transition` | This is the ID of a transition that moves issues to a closed state. You can find this number under JIRA workflow administration ([see screenshot](img/jira_workflow_screenshot.png)). By default, this ID is `2` (in the example image, this is `2` as well) | - -After saving the configuration, your GitLab project will be able to interact -with the linked JIRA project. - -![Jira service page](img/jira_service_page.png) - ---- - -#### GitLab 6.x-7.7 with JIRA v6.x - -_**Note:** GitLab versions 7.8 and up contain various integration improvements. -We strongly recommend upgrading._ - -In `gitlab.yml` enable the JIRA issue tracker section by -[uncommenting these lines][jira-gitlab-yml]. This will make sure that all -issues within GitLab are pointing to the JIRA issue tracker. - -After you set this, you will be able to close issues in JIRA by a commit in -GitLab. - -Go to your project's **Settings** page and fill in the project name for the -JIRA project: - -![Set the JIRA project name in GitLab to 'NEW'](img/jira_project_name.png) - ---- - -You can also enable the JIRA service that will allow you to interact with JIRA -issues. Go to the **Settings > Services > JIRA** and: - -1. Tick the active check box to enable the service -1. Supply the URL to JIRA server, for example http://jira.example.com -1. Supply the username of a user we created under `Configuring JIRA` section, - for example `gitlab` -1. Supply the password of the user -1. Optional: supply the JIRA API version, default is version `2` -1. Optional: supply the JIRA issue transition ID (issue transition to closed). - This is dependent on JIRA settings, default is `2` -1. Hit save - - -![Jira services page](img/jira_service.png) - -[services-templates]: ../project_services/services_templates.md -[jira-gitlab-yml]: https://gitlab.com/subscribers/gitlab-ee/blob/6-8-stable-ee/config/gitlab.yml.example#L111-115 +This document was moved under [project_services/jira](../project_services/jira.md). -- cgit v1.2.1 From 468dbfe937b60cc0b06f062fd01dcfd70450bcf1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marcin=20Zaj=C4=85czkowski?= Date: Mon, 25 Jan 2016 13:06:09 +0000 Subject: Update Slack integration configuration --- doc/integration/slack.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) (limited to 'doc/integration') diff --git a/doc/integration/slack.md b/doc/integration/slack.md index 84f1d74c058..ecbe0d3e887 100644 --- a/doc/integration/slack.md +++ b/doc/integration/slack.md @@ -6,15 +6,17 @@ To enable Slack integration you must create an Incoming WebHooks integration on 1. [Sign in to Slack](https://slack.com/signin) -1. Select **Configure Integrations** from the dropdown next to your team name. +1. Select **Apps & Custom Integrations** from the dropdown next to your team name. -1. Select the **All Services** tab +1. Click the **Configure** link (right-upper corner). -1. Click **Add** next to Incoming Webhooks +1. Select the **Custom integrations** tab. -1. Pick Incoming WebHooks +1. Click the **Incoming WebHooks** row. -1. Choose the channel name you want to send notifications to +1. Click the **Add configuration** button. + +1. Choose the channel name you want to send notifications to. 1. Click **Add Incoming WebHooks Integration** - Optional step; You can change bot's name and avatar by clicking modifying the bot name or avatar under **Integration Settings**. -- cgit v1.2.1 From 81d79c7f0a8e8bd54cf39a886ab89ec4532f34b1 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 25 Jan 2016 23:52:04 +0100 Subject: Fix typos, grammar and styling --- doc/integration/external-issue-tracker.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/integration') diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index e25af546527..a2d7e922aad 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -19,7 +19,7 @@ To enable an external issue tracker you must configure the appropriate **Service Visit the links below for details: - [Redmine](../project_services/redmine.md) -- [Jira](../proect_services/jira.md) +- [Jira](../project_services/jira.md) ### Service Template -- cgit v1.2.1