diff options
Diffstat (limited to 'doc')
25 files changed, 554 insertions, 196 deletions
diff --git a/doc/administration/index.md b/doc/administration/index.md index 58922b71ae7..0b199eecefd 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -106,20 +106,22 @@ server with IMAP authentication on Ubuntu, to be used with Reply by email. ## Monitoring GitLab -- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. - - [IP whitelist](monitoring/ip_whitelist.md): Monitor endpoints that provide health check information when probed. -- [Monitoring GitHub imports](monitoring/github_imports.md): GitLab's GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer. +- [Monitoring GitLab](monitoring/index.md): + - [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. + - [IP whitelist](monitoring/ip_whitelist.md): Monitor endpoints that provide health check information when probed. + - [Monitoring GitHub imports](monitoring/github_imports.md): GitLab's GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer. - [Conversational Development (ConvDev) Index](../user/admin_area/monitoring/convdev.md): Provides an overview of your entire instance's feature usage. ### Performance Monitoring -- [GitLab Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring. -- [GitLab performance monitoring with InfluxDB](monitoring/performance/introduction.md): Configure GitLab and InfluxDB for measuring performance metrics. - - [InfluxDB Schema](monitoring/performance/influxdb_schema.md): Measurements stored in InfluxDB. -- [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. -- [GitLab performance monitoring with Grafana](monitoring/prometheus/index.md): Configure GitLab to visualize time series metrics through graphs and dashboards. -- [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. -- [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page. +- [GitLab Performance Monitoring](monitoring/performance/index.md): + - [Enable Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring. + - [GitLab performance monitoring with InfluxDB](monitoring/performance/influxdb_configuration.md): Configure GitLab and InfluxDB for measuring performance metrics. + - [InfluxDB Schema](monitoring/performance/influxdb_schema.md): Measurements stored in InfluxDB. + - [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. + - [GitLab performance monitoring with Grafana](monitoring/performance/grafana_configuration.md): Configure GitLab to visualize time series metrics through graphs and dashboards. + - [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. + - [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page. ## Troubleshooting diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md new file mode 100644 index 00000000000..d6333ee62b4 --- /dev/null +++ b/doc/administration/monitoring/index.md @@ -0,0 +1,9 @@ +# Monitoring GitLab + +Explore our features to monitor your GitLab instance: + +- [Performance monitoring](performance/index.md): GitLab Performance Monitoring makes it possible to measure a wide variety of statistics of your instance. +- [Prometheus](prometheus/index.md): Prometheus is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. +- [GitHub imports](github_imports.md): Monitor the health and progress of GitLab's GitHub importer with various Prometheus metrics. +- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. + - [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that provide health check information when probed. diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md new file mode 100644 index 00000000000..f5f0363ed38 --- /dev/null +++ b/doc/administration/monitoring/performance/index.md @@ -0,0 +1,72 @@ +# GitLab Performance Monitoring + +GitLab comes with its own application performance measuring system as of GitLab +8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the +Community and Enterprise editions. + +Apart from this introduction, you are advised to read through the following +documents in order to understand and properly configure GitLab Performance Monitoring: + +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Install/Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) +- [Grafana Install/Configuration](grafana_configuration.md) +- [Performance bar](performance_bar.md) +- [Request profiling](request_profiling.md) + +>**Note:** +Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect +metrics. It will eventually replace InfluxDB when their metrics collection is +on par. Read more in the [Prometheus documentation](../prometheus/index.md). + +## Introduction to GitLab Performance Monitoring + +GitLab Performance Monitoring makes it possible to measure a wide variety of statistics +including (but not limited to): + +- The time it took to complete a transaction (a web request or Sidekiq job). +- The time spent in running SQL queries and rendering HAML views. +- The time spent executing (instrumented) Ruby methods. +- Ruby object allocations, and retained objects in particular. +- System statistics such as the process' memory usage and open file descriptors. +- Ruby garbage collection statistics. + +Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored +data can be visualized using [Grafana][grafana] or any other application that +supports reading data from InfluxDB. Alternatively data can be queried using the +InfluxDB CLI. + +## Metric Types + +Two types of metrics are collected: + +1. Transaction specific metrics. +1. Sampled metrics, collected at a certain interval in a separate thread. + +### Transaction Metrics + +Transaction metrics are metrics that can be associated with a single +transaction. This includes statistics such as the transaction duration, timings +of any executed SQL queries, time spent rendering HAML views, etc. These metrics +are collected for every Rack request and Sidekiq job processed. + +### Sampled Metrics + +Sampled metrics are metrics that can't be associated with a single transaction. +Examples include garbage collection statistics and retained Ruby objects. These +metrics are collected at a regular interval. This interval is made up out of two +parts: + +1. A user defined interval. +1. A randomly generated offset added on top of the interval, the same offset + can't be used twice in a row. + +The actual interval can be anywhere between a half of the defined interval and a +half above the interval. For example, for a user defined interval of 15 seconds +the actual interval can be anywhere between 7.5 and 22.5. The interval is +re-generated for every sampling run instead of being generated once and re-used +for the duration of the process' lifetime. + +[influxdb]: https://influxdata.com/time-series-platform/influxdb/ +[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ +[grafana]: http://grafana.org/ diff --git a/doc/administration/monitoring/performance/introduction.md b/doc/administration/monitoring/performance/introduction.md index 17c2b4b70d3..37a5388d2fc 100644 --- a/doc/administration/monitoring/performance/introduction.md +++ b/doc/administration/monitoring/performance/introduction.md @@ -1,70 +1 @@ -# GitLab Performance Monitoring - -GitLab comes with its own application performance measuring system as of GitLab -8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the -Community and Enterprise editions. - -Apart from this introduction, you are advised to read through the following -documents in order to understand and properly configure GitLab Performance Monitoring: - -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Install/Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) -- [Grafana Install/Configuration](grafana_configuration.md) - ->**Note:** -Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect -metrics. It will eventually replace InfluxDB when their metrics collection is -on par. Read more in the [Prometheus documentation](../prometheus/index.md). - -## Introduction to GitLab Performance Monitoring - -GitLab Performance Monitoring makes it possible to measure a wide variety of statistics -including (but not limited to): - -- The time it took to complete a transaction (a web request or Sidekiq job). -- The time spent in running SQL queries and rendering HAML views. -- The time spent executing (instrumented) Ruby methods. -- Ruby object allocations, and retained objects in particular. -- System statistics such as the process' memory usage and open file descriptors. -- Ruby garbage collection statistics. - -Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored -data can be visualized using [Grafana][grafana] or any other application that -supports reading data from InfluxDB. Alternatively data can be queried using the -InfluxDB CLI. - -## Metric Types - -Two types of metrics are collected: - -1. Transaction specific metrics. -1. Sampled metrics, collected at a certain interval in a separate thread. - -### Transaction Metrics - -Transaction metrics are metrics that can be associated with a single -transaction. This includes statistics such as the transaction duration, timings -of any executed SQL queries, time spent rendering HAML views, etc. These metrics -are collected for every Rack request and Sidekiq job processed. - -### Sampled Metrics - -Sampled metrics are metrics that can't be associated with a single transaction. -Examples include garbage collection statistics and retained Ruby objects. These -metrics are collected at a regular interval. This interval is made up out of two -parts: - -1. A user defined interval. -1. A randomly generated offset added on top of the interval, the same offset - can't be used twice in a row. - -The actual interval can be anywhere between a half of the defined interval and a -half above the interval. For example, for a user defined interval of 15 seconds -the actual interval can be anywhere between 7.5 and 22.5. The interval is -re-generated for every sampling run instead of being generated once and re-used -for the duration of the process' lifetime. - -[influxdb]: https://influxdata.com/time-series-platform/influxdb/ -[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ -[grafana]: http://grafana.org/ +This document was moved to [another location](index.md). diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index 1304476e678..3a2cced37bf 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -89,9 +89,11 @@ email address in order to sign up. If you also host a public-facing GitLab instance at `hooli.com` and set your incoming email domain to `hooli.com`, an attacker could abuse the "Create new -issue by email" feature by using a project's unique address as the email when -signing up for Slack, which would send a confirmation email, which would create -a new issue on the project owned by the attacker, allowing them to click the +issue by email" or +"[Create new merge request by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email)" +features by using a project's unique address as the email when signing up for +Slack, which would send a confirmation email, which would create a new issue or +merge request on the project owned by the attacker, allowing them to click the confirmation link and validate their account on your company's private Slack instance. diff --git a/doc/api/groups.md b/doc/api/groups.md index c1b5737c247..de730cdd869 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -365,13 +365,15 @@ POST /groups Parameters: -- `name` (required) - The name of the group -- `path` (required) - The path of the group -- `description` (optional) - The group's description -- `visibility` (optional) - The group's visibility. Can be `private`, `internal`, or `public`. -- `lfs_enabled` (optional) - Enable/disable Large File Storage (LFS) for the projects in this group -- `request_access_enabled` (optional) - Allow users to request member access. -- `parent_id` (optional) - The parent group id for creating nested group. +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `name` | string | yes | The name of the group | +| `path` | string | yes | The path of the group | +| `description` | string | no | The group's description | +| `visibility` | string | no | The group's visibility. Can be `private`, `internal`, or `public`. | +| `lfs_enabled` | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group | +| `request_access_enabled` | boolean | no | Allow users to request member access. | +| `parent_id` | integer | no | The parent group id for creating nested group. | ## Transfer project to group @@ -383,8 +385,10 @@ POST /groups/:id/projects/:project_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user -- `project_id` (required) - The ID or path of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `project_id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | ## Update group diff --git a/doc/api/tags.md b/doc/api/tags.md index bebe6536b6e..fa25dc76452 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -12,7 +12,11 @@ GET /projects/:id/repository/tags Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string| yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user| +| `order_by` | string | no | Return tags ordered by `name` or `updated` fields. Default is `updated` | +| `sort` | string | no | Return tags sorted in `asc` or `desc` order. Default is `desc` | ```json [ diff --git a/doc/development/README.md b/doc/development/README.md index 7e4c767692a..b624aa37c70 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -37,6 +37,7 @@ comments: false - [`Gemfile` guidelines](gemfile.md) - [Sidekiq debugging](sidekiq_debugging.md) - [Gotchas](gotchas.md) to avoid +- [Avoid modules with instance variables](module_with_instance_variables.md) if possible - [Issue and merge requests state models](object_state_models.md) - [How to dump production data to staging](db_dump.md) - [Working with the GitHub importer](github_importer.md) @@ -81,10 +82,9 @@ comments: false ## Documentation guides -- [Documentation styleguide](doc_styleguide.md): Use this styleguide if you are - contributing to the documentation. - [Writing documentation](writing_documentation.md) - - [Distinction between general documentation and technical articles](writing_documentation.md#distinction-between-general-documentation-and-technical-articles) +- [Documentation styleguide](doc_styleguide.md) +- [Markdown](../user/markdown.md) ## Internationalization (i18n) guides diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index c2ca8966a3f..5786287d00c 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -8,7 +8,7 @@ might encounter or should avoid during development of GitLab CE and EE. Consider the following factory: ```ruby -FactoryGirl.define do +FactoryBot.define do factory :label do sequence(:title) { |n| "label#{n}" } end @@ -53,7 +53,7 @@ When run, this spec doesn't do what we might expect: (compared using ==) ``` -That's because FactoryGirl sequences are not reseted for each example. +That's because FactoryBot sequences are not reseted for each example. Please remember that sequence-generated values exist only to avoid having to explicitly set attributes that have a uniqueness constraint when using a factory. diff --git a/doc/development/module_with_instance_variables.md b/doc/development/module_with_instance_variables.md new file mode 100644 index 00000000000..48a1b7f847e --- /dev/null +++ b/doc/development/module_with_instance_variables.md @@ -0,0 +1,242 @@ +## Modules with instance variables could be considered harmful + +### Background + +Rails somehow encourages people using modules and instance variables +everywhere. For example, using instance variables in the controllers, +helpers, and views. They're also encouraging the use of +`ActiveSupport::Concern`, which further strengthens the idea of +saving everything in a giant, single object, and people could access +everything in that one giant object. + +### The problems + +Of course this is convenient to develop, because we just have everything +within reach. However this has a number of downsides when that chosen object +is growing, it would later become out of control for the same reason. + +There are just too many things in the same context, and we don't know if +those things are tightly coupled or not, depending on each others or not. +It's very hard to tell when the complexity grows to a point, and it makes +tracking the code also extremely hard. For example, a class could be using +3 different instance variables, and all of them could be initialized and +manipulated from 3 different modules. It's hard to track when those variables +start giving us troubles. We don't know which module would suddenly change +one of the variables. Everything could touch anything. + +### Similar concerns + +People are saying multiple inheritance is bad. Mixing multiple modules with +multiple instance variables scattering everywhere suffer from the same issue. +The same applies to `ActiveSupport::Concern`. See: +[Consider replacing concerns with dedicated classes & composition]( +https://gitlab.com/gitlab-org/gitlab-ce/issues/23786) + +There's also a similar idea: +[Use decorators and interface segregation to solve overgrowing models problem]( +https://gitlab.com/gitlab-org/gitlab-ce/issues/13484) + +Note that `included` doesn't solve the whole issue. They define the +dependencies, but they still allow each modules to talk implicitly via the +instance variables in the final giant object, and that's where the problem is. + +### Solutions + +We should split the giant object into multiple objects, and they communicate +with each other with the API, i.e. public methods. In short, composition over +inheritance. This way, each smaller objects would have their own respective +limited states, i.e. instance variables. If one instance variable goes wrong, +we would be very clear that it's from that single small object, because +no one else could be touching it. + +With clearly defined API, this would make things less coupled and much easier +to debug and track, and much more extensible for other objects to use, because +they communicate in a clear way, rather than implicit dependencies. + +### Acceptable use + +However, it's not always bad to use instance variables in a module, +as long as it's contained in the same module; that is, no other modules or +objects are touching them, then it would be an acceptable use. + +We especially allow the case where a single instance variable is used with +`||=` to setup the value. This would look like: + +``` ruby +module M + def f + @f ||= true + end +end +``` + +Unfortunately it's not easy to code more complex rules into the cop, so +we rely on people's best judgement. If we could find another good pattern +we could easily add to the cop, we should do it. + +### How to rewrite and avoid disabling this cop + +Even if we could just disable the cop, we should avoid doing so. Some code +could be easily rewritten in simple form. Consider this acceptable method: + +``` ruby +module Gitlab + module Emoji + def emoji_unicode_version(name) + @emoji_unicode_versions_by_name ||= + JSON.parse(File.read(Rails.root.join('fixtures', 'emojis', 'emoji-unicode-version-map.json'))) + @emoji_unicode_versions_by_name[name] + end + end +end +``` + +This method is totally fine because it's already self-contained. No other +methods should be using `@emoji_unicode_versions_by_name` and we're good. +However it's still offending the cop because it's not just `||=`, and the +cop is not smart enough to judge that this is fine. + +On the other hand, we could split this method into two: + +``` ruby +module Gitlab + module Emoji + def emoji_unicode_version(name) + emoji_unicode_versions_by_name[name] + end + + private + + def emoji_unicode_versions_by_name + @emoji_unicode_versions_by_name ||= + JSON.parse(File.read(Rails.root.join('fixtures', 'emojis', 'emoji-unicode-version-map.json'))) + end + end +end +``` + +Now the cop won't complain. Here's a bad example which we could rewrite: + +``` ruby +module SpamCheckService + def filter_spam_check_params + @request = params.delete(:request) + @api = params.delete(:api) + @recaptcha_verified = params.delete(:recaptcha_verified) + @spam_log_id = params.delete(:spam_log_id) + end + + def spam_check(spammable, user) + spam_service = SpamService.new(spammable, @request) + + spam_service.when_recaptcha_verified(@recaptcha_verified, @api) do + user.spam_logs.find_by(id: @spam_log_id)&.update!(recaptcha_verified: true) + end + end +end +``` + +There are several implicit dependencies here. First, `params` should be +defined before use. Second, `filter_spam_check_params` should be called +before `spam_check`. These are all implicit and the includer could be using +those instance variables without awareness. + +This should be rewritten like: + +``` ruby +class SpamCheckService + def initialize(request:, api:, recaptcha_verified:, spam_log_id:) + @request = request + @api = api + @recaptcha_verified = recaptcha_verified + @spam_log_id = spam_log_id + end + + def spam_check(spammable, user) + spam_service = SpamService.new(spammable, @request) + + spam_service.when_recaptcha_verified(@recaptcha_verified, @api) do + user.spam_logs.find_by(id: @spam_log_id)&.update!(recaptcha_verified: true) + end + end +end +``` + +And use it like: + +``` ruby +class UpdateSnippetService < BaseService + def execute + # ... + spam = SpamCheckService.new(params.slice!(:request, :api, :recaptcha_verified, :spam_log_id)) + + spam.check(snippet, current_user) + # ... + end +end +``` + +This way, all those instance variables are isolated in `SpamCheckService` +rather than whatever includes the module, and those modules which were also +included, making it much easier to track down any issues, +and reducing the chance of having name conflicts. + +### How to disable this cop + +Put the disabling comment right after your code in the same line: + +``` ruby +module M + def violating_method + @f + @g # rubocop:disable Gitlab/ModuleWithInstanceVariables + end +end +``` + +If there are multiple lines, you could also enable and disable for a section: + +``` ruby +module M + # rubocop:disable Gitlab/ModuleWithInstanceVariables + def violating_method + @f = 0 + @g = 1 + @h = 2 + end + # rubocop:enable Gitlab/ModuleWithInstanceVariables +end +``` + +Note that you need to enable it at some point, otherwise everything below +won't be checked. + +### Things we might need to ignore right now + +Because of the way Rails helpers and mailers work, we might not be able to +avoid the use of instance variables there. For those cases, we could ignore +them at the moment. At least we're not going to share those modules with +other random objects, so they're still somewhat isolated. + +### Instance variables in views + +They're bad because we can't easily tell who's using the instance variables +(from controller's point of view) and where we set them up (from partials' +point of view), making it extremely hard to track data dependency. + +We're trying to use something like this instead: + +``` haml += render 'projects/commits/commit', commit: commit, ref: ref, project: project +``` + +And in the partial: + +``` haml +- ref = local_assigns.fetch(:ref) +- commit = local_assigns.fetch(:commit) +- project = local_assigns.fetch(:project) +``` + +This way it's clearer where those values were coming from, and we gain the +benefit to have typo check over using instance variables. In the future, +we should also forbid the use of instance variables in partials. diff --git a/doc/development/performance.md b/doc/development/performance.md index 04419650b12..e7c5a6ca07a 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -37,7 +37,7 @@ graphs/dashboards. GitLab provides built-in tools to aid the process of improving performance: * [Sherlock](profiling.md#sherlock) -* [GitLab Performance Monitoring](../administration/monitoring/performance/introduction.md) +* [GitLab Performance Monitoring](../administration/monitoring/performance/index.md) * [Request Profiling](../administration/monitoring/performance/request_profiling.md) * [QueryRecoder](query_recorder.md) for preventing `N+1` regressions diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 8b7b015427f..edb8f372ea3 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -8,8 +8,8 @@ and effective _as well as_ fast. Here are some things to keep in mind regarding test performance: -- `double` and `spy` are faster than `FactoryGirl.build(...)` -- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`. +- `double` and `spy` are faster than `FactoryBot.build(...)` +- `FactoryBot.build(...)` and `.build_stubbed` are faster than `.create`. - Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, `spy`, or `double` will do. Database persistence is slow! - Don't mark a feature as requiring JavaScript (through `@javascript` in @@ -254,13 +254,13 @@ end ### Factories -GitLab uses [factory_girl] as a test fixture replacement. +GitLab uses [factory_bot] as a test fixture replacement. - Factory definitions live in `spec/factories/`, named using the pluralization of their corresponding model (`User` factories are defined in `users.rb`). - There should be only one top-level factory definition per file. -- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and - should) call `create(...)` instead of `FactoryGirl.create(...)`. +- FactoryBot methods are mixed in to all RSpec groups. This means you can (and + should) call `create(...)` instead of `FactoryBot.create(...)`. - Make use of [traits] to clean up definitions and usages. - When defining a factory, don't define attributes that are not required for the resulting record to pass validation. @@ -269,8 +269,8 @@ GitLab uses [factory_girl] as a test fixture replacement. - Factories don't have to be limited to `ActiveRecord` objects. [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). -[factory_girl]: https://github.com/thoughtbot/factory_girl -[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits +[factory_bot]: https://github.com/thoughtbot/factory_bot +[traits]: http://www.rubydoc.info/gems/factory_bot/file/GETTING_STARTED.md#Traits ### Fixtures diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 8045bbad7ba..65386f231a0 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -33,7 +33,7 @@ changes should be tested. ## [Testing best practices](best_practices.md) -Everything you should know about how to write good tests: RSpec, FactoryGirl, +Everything you should know about how to write good tests: RSpec, FactoryBot, system tests, parameterized tests etc. --- diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 48e04a40050..43a79ffcaa5 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,75 +1,14 @@ # Writing documentation - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - - **Technical Articles**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). + - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). -## Distinction between General Documentation and Technical Articles - -### General documentation - -General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. - -### Technical Articles - -Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. - -They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. - -A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. - -They live under `doc/articles/article-title/index.md`, and their images should be placed under `doc/articles/article-title/img/`. Find a list of existing [technical articles](../articles/index.md) here. - -#### Types of Technical Articles - -- **User guides**: technical content to guide regular users from point A to point B -- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B -- **Technical Overviews**: technical content describing features, solutions, and third-party integrations -- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives - -#### Understanding guides, tutorials, and technical overviews - -Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. - -A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. - -- Live example: "GitLab Pages from A to Z - [Part 1](../user/project/pages/getting_started_part_one.md) to [Part 4](../user/project/pages/getting_started_part_four.md)" - -A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. -It does not only describes steps 2 and 3, but also shows you how to accomplish them. - -- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) - -A **technical overview** is a description of what a certain feature is, and what it does, but does not walk -through the process of how to use it systematically. - -- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) - -#### Special format - -Every **Technical Article** contains, in the very beginning, a blockquote with the following information: - -- A reference to the **type of article** (user guide, admin guide, tech overview, tutorial) -- A reference to the **knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) -- A reference to the **author's name** and **GitLab.com handle** -- A reference of the **publication date** - -```md -> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Name Surname](https://gitlab.com/username) || -> **Publication date:** AAAA/MM/DD -``` - -#### Technical Articles - Writing Method - -Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. - ## Documentation style guidelines All the docs follow the same [styleguide](doc_styleguide.md). -### Contributing to docs +## Contributing to docs Whenever a feature is changed, updated, introduced, or deprecated, the merge request introducing these changes must be accompanied by the documentation @@ -118,13 +57,31 @@ and for every **major** feature present in Community Edition. Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. -## Testing +### Previewing locally -We try to treat documentation as code, thus have implemented some testing. +To preview your changes to documentation locally, please follow +this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). + +### Testing + +We treat documentation as code, thus have implemented some testing. Currently, the following tests are in place: 1. `docs lint`: Check that all internal (relative) links work correctly and - that all cURL examples in API docs use the full switches. + that all cURL examples in API docs use the full switches. It's recommended + to [check locally](#previewing-locally) before pushing to GitLab by executing the command + `bundle exec nanoc check internal_links` on your local + [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. +1. [`ee_compat_check`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): + When you submit a merge request to GitLab Community Edition (CE), + there is this additional job that runs against Enterprise Edition (EE) + and checks if your changes can apply cleanly to the EE codebase. + If that job fails, read the instructions in the job log for what to do next. + As CE is merged into EE once a day, it's important to avoid merge conflicts. + Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is + essential to avoid them. + +### Branch naming If your contribution contains **only** documentation changes, you can speed up the CI process by following some branch naming conventions. You have three @@ -139,17 +96,7 @@ choices: If your branch name matches any of the above, it will run only the docs tests. If it doesn't, the whole test suite will run (including docs). ---- - -When you submit a merge request to GitLab Community Edition (CE), there is an -additional job called `ee_compat_check` that runs against Enterprise -Edition (EE) and checks if your changes can apply cleanly to the EE codebase. -If that job fails, read the instructions in the job log for what to do next. -Contributors do not need to submit their changes to EE, GitLab Inc. employees -on the other hand need to make sure that their changes apply cleanly to both -CE and EE. - -## Previewing the changes live +### Previewing the changes live If you want to preview the doc changes of your merge request live, you can use the manual `review-docs-deploy` job in your merge request. You will need at @@ -164,7 +111,7 @@ You will need to push a branch to those repositories, it doesn't work for forks. TIP: **Tip:** If your branch contains only documentation changes, you can use -[special branch names](#testing) to avoid long running pipelines. +[special branch names](#branch-naming) to avoid long running pipelines. In the mini pipeline graph, you should see an `>>` icon. Clicking on it will reveal the `review-docs-deploy` job. Hit the play button for the job to start. @@ -209,12 +156,12 @@ working on. If you don't, the remote docs branch won't be removed either, and the server where the Review Apps are hosted will eventually be out of disk space. -### Behind the scenes +#### Technical aspects If you want to know the hot details, here's what's really happening: 1. You manually run the `review-docs-deploy` job in a CE/EE merge request. -1. The job runs the [`scirpts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) +1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) script with the `deploy` flag, which in turn: 1. Takes your branch name and applies the following: - The slug of the branch name is used to avoid special characters since @@ -243,3 +190,65 @@ The following GitLab features are used among others: - [Review Apps](../ci/review_apps/index.md) - [Artifacts](../ci/yaml/README.md#artifacts) - [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) + +## General Documentation vs Technical Articles + +### General documentation + +General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. + +### Technical Articles + +Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. + +They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. + +A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. + +They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/article-title/`. + +#### Types of Technical Articles + +- **User guides**: technical content to guide regular users from point A to point B +- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B +- **Technical Overviews**: technical content describing features, solutions, and third-party integrations +- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives + +#### Understanding guides, tutorials, and technical overviews + +Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. + +A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. + +- Live example: "GitLab Pages from A to Z - [Part 1](../user/project/pages/getting_started_part_one.md) to [Part 4](../user/project/pages/getting_started_part_four.md)" + +A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. +It does not only describes steps 2 and 3, but also shows you how to accomplish them. + +- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) + +A **technical overview** is a description of what a certain feature is, and what it does, but does not walk +through the process of how to use it systematically. + +- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) + +#### Special format + +Every **Technical Article** contains, in the very beginning, a blockquote with the following information: + +- A reference to the **type of article** (user guide, admin guide, tech overview, tutorial) +- A reference to the **knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) +- A reference to the **author's name** and **GitLab.com handle** +- A reference of the **publication date** + +```md +> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Name Surname](https://gitlab.com/username) || +> **Publication date:** AAAA-MM-DD +``` + +#### Technical Articles - Writing Method + +Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. + diff --git a/doc/install/installation.md b/doc/install/installation.md index 6c6e5db4cd9..56888b05609 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -80,7 +80,7 @@ Make sure you have the right version of Git installed # Install Git sudo apt-get install -y git-core - # Make sure Git is version 2.13.6 or higher + # Make sure Git is version 2.14.3 or higher git --version Is the system packaged Git too old? Remove it and compile from source. diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 0e20b8096e9..20087a981f9 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -229,16 +229,18 @@ In order to enable/disable an OmniAuth provider, go to Admin Area -> Settings -> ## Keep OmniAuth user profiles up to date You can enable profile syncing from selected OmniAuth providers and for all or for specific user information. - + +When authenticating using LDAP, the user's email is always synced. + ```ruby gitlab_rails['sync_profile_from_provider'] = ['twitter', 'google_oauth2'] gitlab_rails['sync_profile_attributes'] = ['name', 'email', 'location'] ``` - + **For installations from source** - + ```yaml omniauth: sync_profile_from_provider: ['twitter', 'google_oauth2'] - sync_profile_claims_from_provider: ['email', 'location'] - ```
\ No newline at end of file + sync_profile_attributes: ['email', 'location'] + ``` diff --git a/doc/monitoring/performance/introduction.md b/doc/monitoring/performance/introduction.md index ae88baa0c14..4d6f02b6547 100644 --- a/doc/monitoring/performance/introduction.md +++ b/doc/monitoring/performance/introduction.md @@ -1 +1 @@ -This document was moved to [administration/monitoring/performance/introduction](../../administration/monitoring/performance/introduction.md). +This document was moved to [administration/monitoring/performance/introduction](../../administration/monitoring/performance/index.md). diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index d100b431721..0b48596006d 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -19,6 +19,7 @@ project in an easy and automatic way: 1. [Auto Build](#auto-build) 1. [Auto Test](#auto-test) 1. [Auto Code Quality](#auto-code-quality) +1. [Auto SAST (Static Application Security Testing)](#auto-sast) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto Deploy](#auto-deploy) 1. [Auto Monitoring](#auto-monitoring) @@ -147,6 +148,10 @@ has a `.gitlab-ci.yml` or not: do that in a branch to test Auto DevOps before committing to `master`. NOTE: **Note:** +Starting with GitLab 10.3, when enabling Auto DevOps, a pipeline is +automatically run on the default branch. + +NOTE: **Note:** If you are a GitLab Administrator, you can enable Auto DevOps instance wide in **Admin Area > Settings > Continuous Integration and Deployment**. Doing that, all the projects that haven't explicitly set an option will have Auto DevOps @@ -198,6 +203,18 @@ out. In GitLab Enterprise Edition Starter, differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). +### Auto SAST + +> Introduced in [GitLab Enterprise Edition Ultimate][ee] 10.3. + +Static Application Security Testing (SAST) uses the +[gl-sast Docker image](https://gitlab.com/gitlab-org/gl-sast) to run static +analysis on the current code and checks for potential security issues. Once the +report is created, it's uploaded as an artifact which you can later download and +check out. + +Any security warnings are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). + ### Auto Review Apps NOTE: **Note:** @@ -536,3 +553,4 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/ [postgresql]: https://www.postgresql.org/ [Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml [GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md +[ee]: https://about.gitlab.com/gitlab-ee/ diff --git a/doc/user/discussions/img/commit_comment_mr_context.png b/doc/user/discussions/img/commit_comment_mr_context.png Binary files differnew file mode 100644 index 00000000000..b363e0035e8 --- /dev/null +++ b/doc/user/discussions/img/commit_comment_mr_context.png diff --git a/doc/user/discussions/img/commit_comment_mr_discussions_tab.png b/doc/user/discussions/img/commit_comment_mr_discussions_tab.png Binary files differnew file mode 100644 index 00000000000..2b06cdcc055 --- /dev/null +++ b/doc/user/discussions/img/commit_comment_mr_discussions_tab.png diff --git a/doc/user/discussions/img/merge_request_commits_tab.png b/doc/user/discussions/img/merge_request_commits_tab.png Binary files differnew file mode 100644 index 00000000000..41a3648f390 --- /dev/null +++ b/doc/user/discussions/img/merge_request_commits_tab.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 2206b2860f4..eacfe2baa27 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -32,6 +32,43 @@ hide discussions that are no longer relevant. Comments and discussions can be resolved by anyone with at least Developer access to the project or the author of the merge request. +### Commit discussions in the context of a merge request + +> [Introduced][ce-31847] in GitLab 10.3. + +For reviewers with commit-based workflow, it may be useful to add discussions to +specific commit diffs in the context of a merge request. These discussions will +persist through a commit ID change when: + +- force-pushing after a rebase +- amending a commit + +To create a commit diff discussion: + +1. Navigate to the merge request **Commits** tab. A list of commits that + constitute the merge request will be shown. + +  + +1. Navigate to a specific commit, click on the **Changes** tab (where you + will only be presented diffs from the selected commit), and leave a comment. + +  + +1. Any discussions created this way will be shown in the merge request's + **Discussions** tab and are resolvable. + +  + +Discussions created this way will only appear in the original merge request +and not when navigating to that commit under your project's +**Repository > Commits** page. + +TIP: **Tip:** +When a link of a commit reference is found in a discussion inside a merge +request, it will be automatically converted to a link in the context of the +current merge request. + ### Jumping between unresolved discussions When a merge request has a large number of comments it can be difficult to track @@ -133,6 +170,15 @@ From now on, any discussions on a diff will be resolved by default if a push makes that diff section outdated. Discussions on lines that don't change and top-level resolvable discussions are not automatically resolved. +## Commit discussions + +You can add comments and discussion threads to a particular commit under your +project's **Repository > Commits**. + +CAUTION: **Attention:** +Discussions created this way will be lost if the commit ID changes after a +force push. + ## Threaded discussions > [Introduced][ce-7527] in GitLab 9.1. @@ -229,6 +275,7 @@ edit existing comments. Non-team members are restricted from adding or editing c [ce-14053]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14053 [ce-14061]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14061 [ce-14531]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14531 +[ce-31847]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31847 [resolve-discussion-button]: img/resolve_discussion_button.png [resolve-comment-button]: img/resolve_comment_button.png [discussion-view]: img/discussion_view.png diff --git a/doc/user/markdown.md b/doc/user/markdown.md index a671c92640a..552abac747b 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -195,12 +195,23 @@ With inline diffs tags you can display {+ additions +} or [- deletions -]. The wrapping tags can be either curly braces or square brackets [+ additions +] or {- deletions -}. +Examples: + +``` +- {+ additions +} +- [+ additions +] +- {- deletions -} +- [- deletions -] +``` + However the wrapping tags cannot be mixed as such: +``` - {+ additions +] - [+ additions +} - {- deletions -] - [- deletions -} +``` ### Emoji diff --git a/doc/user/project/merge_requests/img/create_from_email.png b/doc/user/project/merge_requests/img/create_from_email.png Binary files differnew file mode 100644 index 00000000000..71eb4bf267d --- /dev/null +++ b/doc/user/project/merge_requests/img/create_from_email.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index b5c3f74a113..7037d7f5989 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -27,7 +27,7 @@ With GitLab merge requests, you can: - [Resolve merge conflicts from the UI](#resolve-conflicts) - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch -- [Create new merge requests by email](#create_by_email) +- [Create new merge requests by email](#create-new-merge-requests-by-email) With **[GitLab Enterprise Edition][ee]**, you can also: @@ -139,7 +139,12 @@ address. The address can be obtained on the merge requests page by clicking on a **Email a new merge request to this project** button. The subject will be used as the source branch name for the new merge request and the target branch will be the default branch for the project. The message body (if not empty) -will be used as the merge request description. +will be used as the merge request description. You need +["Reply by email"](../../../administration/reply_by_email.md) enabled to use +this feature. If it's not enabled to your instance, you may ask your GitLab +administrator to do so. + + ## Revert changes |
