diff options
| author | Filipa Lacerda <filipa@gitlab.com> | 2018-05-04 14:58:47 +0100 |
|---|---|---|
| committer | Filipa Lacerda <filipa@gitlab.com> | 2018-05-04 14:58:47 +0100 |
| commit | 1983356d647290fe38ca21bbbca43fe2d6292913 (patch) | |
| tree | d07fba5693e239993dfc6d1f724b2103f90a3fa6 /doc | |
| parent | 703f45632292e7fc45359d0144cd616725bf9b0d (diff) | |
| parent | 4bf47cd76fd69a26b7b2b4ac029f088ec5493712 (diff) | |
| download | gitlab-ce-1983356d647290fe38ca21bbbca43fe2d6292913.tar.gz | |
Merge branch 'master' into 44427-state-management-with-vuext
* master: (1063 commits)
Replace commits spinach tests with RSpec analog
Update repository.rb
Add note about rebase/squash duplication in Gitaly
Resolve "Reconcile project templates with Auto DevOps"
Move import project pane to a separate partial
Inform the user when there are no project import options available
Clarify location of Vue templates
Make add_index_to_namespaces_runners_token migration reversible
Fix lambda arguments in Grape entities
Update grape-entity 0.6.0 -> 0.7.1
Fix constants in backfill_runner_type_for_ci_runners_post_migrate.rb
Use limited_counter_with_delimiter in the admin user list tabs
Remove a warning from spec/features/admin/admin_users_spec.rb
Use smallint for runner_type since its an enum
Dont remove duplicates in Runner.owned_or_shared since its not necessary
Change the docs license to CC BY-SA
Remove unnecessary disable transaction in add_ci_runner_namespaces
Split migration to add and index namespaces.runners_token
Output some useful information when running the rails console
Revert "Use factory in specs for ProjectCiCdSettings"
...
Diffstat (limited to 'doc')
149 files changed, 2628 insertions, 1002 deletions
diff --git a/doc/README.md b/doc/README.md index 604f7244a34..a2e152ce383 100644 --- a/doc/README.md +++ b/doc/README.md @@ -15,8 +15,8 @@ To understand what features you have access to, check the [GitLab subscriptions] | General documentation | GitLab CI/CD docs | | :----- | :----- | -| [User documentation](user/index.md) | [GitLab CI/CD](ci/README.md) | -| [Administrator documentation](administration/index.md) | [GitLab CI/CD quick start guide](ci/quick_start/README.md) | +| [User documentation](user/index.md) | [GitLab CI/CD quick start guide](ci/quick_start/README.md) | +| [Administrator documentation](administration/index.md) | [GitLab CI/CD examples](ci/examples/README.md) | | [Contributor documentation](#contributor-documentation) | [Configuring `.gitlab-ci.yml`](ci/yaml/README.md) | | [Getting started with GitLab](#getting-started-with-gitlab) | [Using Docker images](ci/docker/using_docker_images.md) | | [API](api/README.md) | [Auto DevOps](topics/autodevops/index.md) | @@ -80,6 +80,7 @@ on projects and code. - [Search through GitLab](user/search/index.md): Search for issues, merge requests, projects, groups, todos, and issues in Issue Boards. - [Snippets](user/snippets.md): Snippets allow you to create little bits of code. - [Wikis](user/project/wiki/index.md): Enhance your repository documentation with built-in wikis. +- [Web IDE](user/project/web_ide/index.md) #### Repositories @@ -89,6 +90,7 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i - [Create a file](user/project/repository/web_editor.md#create-a-file) - [Upload a file](user/project/repository/web_editor.md#upload-a-file) - [File templates](user/project/repository/web_editor.md#template-dropdowns) + - [Jupyter Notebook files](user/project/repository/index.md#jupyter-notebook-files) - [Create a directory](user/project/repository/web_editor.md#create-a-directory) - [Start a merge request](user/project/repository/web_editor.md#tips) (when committing via UI) - [Branches](user/project/repository/branches/index.md) @@ -99,6 +101,14 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i - [Commits](user/project/repository/index.md#commits) - [Signing commits](user/project/repository/gpg_signed_commits/index.md): use GPG to sign your commits. +#### Merge Requests + +- [Merge Requests](user/project/merge_requests/index.md) + - [Work In Progress "WIP" Merge Requests](user/project/merge_requests/work_in_progress_merge_requests.md) + - [Merge Request discussion resolution](user/discussions/index.md#moving-a-single-discussion-to-a-new-issue): Resolve discussions, move discussions in a merge request to an issue, only allow merge requests to be merged if all discussions are resolved. + - [Checkout merge requests locally](user/project/merge_requests/index.md#checkout-merge-requests-locally) + - [Cherry-pick](user/project/merge_requests/cherry_pick_changes.md) + #### Integrations - [Project Services](user/project/integrations/project_services.md): Integrate a project with external services, such as CI and chat. @@ -112,18 +122,16 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i ### Verify -Spot errors sooner and shorten feedback cycles with built-in code review, code testing, -Code Quality, and Review Apps. Customize your approval workflow controls, automatically -test the quality of your code, and spin up a staging environment for every code change. -GitLab Continuous Integration is the most popular next generation testing system that -auto scales to run your tests faster. +Spot errors sooner, improve security and shorten feedback cycles with built-in +static code analysis, code testing, code quality, dependency checking and review +apps. Customize your approval workflow controls, automatically test the quality +of your code, and spin up a staging environment for every code change. GitLab +Continuous Integration is the most popular next generation testing system that +scales to run your tests faster. -- [Merge Requests](user/project/merge_requests/index.md) - - [Work In Progress Merge Requests](user/project/merge_requests/work_in_progress_merge_requests.md) - - [Merge Request discussion resolution](user/discussions/index.md#moving-a-single-discussion-to-a-new-issue): Resolve discussions, move discussions in a merge request to an issue, only allow merge requests to be merged if all discussions are resolved. - - [Checkout merge requests locally](user/project/merge_requests/index.md#checkout-merge-requests-locally) - - [Cherry-pick](user/project/merge_requests/cherry_pick_changes.md) +- [GitLab CI/CD](ci/README.md): Explore the features and capabilities of Continuous Integration, Continuous Delivery, and Continuous Deployment with GitLab. - [Review Apps](ci/review_apps/index.md): Preview changes to your app right from a merge request. +- [Pipeline Graphs](ci/pipelines.md#pipeline-graphs) ### Package @@ -131,7 +139,6 @@ GitLab Container Registry gives you the enhanced security and access controls of custom Docker images without 3rd party add-ons. Easily upload and download images from GitLab CI/CD with full Git repository management integration. -- [GitLab CI/CD](ci/README.md): Explore the features and capabilities of Continuous Integration, Continuous Delivery, and Continuous Deployment with GitLab. - [GitLab Container Registry](user/project/container_registry.md): Learn how to use GitLab's built-in Container Registry. ### Release @@ -140,9 +147,11 @@ Spend less time configuring your tools, and more time creating. Whether you’re deploying to one server or thousands, build, test, and release your code confidently and securely with GitLab’s built-in Continuous Delivery and Deployment. -- [GitLab Pages](user/project/pages/index.md): Build, test, and deploy a static site directly from GitLab. - [Auto Deploy](topics/autodevops/index.md#auto-deploy): Configure GitLab CI for the deployment of your application. - [Environments and deployments](ci/environments.md): With environments, you can control the continuous deployment of your software within GitLab. +- [GitLab Pages](user/project/pages/index.md): Build, test, and deploy a static site directly from GitLab. +- [Scheduled Pipelines](user/project/pipelines/schedules.md) +- [Protected Runners](ci/runners/README.md#protected-runners) ### Configure @@ -151,6 +160,9 @@ Auto Devops. Best practice templates get you started with minimal to zero configuration. Then customize everything from buildpacks to CI/CD. - [Auto DevOps](topics/autodevops/index.md) +- [Deployment of Helm, Ingress, and Prometheus on Kubernetes](user/project/clusters/index.md#installing-applications) +- [Protected secret variables](ci/variables/README.md#protected-secret-variables) +- [Easy creation of Kubernetes clusters on GKE](user/project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) ### Monitor @@ -159,8 +171,12 @@ applications are always responsive and available. GitLab collects and displays performance metrics for deployed apps using Prometheus so you can know in an instant how code changes impact your production environment. +- [GitLab Prometheus](administration/monitoring/prometheus/index.md): Configure the bundled Prometheus to collect various metrics from your GitLab instance. +- [Prometheus project integration](user/project/integrations/prometheus.md): Configure the Prometheus integration per project and monitor your CI/CD environments. +- [Prometheus metrics](user/project/integrations/prometheus_library/metrics.md): Let Prometheus collect metrics from various services, like Kubernetes, NGINX, NGINX ingress controller, HAProxy, and Amazon Cloud Watch. +- [GitLab Performance Monitoring](administration/monitoring/performance/index.md): Use InfluxDB and Grafana to monitor the performance of your GitLab instance (will be eventually replaced by Prometheus). +- [Health check](user/admin_area/monitoring/health_check.md): GitLab provides liveness and readiness probes to indicate service health and reachability to required services. - [GitLab Cycle Analytics](user/project/cycle_analytics.md): Cycle Analytics measures the time it takes to go from an [idea to production](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) for each project you have. -- [GitLab Performance Monitoring](administration/monitoring/performance/index.md) ## Getting started with GitLab @@ -179,7 +195,7 @@ instant how code changes impact your production environment. ### Git and GitLab - [Git](topics/git/index.md): Getting started with Git, branching strategies, Git LFS, advanced use. -- [Git cheatsheet](https://gitlab.com/gitlab-com/marketing/raw/master/design/print/git-cheatsheet/print-pdf/git-cheatsheet.pdf): Download a PDF describing the most used Git operations. +- [Git cheatsheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf): Download a PDF describing the most used Git operations. - [GitLab Flow](workflow/gitlab_flow.md): explore the best of Git with the GitLab Flow strategy. ## Administrator documentation diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md index b51e705ab52..8b00f52ffc1 100644 --- a/doc/administration/auth/jwt.md +++ b/doc/administration/auth/jwt.md @@ -50,7 +50,7 @@ JWT will provide you with a secret key for you to use. required_claims: ["name", "email"], info_map: { name: "name", email: "email" }, auth_url: 'https://example.com/', - valid_within: nil, + valid_within: null, } } ``` diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index d8928a7fe4c..ad8ffc46559 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -75,43 +75,33 @@ Notice several options that you should consider using: | `nobootwait` | Don't halt boot process waiting for this mount to become available | `lookupcache=positive` | Tells the NFS client to honor `positive` cache results but invalidates any `negative` cache results. Negative cache results cause problems with Git. Specifically, a `git push` can fail to register uniformly across all NFS clients. The negative cache causes the clients to 'remember' that the files did not exist previously. -## Mount locations +## A single NFS mount -When using default Omnibus configuration you will need to share 5 data locations -between all GitLab cluster nodes. No other locations should be shared. The -following are the 5 locations you need to mount: - -| Location | Description | Default configuration | -| -------- | ----------- | --------------------- | -| `/var/opt/gitlab/git-data` | Git repository data. This will account for a large portion of your data | `git_data_dirs({"default" => "/var/opt/gitlab/git-data"})` -| `/var/opt/gitlab/.ssh` | SSH `authorized_keys` file and keys used to import repositories from some other Git services | `user['home'] = '/var/opt/gitlab/'` -| `/var/opt/gitlab/gitlab-rails/uploads` | User uploaded attachments | `gitlab_rails['uploads_directory'] = '/var/opt/gitlab/gitlab-rails/uploads'` -| `/var/opt/gitlab/gitlab-rails/shared` | Build artifacts, GitLab Pages, LFS objects, temp files, etc. If you're using LFS this may also account for a large portion of your data | `gitlab_rails['shared_path'] = '/var/opt/gitlab/gitlab-rails/shared'` -| `/var/opt/gitlab/gitlab-ci/builds` | GitLab CI build traces | `gitlab_ci['builds_directory'] = '/var/opt/gitlab/gitlab-ci/builds'` +It's recommended to nest all gitlab data dirs within a mount, that allows automatic +restore of backups without manually moving existing data. -Other GitLab directories should not be shared between nodes. They contain -node-specific files and GitLab code that does not need to be shared. To ship -logs to a central location consider using remote syslog. GitLab Omnibus packages -provide configuration for [UDP log shipping][udp-log-shipping]. - -### Consolidating mount points - -If you don't want to configure 5-6 different NFS mount points, you have a few -alternative options. +``` +mountpoint +└── gitlab-data + ├── builds + ├── git-data + ├── home-git + ├── shared + └── uploads +``` -#### Change default file locations +To do so, we'll need to configure Omnibus with the paths to each directory nested +in the mount point as follows: -Omnibus allows you to configure the file locations. With custom configuration -you can specify just one main mountpoint and have all of these locations -as subdirectories. Mount `/gitlab-data` then use the following Omnibus +Mount `/gitlab-nfs` then use the following Omnibus configuration to move each data location to a subdirectory: ```ruby -git_data_dirs({"default" => "/gitlab-data/git-data"}) -user['home'] = '/gitlab-data/home' -gitlab_rails['uploads_directory'] = '/gitlab-data/uploads' -gitlab_rails['shared_path'] = '/gitlab-data/shared' -gitlab_ci['builds_directory'] = '/gitlab-data/builds' +git_data_dirs({"default" => "/gitlab-nfs/gitlab-data/git-data"}) +user['home'] = '/gitlab-nfs/gitlab-data/home' +gitlab_rails['uploads_directory'] = '/gitlab-nfs/gitlab-data/uploads' +gitlab_rails['shared_path'] = '/gitlab-nfs/gitlab-data/shared' +gitlab_ci['builds_directory'] = '/gitlab-nfs/gitlab-data/builds' ``` To move the `git` home directory, all GitLab services must be stopped. Run @@ -122,22 +112,52 @@ Run `sudo gitlab-ctl reconfigure` to start using the central location. Please be aware that if you had existing data you will need to manually copy/rsync it to these new locations and then restart GitLab. -#### Bind mounts +## Bind mounts + +Alternatively to changing the configuration in Omnibus, bind mounts can be used +to store the data on an NFS mount. Bind mounts provide a way to specify just one NFS mount and then bind the default GitLab data locations to the NFS mount. Start by defining your single NFS mount point as you normally would in `/etc/fstab`. Let's assume your -NFS mount point is `/gitlab-data`. Then, add the following bind mounts in +NFS mount point is `/gitlab-nfs`. Then, add the following bind mounts in `/etc/fstab`: ```bash -/gitlab-data/git-data /var/opt/gitlab/git-data none bind 0 0 -/gitlab-data/.ssh /var/opt/gitlab/.ssh none bind 0 0 -/gitlab-data/uploads /var/opt/gitlab/gitlab-rails/uploads none bind 0 0 -/gitlab-data/shared /var/opt/gitlab/gitlab-rails/shared none bind 0 0 -/gitlab-data/builds /var/opt/gitlab/gitlab-ci/builds none bind 0 0 +/gitlab-nfs/gitlab-data/git-data /var/opt/gitlab/git-data none bind 0 0 +/gitlab-nfs/gitlab-data/.ssh /var/opt/gitlab/.ssh none bind 0 0 +/gitlab-nfs/gitlab-data/uploads /var/opt/gitlab/gitlab-rails/uploads none bind 0 0 +/gitlab-nfs/gitlab-data/shared /var/opt/gitlab/gitlab-rails/shared none bind 0 0 +/gitlab-nfs/gitlab-data/builds /var/opt/gitlab/gitlab-ci/builds none bind 0 0 ``` +Using bind mounts will require manually making sure the data directories +are empty before attempting a restore. Read more about the +[restore prerequisites](../../raketasks/backup_restore.md). + +## Multiple NFS mounts + +When using default Omnibus configuration you will need to share 5 data locations +between all GitLab cluster nodes. No other locations should be shared. The +following are the 5 locations need to be shared: + +| Location | Description | Default configuration | +| -------- | ----------- | --------------------- | +| `/var/opt/gitlab/git-data` | Git repository data. This will account for a large portion of your data | `git_data_dirs({"default" => "/var/opt/gitlab/git-data"})` +| `/var/opt/gitlab/.ssh` | SSH `authorized_keys` file and keys used to import repositories from some other Git services | `user['home'] = '/var/opt/gitlab/'` +| `/var/opt/gitlab/gitlab-rails/uploads` | User uploaded attachments | `gitlab_rails['uploads_directory'] = '/var/opt/gitlab/gitlab-rails/uploads'` +| `/var/opt/gitlab/gitlab-rails/shared` | Build artifacts, GitLab Pages, LFS objects, temp files, etc. If you're using LFS this may also account for a large portion of your data | `gitlab_rails['shared_path'] = '/var/opt/gitlab/gitlab-rails/shared'` +| `/var/opt/gitlab/gitlab-ci/builds` | GitLab CI build traces | `gitlab_ci['builds_directory'] = '/var/opt/gitlab/gitlab-ci/builds'` + +Other GitLab directories should not be shared between nodes. They contain +node-specific files and GitLab code that does not need to be shared. To ship +logs to a central location consider using remote syslog. GitLab Omnibus packages +provide configuration for [UDP log shipping][udp-log-shipping]. + +Having multiple NFS mounts will require manually making sure the data directories +are empty before attempting a restore. Read more about the +[restore prerequisites](../../raketasks/backup_restore.md). + --- Read more on high-availability configuration: diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index fd2677996b1..031fb31ca4f 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -323,7 +323,7 @@ The prerequisites for a HA Redis setup are the following: # machines to connect to it. redis['port'] = 6379 - # The same password for Redeis authentication you set up for the master node. + # The same password for Redis authentication you set up for the master node. redis['password'] = 'redis-password-goes-here' # The IP of the master Redis node. @@ -650,6 +650,47 @@ gitlab_rails['redis_sentinels'] = [ Omnibus GitLab configures some things behind the curtains to make the sysadmins' lives easier. If you want to know what happens underneath keep reading. +### Running multiple Redis clusters + +GitLab supports running [separate Redis clusters for different persistent +classes](https://docs.gitlab.com/omnibus/settings/redis.html#running-with-multiple-redis-instances): +cache, queues, and shared_state. To make this work with Sentinel: + +1. Set the appropriate variable in `/etc/gitlab/gitlab.rb` for each instance you are using: + + ```ruby + gitlab_rails['redis_cache_instance'] = REDIS_CACHE_URL + gitlab_rails['redis_queues_instance'] = REDIS_QUEUES_URL + gitlab_rails['redis_shared_state_instance'] = REDIS_SHARED_STATE_URL + ``` + **Note**: Redis URLs should be in the format: `redis://:PASSWORD@SENTINEL_MASTER_NAME` + + 1. PASSWORD is the plaintext password for the Redis instance + 2. SENTINEL_MASTER_NAME is the Sentinel master name (e.g. `gitlab-redis-cache`) +1. Include an array of hashes with host/port combinations, such as the following: + + ```ruby + gitlab_rails['redis_cache_sentinels'] = [ + { host: REDIS_CACHE_SENTINEL_HOST, port: PORT1 }, + { host: REDIS_CACHE_SENTINEL_HOST2, port: PORT2 } + ] + gitlab_rails['redis_queues_sentinels'] = [ + { host: REDIS_QUEUES_SENTINEL_HOST, port: PORT1 }, + { host: REDIS_QUEUES_SENTINEL_HOST2, port: PORT2 } + ] + gitlab_rails['redis_shared_state_sentinels'] = [ + { host: SHARED_STATE_SENTINEL_HOST, port: PORT1 }, + { host: SHARED_STATE_SENTINEL_HOST2, port: PORT2 } + ] + ``` +1. Note that for each persistence class, GitLab will default to using the + configuration specified in `gitlab_rails['redis_sentinels']` unless + overriden by the settings above. +1. Be sure to include BOTH configuration options for each persistent classes. For example, + if you choose to configure a cache instance, you must specify both `gitlab_rails['redis_cache_instance']` + and `gitlab_rails['redis_cache_sentinels']` for GitLab to generate the proper configuration files. +1. Run `gitlab-ctl reconfigure` + ### Control running services In the previous example, we've used `redis_sentinel_role` and diff --git a/doc/administration/index.md b/doc/administration/index.md index 60a45426636..b472ca5b4d8 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -1,4 +1,4 @@ -# Administrator documentation +# Administrator documentation **[CORE ONLY]** Learn how to administer your GitLab instance (Community Edition and Enterprise Edition). @@ -39,6 +39,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [GitLab Pages configuration for GitLab source installations](pages/source.md): Enable and configure GitLab Pages on [source installations](../install/installation.md#installation-from-source). - [Environment variables](environment_variables.md): Supported environment variables that can be used to override their defaults values in order to configure GitLab. +- [Plugins](plugins.md): With custom plugins, GitLab administrators can introduce custom integrations without modifying GitLab's source code. #### Customizing GitLab's appearance @@ -85,7 +86,6 @@ created in snippets, wikis, and repos. - [Postfix for incoming email](reply_by_email_postfix_setup.md): Set up a basic Postfix mail server with IMAP authentication on Ubuntu for incoming emails. -server with IMAP authentication on Ubuntu, to be used with Reply by email. - [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. [reply by email]: reply_by_email.md diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index ac3a12930c3..77fe4d561a1 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -107,7 +107,8 @@ For source installations the following settings are nested under `artifacts:` an | Setting | Description | Default | |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | -| `remote_directory` | The bucket name where Artfacts will be stored| | +| `remote_directory` | The bucket name where Artifacts will be stored| | +| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. Currently only `Google` provider is supported | `false` | | `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | | `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | @@ -147,7 +148,7 @@ _The artifacts are stored by default in ``` NOTE: For GitLab 9.4+, if you are using AWS IAM profiles, be sure to omit the - AWS access key and secret acces key/value pairs. For example: + AWS access key and secret access key/value pairs. For example: ```ruby gitlab_rails['artifacts_object_store_connection'] = { diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index f495990d9a4..69600cad25c 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -46,7 +46,7 @@ In this experimental phase, only a few metrics are available: | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | filesystem_circuitbreaker_latency_seconds | Gauge | 9.5 | Time spent validating if a storage is accessible | -| filesystem_circuitbreaker | Gauge | 9.5 | Wether or not the circuit for a certain shard is broken or not | +| filesystem_circuitbreaker | Gauge | 9.5 | Whether or not the circuit for a certain shard is broken or not | | circuitbreaker_storage_check_duration_seconds | Histogram | 10.3 | Time a single storage probe took | ## Metrics shared directory diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index f43c89dad87..3d24812c66a 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -62,7 +62,14 @@ To change the address/port that Prometheus listens on: ``` Replace `localhost:9090` with the address/port you want Prometheus to - listen on. + listen on. If you would like to allow access to Prometheus to hosts other + than `localhost`, leave out the host, or use `0.0.0.0` to allow public access: + + ```ruby + prometheus['listen_address'] = ':9090' + # or + prometheus['listen_address'] = '0.0.0.0:9090' + ``` 1. Save the file and [reconfigure GitLab][reconfigure] for the changes to take effect diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index bd6c7bb07b5..89331238ce4 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -31,7 +31,7 @@ GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to check whether the user is authorized to access GitLab. -Add the following to your `sshd_config` file. This is usuaully located at +Add the following to your `sshd_config` file. This is usually located at `/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using Omnibus Docker: diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 00c631fdaae..9b3b1e48efd 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -119,11 +119,17 @@ The Pages daemon doesn't listen to the outside world. 1. Set the external URL for GitLab Pages in `/etc/gitlab/gitlab.rb`: - ```ruby + ```shell pages_external_url 'http://example.io' ``` 1. [Reconfigure GitLab][reconfigure] +1. Restart gitlab-pages by running the following command: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` + Watch the [video tutorial][video-admin] for this configuration. @@ -143,7 +149,7 @@ outside world. 1. Place the certificate and key inside `/etc/gitlab/ssl` 1. In `/etc/gitlab/gitlab.rb` specify the following configuration: - ```ruby + ```shell pages_external_url 'https://example.io' pages_nginx['redirect_http_to_https'] = true @@ -155,6 +161,11 @@ outside world. respectively. 1. [Reconfigure GitLab][reconfigure] +1. Restart gitlab-pages by running the following command: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` ## Advanced configuration @@ -180,7 +191,7 @@ world. Custom domains are supported, but no TLS. 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby + ```shell pages_external_url "http://example.io" nginx['listen_addresses'] = ['1.1.1.1'] pages_nginx['enable'] = false @@ -192,6 +203,11 @@ world. Custom domains are supported, but no TLS. listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] +1. Restart gitlab-pages by running the following command: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` ### Custom domains with TLS support @@ -210,7 +226,7 @@ world. Custom domains and TLS are supported. 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby + ```shell pages_external_url "https://example.io" nginx['listen_addresses'] = ['1.1.1.1'] pages_nginx['enable'] = false @@ -225,6 +241,11 @@ world. Custom domains and TLS are supported. listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] +1. Restart gitlab-pages by running the following command: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` ### Custom domain verification @@ -247,11 +268,16 @@ are stored. If you wish to store them in another location you must set it up in `/etc/gitlab/gitlab.rb`: - ```ruby + ```shell gitlab_rails['pages_path'] = "/mnt/storage/pages" ``` 1. [Reconfigure GitLab][reconfigure] +1. Restart gitlab-pages by running the following command: + + ```shell + sudo gitlab-ctl restart gitlab-pages + ``` ## Set maximum pages size diff --git a/doc/administration/plugins.md b/doc/administration/plugins.md index c91ac3012b9..4302667caf5 100644 --- a/doc/administration/plugins.md +++ b/doc/administration/plugins.md @@ -1,66 +1,85 @@ -# Plugins +# GitLab Plugin system -**Note:** Plugins must be configured on the filesystem of the GitLab -server. Only GitLab server administrators will be able to complete these tasks. -Please explore [system hooks] or [webhooks] as an option if you do not -have filesystem access. +> Introduced in GitLab 10.6. -Introduced in GitLab 10.6. +With custom plugins, GitLab administrators can introduce custom integrations +without modifying GitLab's source code. -A plugin will run on each event so it's up to you to filter events or projects within a plugin code. You can have as many plugins as you want. Each plugin will be triggered by GitLab asynchronously in case of an event. For a list of events please see [system hooks] documentation. +NOTE: **Note:** +Instead of writing and supporting your own plugin you can make changes +directly to the GitLab source code and contribute back upstream. This way we can +ensure functionality is preserved across versions and covered by tests. + +NOTE: **Note:** +Plugins must be configured on the filesystem of the GitLab server. Only GitLab +server administrators will be able to complete these tasks. Explore +[system hooks] or [webhooks] as an option if you do not have filesystem access. + +A plugin will run on each event so it's up to you to filter events or projects +within a plugin code. You can have as many plugins as you want. Each plugin will +be triggered by GitLab asynchronously in case of an event. For a list of events +see the [system hooks] documentation. ## Setup -Plugins must be placed directly into `plugins` directory, subdirectories will be ignored. -There is an `example` directory inside `plugins` where you can find some basic examples. +The plugins must be placed directly into the `plugins` directory, subdirectories +will be ignored. There is an +[`example` directory inside `plugins`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/plugins/examples) +where you can find some basic examples. Follow the steps below to set up a custom hook: -1. On the GitLab server, navigate to the project's plugin directory. +1. On the GitLab server, navigate to the plugin directory. For an installation from source the path is usually `/home/git/gitlab/plugins/`. For Omnibus installs the path is usually `/opt/gitlab/embedded/service/gitlab-rails/plugins`. -1. Inside the `plugins` directory, create a file with a name of your choice, but without spaces or special characters. + + For [highly available] configurations, your hook file should exist on each + application server. + +1. Inside the `plugins` directory, create a file with a name of your choice, + without spaces or special characters. 1. Make the hook file executable and make sure it's owned by the git user. -1. Write the code to make the plugin function as expected. Plugin can be - in any language. Ensure the 'shebang' at the top properly reflects the language - type. For example, if the script is in Ruby the shebang will probably be - `#!/usr/bin/env ruby`. -1. The data to the plugin will be provided as JSON on STDIN. It will be exactly same as one for [system hooks] +1. Write the code to make the plugin function as expected. That can be + in any language, and ensure the 'shebang' at the top properly reflects the + language type. For example, if the script is in Ruby the shebang will + probably be `#!/usr/bin/env ruby`. +1. The data to the plugin will be provided as JSON on STDIN. It will be exactly + same as for [system hooks] -That's it! Assuming the plugin code is properly implemented the hook will fire -as appropriate. Plugins file list is updated for each event. There is no need to restart GitLab to apply a new plugin. +That's it! Assuming the plugin code is properly implemented, the hook will fire +as appropriate. The plugins file list is updated for each event, there is no +need to restart GitLab to apply a new plugin. If a plugin executes with non-zero exit code or GitLab fails to execute it, a message will be logged to `plugin.log`. ## Validation -Writing own plugin can be tricky and its easier if you can check it without altering the system. -We provided a rake task you can use with staging environment to test your plugin before using it in production. -The rake task will use a sample data and execute each of plugins. By output you should be able to determine if -system sees your plugin and if it was executed without errors. +Writing your own plugin can be tricky and it's easier if you can check it +without altering the system. A rake task is provided so that you can use it +in a staging environment to test your plugin before using it in production. +The rake task will use a sample data and execute each of plugin. The output +should be enough to determine if the system sees your plugin and if it was +executed without errors. ```bash # Omnibus installations sudo gitlab-rake plugins:validate # Installations from source +cd /home/git/gitlab bundle exec rake plugins:validate RAILS_ENV=production ``` -Example of output can be next: +Example of output: ``` --> bundle exec rake plugins:validate RAILS_ENV=production Validating plugins from /plugins directory * /home/git/gitlab/plugins/save_to_file.clj succeed (zero exit code) * /home/git/gitlab/plugins/save_to_file.rb failure (non-zero exit code) ``` -[hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks [system hooks]: ../system_hooks/system_hooks.md [webhooks]: ../user/project/integrations/webhooks.md -[5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073 -[93]: https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 - +[highly available]: ./high_availability/README.md
\ No newline at end of file diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index a82735cc72c..7f0bd8f04e3 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -65,6 +65,7 @@ For source installations the following settings are nested under `uploads:` and |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | | `remote_directory` | The bucket name where Uploads will be stored| | +| `direct_upload` | Set to true to enable direct upload of Uploads without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. This is beta option as it uses inefficient way of uploading data (via Unicorn). The accelerated uploads gonna be implemented in future releases | `false` | | `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | | `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | @@ -103,7 +104,7 @@ _The uploads are stored by default in ``` >**Note:** -If you are using AWS IAM profiles, be sure to omit the AWS access key and secret acces key/value pairs. +If you are using AWS IAM profiles, be sure to omit the AWS access key and secret access key/value pairs. ```ruby gitlab_rails['uploads_object_store_connection'] = { diff --git a/doc/api/README.md b/doc/api/README.md index ae4481b400e..e777fc63d2b 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -13,6 +13,7 @@ following locations: - [Broadcast Messages](broadcast_messages.md) - [Project-level Variables](project_level_variables.md) - [Group-level Variables](group_level_variables.md) +- [Code Snippets](snippets.md) - [Commits](commits.md) - [Custom Attributes](custom_attributes.md) - [Deployments](deployments.md) @@ -85,6 +86,29 @@ have been resolved to our satisfaction by the relicensing of the reference implementations under MIT, and the use of the OWF license for the GraphQL specification. +## Compatibility Guidelines + +The HTTP API is versioned using a single number, the current one being 4. This +number symbolises the same as the major version number as described by +[SemVer](https://semver.org/). This mean that backward incompatible changes +will require this version number to change. However, the minor version is +not explicit. This allows for a stable API endpoint, but also means new +features can be added to the API in the same version number. + +New features and bug fixes are released in tandem with a new GitLab, and apart +from incidental patch and security releases, are released on the 22nd each +month. Backward incompatible changes (e.g. endpoints removal, parameters +removal etc.), as well as removal of entire API versions are done in tandem +with a major point release of GitLab itself. All deprecations and changes +between two versions should be listed in the documentation. For the changes +between v3 and v4; please read the [v3 to v4 documentation](v3_to_v4.md) + +#### Current status + +Currently two API versions are available, v3 and v4. v3 is deprecated and +will soon be removed. Deletion is scheduled for +[GitLab 11.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/36819). + ## Basic usage API requests should be prefixed with `api` and the API version. The API version @@ -269,7 +293,7 @@ The following table gives an overview of how the API functions generally behave. | `GET` | Access one or more resources and return the result as JSON. | | `POST` | Return `201 Created` if the resource is successfully created and return the newly created resource as JSON. | | `GET` / `PUT` | Return `200 OK` if the resource is accessed or modified successfully. The (modified) result is returned as JSON. | -| `DELETE` | Returns `204 No Content` if the resuource was deleted successfully. | +| `DELETE` | Returns `204 No Content` if the resource was deleted successfully. | The following table shows the possible return codes for API requests. diff --git a/doc/api/commits.md b/doc/api/commits.md index db0a80d04d9..d1584cf64de 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -539,6 +539,8 @@ Example response: ## List Merge Requests associated with a commit +> [Introduced][ce-18004] in GitLab 10.7. + Get a list of Merge Requests related to the specified commit. ``` @@ -608,3 +610,4 @@ Example response: [ce-6096]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6096 "Multi-file commit" [ce-8047]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8047 [ce-15026]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15026 +[ce-18004]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18004 diff --git a/doc/api/discussions.md b/doc/api/discussions.md index c341b7f2009..65e2f9d6cd9 100644 --- a/doc/api/discussions.md +++ b/doc/api/discussions.md @@ -1,6 +1,6 @@ # Discussions API -Discussions are set of related notes on snippets or issues. +Discussions are set of related notes on snippets, issues, merge requests or commits. ## Issues @@ -61,7 +61,8 @@ GET /projects/:id/issues/:issue_iid/discussions "system": false, "noteable_id": 3, "noteable_type": "Issue", - "noteable_iid": null + "noteable_iid": null, + "resolvable": false } ] }, @@ -87,7 +88,8 @@ GET /projects/:id/issues/:issue_iid/discussions "system": false, "noteable_id": 3, "noteable_type": "Issue", - "noteable_iid": null + "noteable_iid": null, + "resolvable": false } ] } @@ -265,7 +267,8 @@ GET /projects/:id/snippets/:snippet_id/discussions "system": false, "noteable_id": 3, "noteable_type": "Snippet", - "noteable_id": null + "noteable_id": null, + "resolvable": false } ] }, @@ -291,7 +294,8 @@ GET /projects/:id/snippets/:snippet_id/discussions "system": false, "noteable_id": 3, "noteable_type": "Snippet", - "noteable_id": null + "noteable_id": null, + "resolvable": false } ] } @@ -409,3 +413,574 @@ Parameters: ```bash curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/636 ``` + +## Merge requests + +### List project merge request discussions + +Gets a list of all discussions for a single merge request. + +``` +GET /projects/:id/merge_requests/:merge_request_iid/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Merge request", + "noteable_iid": null, + "resolved": false, + "resolvable": true, + "resolved_by": null + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Merge request", + "noteable_iid": null, + "resolved": false, + "resolvable": true, + "resolved_by": null + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Merge request", + "noteable_iid": null, + "resolved": false, + "resolvable": true, + "resolved_by": null + } + ] + } +] +``` + +Diff comments contain also position: + +```json +[ + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": false, + "notes": [ + { + "id": 1128, + "type": DiffNote, + "body": "diff comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Merge request", + "noteable_iid": null, + "position": { + "base_sha": "b5d6e7b1613fca24d250fa8e5bc7bcc3dd6002ef", + "start_sha": "7c9c2ead8a320fb7ba0b4e234bd9529a2614e306", + "head_sha": "4803c71e6b1833ca72b8b26ef2ecd5adc8a38031", + "old_path": "package.json", + "new_path": "package.json", + "position_type": "text", + "old_line": 27, + "new_line": 27 + }, + "resolved": false, + "resolvable": true, + "resolved_by": null + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions +``` + +### Get single merge request discussion + +Returns a single discussion for a specific project merge request + +``` +GET /projects/:id/merge_requests/:merge_request_iid/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new merge request discussion + +Creates a new discussion to a single project merge request. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /projects/:id/merge_requests/:merge_request_iid/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | +| `position` | hash | no | Position when creating a diff note | +| `position[base_sha]` | string | yes | Base commit SHA in the source branch | +| `position[start_sha]` | string | yes | SHA referencing commit in target branch | +| `position[head_sha]` | string | yes | SHA referencing HEAD of this merge request | +| `position[position_type]` | string | yes | Type of the position reference', allowed values: 'text' or 'image' | +| `position[new_path]` | string | no | File path after change | +| `position[new_line]` | integer | no | Line number after change (for 'text' diff notes) | +| `position[old_path]` | string | no | File path before change | +| `position[old_line]` | integer | no | Line number before change (for 'text' diff notes) | +| `position[width]` | integer | no | Width of the image (for 'image' diff notes) | +| `position[height]` | integer | no | Height of the image (for 'image' diff notes) | +| `position[x]` | integer | no | X coordinate (for 'image' diff notes) | +| `position[y]` | integer | no | Y coordinate (for 'image' diff notes) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions?body=comment +``` + +### Resolve a merge request discussion + +Resolve/unresolve whole discussion of a merge request. + +``` +PUT /projects/:id/merge_requests/:merge_request_iid/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `discussion_id` | integer | yes | The ID of a discussion | +| `resolved` | boolean | yes | Resolve/unresolve the discussion | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7?resolved=true +``` + + +### Add note to existing merge request discussion + +Adds a new note to the discussion. + +``` +POST /projects/:id/merge_requests/:merge_request_iid/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify an existing merge request discussion note + +Modify or resolve an existing discussion note of a merge request. + +``` +PUT /projects/:id/merge_requests/:merge_request_iid/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | no | The content of a discussion (exactly one of `body` or `resolved` must be set | +| `resolved` | boolean | no | Resolve/unresolve the note (exactly one of `body` or `resolved` must be set | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +Resolving a note: + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?resolved=true +``` + +### Delete a merge request discussion note + +Deletes an existing discussion note of a merge request. + +``` +DELETE /projects/:id/merge_requests/:merge_request_iid/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions/636 +``` + +## Commits + +### List project commit discussions + +Gets a list of all discussions for a single commit. + +``` +GET /projects/:id/commits/:commit_id/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Commit", + "noteable_iid": null, + "resolvable": false + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Commit", + "noteable_iid": null, + "resolvable": false + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Commit", + "noteable_iid": null, + "resolvable": false + } + ] + } +] +``` + +Diff comments contain also position: + +```json +[ + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": false, + "notes": [ + { + "id": 1128, + "type": DiffNote, + "body": "diff comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Commit", + "noteable_iid": null, + "position": { + "base_sha": "b5d6e7b1613fca24d250fa8e5bc7bcc3dd6002ef", + "start_sha": "7c9c2ead8a320fb7ba0b4e234bd9529a2614e306", + "head_sha": "4803c71e6b1833ca72b8b26ef2ecd5adc8a38031", + "old_path": "package.json", + "new_path": "package.json", + "position_type": "text", + "old_line": 27, + "new_line": 27 + }, + "resolvable": false + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions +``` + +### Get single commit discussion + +Returns a single discussion for a specific project commit + +``` +GET /projects/:id/commits/:commit_id/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new commit discussion + +Creates a new discussion to a single project commit. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /projects/:id/commits/:commit_id/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | +| `position` | hash | no | Position when creating a diff note | +| `position[base_sha]` | string | yes | Base commit SHA in the source branch | +| `position[start_sha]` | string | yes | SHA referencing commit in target branch | +| `position[head_sha]` | string | yes | SHA referencing HEAD of this commit | +| `position[position_type]` | string | yes | Type of the position reference', allowed values: 'text' or 'image' | +| `position[new_path]` | string | no | File path after change | +| `position[new_line]` | integer | no | Line number after change | +| `position[old_path]` | string | no | File path before change | +| `position[old_line]` | integer | no | Line number before change | +| `position[width]` | integer | no | Width of the image (for 'image' diff notes) | +| `position[height]` | integer | no | Height of the image (for 'image' diff notes) | +| `position[x]` | integer | no | X coordinate (for 'image' diff notes) | +| `position[y]` | integer | no | Y coordinate (for 'image' diff notes) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions?body=comment +``` + +### Add note to existing commit discussion + +Adds a new note to the discussion. + +``` +POST /projects/:id/commits/:commit_id/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify an existing commit discussion note + +Modify or resolve an existing discussion note of a commit. + +``` +PUT /projects/:id/commits/:commit_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | no | The content of a note | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +Resolving a note: + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?resolved=true +``` + +### Delete a commit discussion note + +Deletes an existing discussion note of a commit. + +``` +DELETE /projects/:id/commits/:commit_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `commit_id` | integer | yes | The ID of a commit | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/commits/11/discussions/636 +``` diff --git a/doc/api/group_badges.md b/doc/api/group_badges.md index 3e0683f378d..f2353542a5c 100644 --- a/doc/api/group_badges.md +++ b/doc/api/group_badges.md @@ -1,5 +1,8 @@ # Group badges API +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17082) +in GitLab 10.6. + ## Placeholder tokens Badges support placeholders that will be replaced in real time in both the link and image URL. The allowed placeholders are: @@ -9,7 +12,7 @@ Badges support placeholders that will be replaced in real time in both the link - **%{default_branch}**: will be replaced by the project default branch. - **%{commit_sha}**: will be replaced by the last project's commit sha. -Because these enpoints aren't inside a project's context, the information used to replace the placeholders will be +Because these endpoints aren't inside a project's context, the information used to replace the placeholders will be from the first group's project by creation date. If the group hasn't got any project the original URL with the placeholders will be returned. ## List all badges of a group @@ -182,7 +185,7 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/a Example response: ```json -{ +{ "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", "image_url": "https://shields.io/my/badge", "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", diff --git a/doc/api/groups.md b/doc/api/groups.md index 1aed8aac64e..923fd662a5b 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -10,7 +10,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `skip_groups` | array of integers | no | Skip the group IDs passed | -| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users) | +| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users, `true` for admin) | | `search` | string | no | Return the list of authorized groups matching the search criteria | | `order_by` | string | no | Order groups by `name` or `path`. Default is `name` | | `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | @@ -94,7 +94,7 @@ Parameters: | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) of the parent group | | `skip_groups` | array of integers | no | Skip the group IDs passed | -| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users) | +| `all_available` | boolean | no | Show all the groups you have access to (defaults to `false` for authenticated users, `true` for admin) | | `search` | string | no | Return the list of authorized groups matching the search criteria | | `order_by` | string | no | Order groups by `name` or `path`. Default is `name` | | `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | diff --git a/doc/api/notes.md b/doc/api/notes.md index aa38d22845c..d29c5b94915 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -39,7 +39,8 @@ GET /projects/:id/issues/:issue_iid/notes?sort=asc&order_by=updated_at "system": true, "noteable_id": 377, "noteable_type": "Issue", - "noteable_iid": 377 + "noteable_iid": 377, + "resolvable": false }, { "id": 305, @@ -58,7 +59,8 @@ GET /projects/:id/issues/:issue_iid/notes?sort=asc&order_by=updated_at "system": true, "noteable_id": 121, "noteable_type": "Issue", - "noteable_iid": 121 + "noteable_iid": 121, + "resolvable": false } ] ``` @@ -314,7 +316,8 @@ Parameters: "system": false, "noteable_id": 2, "noteable_type": "MergeRequest", - "noteable_iid": 2 + "noteable_iid": 2, + "resolvable": false } ``` diff --git a/doc/api/notification_settings.md b/doc/api/notification_settings.md index f05ae647577..682b90361bd 100644 --- a/doc/api/notification_settings.md +++ b/doc/api/notification_settings.md @@ -23,6 +23,7 @@ new_issue reopen_issue close_issue reassign_issue +issue_due new_merge_request push_to_merge_request reopen_merge_request @@ -75,6 +76,7 @@ curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab | `reopen_issue` | boolean | no | Enable/disable this notification | | `close_issue` | boolean | no | Enable/disable this notification | | `reassign_issue` | boolean | no | Enable/disable this notification | +| `issue_due` | boolean | no | Enable/disable this notification | | `new_merge_request` | boolean | no | Enable/disable this notification | | `push_to_merge_request` | boolean | no | Enable/disable this notification | | `reopen_merge_request` | boolean | no | Enable/disable this notification | @@ -142,6 +144,7 @@ curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab | `reopen_issue` | boolean | no | Enable/disable this notification | | `close_issue` | boolean | no | Enable/disable this notification | | `reassign_issue` | boolean | no | Enable/disable this notification | +| `issue_due` | boolean | no | Enable/disable this notification | | `new_merge_request` | boolean | no | Enable/disable this notification | | `push_to_merge_request` | boolean | no | Enable/disable this notification | | `reopen_merge_request` | boolean | no | Enable/disable this notification | @@ -166,6 +169,7 @@ Example responses: "reopen_issue": false, "close_issue": false, "reassign_issue": false, + "issue_due": false, "new_merge_request": false, "push_to_merge_request": false, "reopen_merge_request": false, diff --git a/doc/api/pipeline_schedules.md b/doc/api/pipeline_schedules.md index c28f48e5fc6..137f1fdddec 100644 --- a/doc/api/pipeline_schedules.md +++ b/doc/api/pipeline_schedules.md @@ -108,7 +108,7 @@ POST /projects/:id/pipeline_schedules | `description` | string | yes | The description of pipeline schedule | | `ref` | string | yes | The branch/tag name will be triggered | | `cron ` | string | yes | The cron (e.g. `0 1 * * *`) ([Cron syntax](https://en.wikipedia.org/wiki/Cron)) | -| `cron_timezone ` | string | no | The timezone supproted by `ActiveSupport::TimeZone` (e.g. `Pacific Time (US & Canada)`) (default: `'UTC'`) | +| `cron_timezone ` | string | no | The timezone supported by `ActiveSupport::TimeZone` (e.g. `Pacific Time (US & Canada)`) (default: `'UTC'`) | | `active ` | boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule will deactivated initially (default: `true`) | ```sh @@ -153,7 +153,7 @@ PUT /projects/:id/pipeline_schedules/:pipeline_schedule_id | `description` | string | no | The description of pipeline schedule | | `ref` | string | no | The branch/tag name will be triggered | | `cron ` | string | no | The cron (e.g. `0 1 * * *`) ([Cron syntax](https://en.wikipedia.org/wiki/Cron)) | -| `cron_timezone ` | string | no | The timezone supproted by `ActiveSupport::TimeZone` (e.g. `Pacific Time (US & Canada)`) or `TZInfo::Timezone` (e.g. `America/Los_Angeles`) | +| `cron_timezone ` | string | no | The timezone supported by `ActiveSupport::TimeZone` (e.g. `Pacific Time (US & Canada)`) or `TZInfo::Timezone` (e.g. `America/Los_Angeles`) | | `active ` | boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule will deactivated initially. | ```sh diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index a6631cab8c3..899f5da6647 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -14,6 +14,7 @@ GET /projects/:id/pipelines | `scope` | string | no | The scope of pipelines, one of: `running`, `pending`, `finished`, `branches`, `tags` | | `status` | string | no | The status of pipelines, one of: `running`, `pending`, `success`, `failed`, `canceled`, `skipped` | | `ref` | string | no | The ref of pipelines | +| `sha` | string | no | The sha or pipelines | | `yaml_errors`| boolean | no | Returns pipelines with invalid configurations | | `name`| string | no | The name of the user who triggered pipelines | | `username`| string | no | The username of the user who triggered pipelines | diff --git a/doc/api/project_badges.md b/doc/api/project_badges.md index 3f6e348b5b4..94389273e9c 100644 --- a/doc/api/project_badges.md +++ b/doc/api/project_badges.md @@ -1,5 +1,8 @@ # Project badges API +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17082) +in GitLab 10.6. + ## Placeholder tokens Badges support placeholders that will be replaced in real time in both the link and image URL. The allowed placeholders are: @@ -179,7 +182,7 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/a Example response: ```json -{ +{ "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", "image_url": "https://shields.io/my/badge", "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index 5467187788a..085437c801a 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -111,10 +111,14 @@ POST /projects/import | `namespace` | integer/string | no | The ID or path of the namespace that the project will be imported to. Defaults to the current user's namespace | | `file` | string | yes | The file to be uploaded | | `path` | string | yes | Name and path for new project | +| `overwrite` | boolean | no | If there is a project with the same path the import will overwrite it. Default to false | +| `override_params` | Hash | no | Supports all fields defined in the [Project API](projects.md) | -To upload a file from your filesystem, use the `--form` argument. This causes +The override params passed will take precedence over all values defined inside the export file. + +To upload a file from your file system, use the `--form` argument. This causes cURL to post data using the header `Content-Type: multipart/form-data`. -The `file=` parameter must point to a file on your filesystem and be preceded +The `file=` parameter must point to a file on your file system and be preceded by `@`. For example: ```console diff --git a/doc/api/projects.md b/doc/api/projects.md index a0cb5aa0820..fe21dfe23f9 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -915,6 +915,29 @@ Example response: } ``` +## Languages + +Get languages used in a project with percentage value. + +``` +GET /projects/:id/languages +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/languages" +``` + +Example response: + +```json +{ + "Ruby": 66.69, + "JavaScript": 22.98, + "HTML": 7.91, + "CoffeeScript": 2.42 +} +``` + ## Archive a project Archives the project if the user is either admin or the project owner of this project. This action is @@ -1375,4 +1398,27 @@ Read more in the [Project Badges](project_badges.md) documentation. ## Issue and merge request description templates -The non-default [issue and merge request description templates](../user/project/description_templates.md) are managed inside the project's repository. So you can manage them via the API through the [Repositories API](repositories.md) and the [Repository Files API](repository_files.md).
\ No newline at end of file +The non-default [issue and merge request description templates](../user/project/description_templates.md) are managed inside the project's repository. So you can manage them via the API through the [Repositories API](repositories.md) and the [Repository Files API](repository_files.md). + +## Download snapshot of a git repository + +> Introduced in GitLab 10.7 + +This endpoint may only be accessed by an administrative user. + +Download a snapshot of the project (or wiki, if requested) git repository. This +snapshot is always in uncompressed [tar](https://en.wikipedia.org/wiki/Tar_(computing)) +format. + +If a repository is corrupted to the point where `git clone` does not work, the +snapshot may allow some of the data to be retrieved. + +``` +GET /projects/:id/snapshot +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `wiki` | boolean | no | Whether to download the wiki, rather than project, repository | + diff --git a/doc/api/repositories.md b/doc/api/repositories.md index 96609cd530f..5aff255c20a 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -183,7 +183,7 @@ GET /projects/:id/repository/contributors Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `order_by` (optional) - Return contributors ordered by `name`, `email`, or `commits` fields. If not given contributors are ordered by commit date. +- `order_by` (optional) - Return contributors ordered by `name`, `email`, or `commits` (orders by commit date) fields. Default is `commits` - `sort` (optional) - Return contributors sorted in `asc` or `desc` order. Default is `asc` Response: diff --git a/doc/api/tags.md b/doc/api/tags.md index fa25dc76452..4af096c3c0c 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -42,6 +42,7 @@ Parameters: "description": "Amazing release. Wow" }, "name": "v1.0.0", + "target": "2695effb5807a22ff3d138d593fd856244e155e7", "message": null } ] @@ -73,6 +74,7 @@ Example Response: { "name": "v5.0.0", "message": null, + "target": "60a8ff033665e1207714d6670fcd7b65304ec02f", "commit": { "id": "60a8ff033665e1207714d6670fcd7b65304ec02f", "short_id": "60a8ff03", @@ -132,12 +134,16 @@ Parameters: "description": "Amazing release. Wow" }, "name": "v1.0.0", + "target: "2695effb5807a22ff3d138d593fd856244e155e7", "message": null } ``` The message will be `null` when creating a lightweight tag otherwise it will contain the annotation. +The target will contain the tag objects ID when creating annotated tags, +otherwise it will contain the commit ID when creating lightweight tags. + In case of an error, status code `405` with an explaining error message is returned. diff --git a/doc/api/todos.md b/doc/api/todos.md index dd4c737b729..27e623007cc 100644 --- a/doc/api/todos.md +++ b/doc/api/todos.md @@ -15,7 +15,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `action` | string | no | The action to be filtered. Can be `assigned`, `mentioned`, `build_failed`, `marked`, or `approval_required`. | +| `action` | string | no | The action to be filtered. Can be `assigned`, `mentioned`, `build_failed`, `marked`, `approval_required`, `unmergeable` or `directly_addressed`. | | `author_id` | integer | no | The ID of an author | | `project_id` | integer | no | The ID of a project | | `state` | string | no | The state of the todo. Can be either `pending` or `done` | diff --git a/doc/api/users.md b/doc/api/users.md index a4447e32908..ca5afa04687 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -55,6 +55,7 @@ GET /users | --------- | ---- | -------- | ----------- | | `order_by` | string | no | Return projects ordered by `id`, `name`, `username`, `created_at`, or `updated_at` fields. Default is `id` | | `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `two_factor` | string | no | Filter users by Two-factor authentication. Filter values are `enabled` or `disabled`. By default it returns all users | ```json [ diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 183808641c0..07b144f6ddd 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -101,12 +101,12 @@ In order to do that, follow the steps: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:latest" \ + --docker-image "docker:stable" \ --docker-privileged ``` The above command will register a new Runner to use the special - `docker:latest` image which is provided by Docker. **Notice that it's using + `docker:stable` image which is provided by Docker. **Notice that it's using the `privileged` mode to start the build and service containers.** If you want to use [docker-in-docker] mode, you always have to use `privileged = true` in your Docker containers. @@ -120,7 +120,7 @@ In order to do that, follow the steps: executor = "docker" [runners.docker] tls_verify = false - image = "docker:latest" + image = "docker:stable" privileged = true disable_cache = false volumes = ["/cache"] @@ -132,7 +132,7 @@ In order to do that, follow the steps: `docker:dind` service): ```yaml - image: docker:latest + image: docker:stable # When using dind, it's wise to use the overlayfs driver for # improved performance. @@ -201,12 +201,12 @@ In order to do that, follow the steps: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:latest" \ + --docker-image "docker:stable" \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` The above command will register a new Runner to use the special - `docker:latest` image which is provided by Docker. **Notice that it's using + `docker:stable` image which is provided by Docker. **Notice that it's using the Docker daemon of the Runner itself, and any containers spawned by docker commands will be siblings of the Runner rather than children of the runner.** This may have complications and limitations that are unsuitable for your workflow. @@ -220,7 +220,7 @@ In order to do that, follow the steps: executor = "docker" [runners.docker] tls_verify = false - image = "docker:latest" + image = "docker:stable" privileged = false disable_cache = false volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] @@ -232,7 +232,7 @@ In order to do that, follow the steps: include the `docker:dind` service as when using the Docker in Docker executor): ```yaml - image: docker:latest + image: docker:stable before_script: - docker info @@ -286,7 +286,7 @@ any image that's used with the `--cache-from` argument must first be pulled Here's a simple `.gitlab-ci.yml` file showing how Docker caching can be utilized: ```yaml -image: docker:latest +image: docker:stable services: - docker:dind @@ -388,7 +388,7 @@ could look like: ```yaml build: - image: docker:latest + image: docker:stable services: - docker:dind stage: build @@ -434,7 +434,7 @@ when needed. Changes to `master` also get tagged as `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:latest +image: docker:stable services: - docker:dind diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index bc5d3840368..7c0f837ea9c 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -86,7 +86,7 @@ services](#accessing-the-services). ### How the health check of services works Services are designed to provide additional functionality which is **network accessible**. -It may be a database like MySQL, or Redis, and even `docker:dind` which +It may be a database like MySQL, or Redis, and even `docker:stable-dind` which allows you to use Docker in Docker. It can be practically anything that is required for the CI/CD job to proceed and is accessed by network. diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 58c4a71cef9..517e25f00f7 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -247,10 +247,21 @@ declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is the basis of [Review apps](review_apps/index.md). >**Note:** -The `name` and `url` parameters can use any of the defined CI variables, +The `name` and `url` parameters can use most of the defined CI variables, including predefined, secure variables and `.gitlab-ci.yml` -[`variables`](yaml/README.md#variables). -You however cannot use variables defined under `script` or on the Runner's side. +[`variables`](yaml/README.md#variables). You however cannot use variables +defined under `script` or on the Runner's side. There are other variables that +are unsupported in environment name context: +- `CI_JOB_ID` +- `CI_JOB_TOKEN` +- `CI_BUILD_ID` +- `CI_BUILD_TOKEN` +- `CI_REGISTRY_USER` +- `CI_REGISTRY_PASSWORD` +- `CI_REPOSITORY_URL` +- `CI_ENVIRONMENT_URL` +- `CI_DEPLOY_USER` +- `CI_DEPLOY_PASSWORD` GitLab Runner exposes various [environment variables][variables] when a job runs, and as such, you can use them as environment names. Let's add another job in diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 691370d7195..0dab07a7f80 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -17,7 +17,7 @@ performance: variables: URL: https://example.com services: - - docker:dind + - docker:stable-dind script: - mkdir gitlab-exporter - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js @@ -94,7 +94,7 @@ performance: stage: performance image: docker:git services: - - docker:dind + - docker:stable-dind dependencies: - review script: diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 92317c77427..d1aa783cc9c 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -17,7 +17,11 @@ codequality: - docker:stable-dind script: - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') - - docker run --env SOURCE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code + - docker run + --env SOURCE_CODE="$PWD" + --volume "$PWD":/code + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code artifacts: paths: [codeclimate.json] ``` diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index c58efc7392a..eb76521cc02 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -30,6 +30,10 @@ sast:container: - mv clair-scanner_linux_amd64 clair-scanner - chmod +x clair-scanner - touch clair-whitelist.yml + - while( ! wget -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; done + - retries=0 + - echo "Waiting for clair daemon to start" + - while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-sast-container-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true artifacts: paths: [gl-sast-container-report.json] diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md index 8df223ee560..a8720f0b7ea 100644 --- a/doc/ci/examples/dast.md +++ b/doc/ci/examples/dast.md @@ -42,9 +42,9 @@ dast: allow_failure: true script: - mkdir /zap/wrk/ - - /zap/zap-baseline.py -J gl-dast-report.json -t $website \ - --auth-url $login_url \ - --auth-username "john.doe@example.com" \ + - /zap/zap-baseline.py -J gl-dast-report.json -t $website + --auth-url $login_url + --auth-username "john.doe@example.com" --auth-password "john-doe-password" || true - cp /zap/wrk/gl-dast-report.json . artifacts: diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index bfc8558a580..3d21c0cc306 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -509,7 +509,7 @@ and unit tests, all running and deployed at every push to master - with shocking Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit, you can see the changes live on your game. -Setting up Continous Integration and Continuous Deployment from the start with Dark Nova enables +Setting up Continuous Integration and Continuous Deployment from the start with Dark Nova enables rapid but stable development. We can easily test changes in a separate [environment](../../../ci/environments.md#introduction-to-environments-and-deployments), or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing and tedious, but having faith in a stable deployment with GitLab CI/CD allows diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index 7f6519fd38e..a2de0408797 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -30,7 +30,7 @@ and GitLab UI._ Many components and concepts are similar to Ruby on Rails or Python's Django. High developer productivity and high application performance are only a few advantages on learning how to use it. -Working on the MVC pattern, it's was designed to be modular and flexible. Easy to mantain a growing +Working on the MVC pattern, it's was designed to be modular and flexible. Easy to maintain a growing app is a plus. Phoenix can run in any OS where Erlang is supported: @@ -48,7 +48,7 @@ Check the [Phoenix learning guide][phoenix-learning-guide] for more information. ### What is Elixir? [Elixir][elixir-site] is a dynamic, functional language created to use all the maturity of Erlang -(30 years old!) in these days, in an easy way. It has similarities with Ruby, specially on sintax, +(30 years old!) in these days, in an easy way. It has similarities with Ruby, specially on syntax, so Ruby developers are quite excited with the rapid growing of Elixir. A full-stack Ruby developer can learn how to use Elixir and Phoenix in just a few weeks! @@ -162,7 +162,7 @@ productive, because every time we, or our co-workers push any code, GitLab CI/CD test the changes, telling us in realtime if anything goes wrong. Certainly, when our application starts to grow, we'll need more developers working on the same -project and this process of building and testing can easely become a mess without proper management. +project and this process of building and testing can easily become a mess without proper management. That's also why GitLab CI/CD is so important to our application. Every time someone pushes its code to GitLab, we'll quickly know if their changes broke something or not. We don't need to stop everything we're doing to test manually and locally every change our team does. @@ -237,7 +237,7 @@ Finished in 0.7 seconds Randomized with seed 610000 ``` -Our test was successfull. It's time to push our files to GitLab. +Our test was successful. It's time to push our files to GitLab. ## Configuring CI/CD Pipeline @@ -302,7 +302,7 @@ template** and select **Elixir**: ``` It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our - database with the login information provided earlier. More important is to respect the identation, + database with the login information provided earlier. More important is to respect the indentation, to avoid syntax errors when running the build. - And finally, we'll let `mix` session intact. @@ -333,7 +333,7 @@ mix: - mix test ``` -For safety, we can check if we get any syntax errors before submiting this file to GitLab. Copy the +For safety, we can check if we get any syntax errors before submitting this file to GitLab. Copy the contents of `.gitlab-ci.yml` and paste it on [GitLab CI/CD Lint tool][ci-lint]. Please note that this link will only work for logged in users. @@ -384,7 +384,7 @@ working properly. When we have a growing application with many developers working on it, or when we have an open source project being watched and contributed by the community, it is really important to have our -code permanently working. GitLab CI/CD is a time saving powerfull tool to help us mantain our code +code permanently working. GitLab CI/CD is a time saving powerful tool to help us maintain our code organized and working. As we could see in this post, GitLab CI/CD is really really easy to configure and use. We have [many diff --git a/doc/ci/img/job_failure_reason.png b/doc/ci/img/job_failure_reason.png Binary files differnew file mode 100644 index 00000000000..a60ce1fb21c --- /dev/null +++ b/doc/ci/img/job_failure_reason.png diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 301cccc80a3..b16cbc61d14 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -73,6 +73,23 @@ cancel the job, retry it, or erase the job trace.  +## Seeing the failure reason for jobs + +> [Introduced][ce-17782] in GitLab 10.7. + +When a pipeline fails or is allowed to fail, there are several places where you +can quickly check the reason it failed: + +- **In the pipeline graph** present on the pipeline detail view. +- **In the pipeline widgets** present in the merge requests and commit pages. +- **In the job views** present in the global and detailed views of a job. + +In any case, if you hover over the failed job you can see the reason it failed. + + + +From [GitLab 10.8][ce-17814] you can also see the reason it failed on the Job detail page. + ## Pipeline graphs > [Introduced][ce-5742] in GitLab 8.11. @@ -263,4 +280,6 @@ runners will not use regular runners, they must be tagged accordingly. [ce-6242]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6242 [ce-7931]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7931 [ce-9760]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9760 +[ce-17782]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17782 +[ce-17814]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17814 [regexp]: https://gitlab.com/gitlab-org/gitlab-ce/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99 diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 60dc2ef9ac5..821413900fd 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -298,6 +298,28 @@ Mentioned briefly earlier, but the following things of Runners can be exploited. We're always looking for contributions that can mitigate these [Security Considerations](https://docs.gitlab.com/runner/security/). +### Resetting the registration token for a Project + +If you think that registration token for a Project was revealed, you should +reset them. It's recommended because such token can be used to register another +Runner to thi Project. It may be next used to obtain the values of secret +variables or clone the project code, that normally may be unavailable for the +attacker. + +To reset the token: + +1. Go to **Settings > CI/CD** for a specified Project +1. Expand the **General pipelines settings** section +1. Find the **Runner token** form field and click the **Reveal value** button +1. Delete the value and save the form +1. After the page is refreshed, expand the **Runners settings** section + and check the registration token - it should be changed + +From now on the old token is not valid anymore and will not allow to register +a new Runner to the project. If you are using any tools to provision and +register new Runners, you should now update the token that is used to the +new value. + ## Determining the IP address of a Runner > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17286) in GitLab 10.6. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 9f268f47e6f..38a988f4507 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -61,7 +61,7 @@ future GitLab releases.** | **CI_RUNNER_EXECUTABLE_ARCH** | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | | **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | | **CI_PIPELINE_TRIGGERED** | all | all | The flag to indicate that job was [triggered] | -| **CI_PIPELINE_SOURCE** | 10.0 | all | The source for this pipeline, one of: push, web, trigger, schedule, api, external. Pipelines created before 9.5 will have unknown as source | +| **CI_PIPELINE_SOURCE** | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, and `pipeline`. For pipelines created before GitLab 9.5, this will show as `unknown` | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the job is run | | **CI_PROJECT_ID** | all | all | The unique id of the current project that GitLab CI uses internally | | **CI_PROJECT_NAME** | 8.10 | 0.5 | The project name that is currently being built (actually it is project folder name) | @@ -87,6 +87,8 @@ future GitLab releases.** | **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job | | **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job | | **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job | +| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| +| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| ## 9.0 Renaming @@ -454,8 +456,8 @@ export CI_REGISTRY_PASSWORD="longalfanumstring" > Variables expressions were added in GitLab 10.7. It is possible to use variables expressions with only / except policies in -`.gitlab-ci.yml`. By using this approach you can limit what builds are going to -be created within a pipeline after pushing code to GitLab. +`.gitlab-ci.yml`. By using this approach you can limit what jobs are going to +be created within a pipeline after pushing a code to GitLab. This is particularly useful in combination with secret variables and triggered pipeline variables. @@ -470,22 +472,21 @@ deploy: - $STAGING ``` -Each provided variables expression is going to be evaluated before creating -a pipeline. +Each expression provided is going to be evaluated before creating a pipeline. If any of the conditions in `variables` evaluates to truth when using `only`, a new job is going to be created. If any of the expressions evaluates to truth when `except` is being used, a job is not going to be created. -This follows usual rules for `only` / `except` policies. +This follows usual rules for [`only` / `except` policies][builds-policies]. ### Supported syntax -Below you can find currently supported syntax reference: +Below you can find supported syntax reference: 1. Equality matching using a string - Example: `$VARIABLE == "some value"` + > Example: `$VARIABLE == "some value"` You can use equality operator `==` to compare a variable content to a string. We support both, double quotes and single quotes to define a string @@ -494,26 +495,64 @@ Below you can find currently supported syntax reference: 1. Checking for an undefined value - It sometimes happens that you want to check whether variable is defined or - not. To do that, you can compare variable to `null` value, like + > Example: `$VARIABLE == null` + + It sometimes happens that you want to check whether a variable is defined + or not. To do that, you can compare a variable to `null` keyword, like `$VARIABLE == null`. This expression is going to evaluate to truth if - variable is not set. + variable is not defined. 1. Checking for an empty variable + > Example: `$VARIABLE == ""` + If you want to check whether a variable is defined, but is empty, you can simply compare it against an empty string, like `$VAR == ''`. 1. Comparing two variables - It is possible to compare two variables. `$VARIABLE_1 == $VARIABLE_2`. + > Example: `$VARIABLE_1 == $VARIABLE_2` + + It is possible to compare two variables. This is going to compare values + of these variables. 1. Variable presence check + > Example: `$STAGING` + If you only want to create a job when there is some variable present, which means that it is defined and non-empty, you can simply use variable name as an expression, like `$STAGING`. If `$STAGING` variable is defined, and is non empty, expression will evaluate to truth. + `$STAGING` value needs to a string, with length higher than zero. + Variable that contains only whitespace characters is not an empty variable. + +### Unsupported predefined variables + +Because GitLab evaluates variables before creating jobs, we do not support a +few variables that depend on persistence layer, like `$CI_JOB_ID`. + +Environments (like `production` or `staging`) are also being created based on +what jobs pipeline consists of, thus some environment-specific variables are +not supported as well. + +We do not support variables containing tokens because of security reasons. + +You can find a full list of unsupported variables below: + +- `CI_JOB_ID` +- `CI_JOB_TOKEN` +- `CI_BUILD_ID` +- `CI_BUILD_TOKEN` +- `CI_REGISTRY_USER` +- `CI_REGISTRY_PASSWORD` +- `CI_REPOSITORY_URL` +- `CI_ENVIRONMENT_URL` +- `CI_DEPLOY_USER` +- `CI_DEPLOY_PASSWORD` + +These variables are also not supported in a context of a +[dynamic environment name][dynamic-environments]. [ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" [eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium" @@ -525,3 +564,6 @@ Below you can find currently supported syntax reference: [triggered]: ../triggers/README.md [triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger [subgroups]: ../../user/group/subgroups/index.md +[builds-policies]: ../yaml/README.md#only-and-except-complex +[dynamic-environments]: ../environments.md#dynamic-environments +[gitlab-deploy-token]: ../../user/project/deploy_tokens/index.md#gitlab-deploy-token diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index be114e7008e..fb6d9826d08 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -308,7 +308,9 @@ except master. ## `only` and `except` (complex) -> Introduced in GitLab 10.0 +> `refs` and `kubernetes` policies introduced in GitLab 10.0 + +> `variables` policy introduced in 10.7 CAUTION: **Warning:** This an _alpha_ feature, and it it subject to change at any time without @@ -354,7 +356,7 @@ deploy: - $STAGING ``` -Learn more about variables expressions on a separate page. +Learn more about variables expressions on [a separate page][variables-expressions]. ## `tags` @@ -869,37 +871,29 @@ skip the download step. - Introduced in GitLab Runner v0.7.0 for non-Windows platforms. - Windows support was added in GitLab Runner v.1.0.0. - From GitLab 9.2, caches are restored before artifacts. -- Currently not all executors are supported. +- Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). - Job artifacts are only collected for successful jobs by default. `artifacts` is used to specify a list of files and directories which should be -attached to the job after success. You can only use paths that are within the -project workspace. To pass artifacts between different jobs, see [dependencies](#dependencies). -Below are some examples. +attached to the job after success. -Send all files in `binaries` and `.config`: +The artifacts will be sent to GitLab after the job finishes successfully and will +be available for download in the GitLab UI. -```yaml -artifacts: - paths: - - binaries/ - - .config -``` +[Read more about artifacts.](../../user/project/pipelines/job_artifacts.md) -Send all Git untracked files: +### `artifacts:paths` -```yaml -artifacts: - untracked: true -``` +You can only use paths that are within the project workspace. To pass artifacts +between different jobs, see [dependencies](#dependencies). -Send all Git untracked files and files in `binaries`: +Send all files in `binaries` and `.config`: ```yaml artifacts: - untracked: true paths: - binaries/ + - .config ``` To disable artifact passing, define the job with empty [dependencies](#dependencies): @@ -933,11 +927,6 @@ release-job: - tags ``` -The artifacts will be sent to GitLab after the job finishes successfully and will -be available for download in the GitLab UI. - -[Read more about artifacts.](../../user/project/pipelines/job_artifacts.md) - ### `artifacts:name` > Introduced in GitLab 8.6 and GitLab Runner v1.1.0. @@ -954,26 +943,30 @@ To create an archive with a name of the current job: job: artifacts: name: "$CI_JOB_NAME" + paths: + - binaries/ ``` To create an archive with a name of the current branch or tag including only -the files that are untracked by Git: +the binaries directory: ```yaml job: artifacts: name: "$CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ ``` To create an archive with a name of the current job and the current branch or -tag including only the files that are untracked by Git: +tag including only the binaries directory: ```yaml job: artifacts: name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ ``` To create an archive with a name of the current [stage](#stages) and branch name: @@ -982,7 +975,8 @@ To create an archive with a name of the current [stage](#stages) and branch name job: artifacts: name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ ``` --- @@ -994,7 +988,8 @@ If you use **Windows Batch** to run your shell scripts you need to replace job: artifacts: name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" - untracked: true + paths: + - binaries/ ``` If you use **Windows PowerShell** to run your shell scripts you need to replace @@ -1004,7 +999,33 @@ If you use **Windows PowerShell** to run your shell scripts you need to replace job: artifacts: name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" - untracked: true + paths: + - binaries/ +``` + +### `artifacts:untracked` + +`artifacts:untracked` is used to add all Git untracked files as artifacts (along +to the paths defined in `artifacts:paths`). + +NOTE: **Note:** +To exclude the folders/files which should not be a part of `untracked` just +add them to `.gitignore`. + +Send all Git untracked files: + +```yaml +artifacts: + untracked: true +``` + +Send all Git untracked files and files in `binaries`: + +```yaml +artifacts: + untracked: true + paths: + - binaries/ ``` ### `artifacts:when` @@ -1552,7 +1573,7 @@ capitalization, the commit will be created but the pipeline will be skipped. Each instance of GitLab CI has an embedded debug tool called Lint, which validates the content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your -project namespace (e.g, `http://gitlab-example.com/gitlab-org/project-123/ci/lint`) +project namespace (e.g, `http://gitlab-example.com/gitlab-org/project-123/-/ci/lint`) ## Using reserved keywords @@ -1574,3 +1595,4 @@ CI with various languages. [ce-7447]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7447 [ce-12909]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12909 [schedules]: ../../user/project/pipelines/schedules.md +[variables-expressions]: ../variables/README.md#variables-expressions diff --git a/doc/development/README.md b/doc/development/README.md index 45e9565f9a7..3c77e99b8cf 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -39,9 +39,9 @@ comments: false - [Sidekiq debugging](sidekiq_debugging.md) - [Gotchas](gotchas.md) to avoid - [Avoid modules with instance variables](module_with_instance_variables.md) if possible -- [Issue and merge requests state models](object_state_models.md) - [How to dump production data to staging](db_dump.md) - [Working with the GitHub importer](github_importer.md) +- [Working with Merge Request diffs](diffs.md) ## Performance guides diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index fc1b202b5eb..46c5baddb9c 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -24,7 +24,7 @@ Some examples where background migrations can be useful: * Migrating events from one table to multiple separate tables. * Populating one column based on JSON stored in another column. -* Migrating data that depends on the output of exernal services (e.g. an API). +* Migrating data that depends on the output of external services (e.g. an API). ## Isolation @@ -46,7 +46,7 @@ See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/ for more details. Make sure that in case that your migration job is going to be retried data -integrity is guarateed. +integrity is guaranteed. ## How It Works @@ -133,11 +133,19 @@ roughly be as follows: 1. Release B: 1. Deploy code so that the application starts using the new column and stops scheduling jobs for newly created data. - 1. In a post-deployment migration you'll need to ensure no jobs remain. To do - so you can use `Gitlab::BackgroundMigration.steal` to process any remaining - jobs before continuing. + 1. In a post-deployment migration you'll need to ensure no jobs remain. + 1. Use `Gitlab::BackgroundMigration.steal` to process any remaining + jobs in Sidekiq. + 1. Reschedule the migration to be run directly (i.e. not through Sidekiq) + on any rows that weren't migrated by Sidekiq. This can happen if, for + instance, Sidekiq received a SIGKILL, or if a particular batch failed + enough times to be marked as dead. 1. Remove the old column. +This may also require a bump to the [import/export version][import-export], if +importing a project from a prior version of GitLab requires the data to be in +the new format. + ## Example To explain all this, let's use the following example: the table `services` has a @@ -296,3 +304,4 @@ for more details. [migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md [issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 [reliable-sidekiq]: https://gitlab.com/gitlab-org/gitlab-ce/issues/36791 +[import-export]: ../user/project/settings/import_export.md diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 1962392a9eb..a9fa5ae834f 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -22,7 +22,7 @@ The `merge_request` value is a reference to a merge request that adds this entry, and the `author` key is used to give attribution to community contributors. **Both are optional**. The `type` field maps the category of the change, -valid options are: added, fixed, changed, deprecated, removed, security, other. **Type field is mandatory**. +valid options are: added, fixed, changed, deprecated, removed, security, performance, other. **Type field is mandatory**. Community contributors and core team members are encouraged to add their name to the `author` field. GitLab team members **should not**. @@ -44,6 +44,7 @@ the `author` field. GitLab team members **should not**. - _Any_ contribution from a community member, no matter how small, **may** have a changelog entry regardless of these guidelines if the contributor wants one. Example: "Fixed a typo on the search results page. (Jane Smith)" +- Performance improvements **should** have a changelog entry. ## Writing good changelog entries diff --git a/doc/development/diffs.md b/doc/development/diffs.md new file mode 100644 index 00000000000..55fc16e0b33 --- /dev/null +++ b/doc/development/diffs.md @@ -0,0 +1,115 @@ +# Working with Merge Request diffs + +Currently we rely on different sources to present merge request diffs, these include: + +- Rugged gem +- Gitaly service +- Database (through `merge_request_diff_files`) +- Redis (cached highlighted diffs) + +We're constantly moving Rugged calls to Gitaly and the progress can be followed through [Gitaly repo](https://gitlab.com/gitlab-org/gitaly). + +## Architecture overview + +When refreshing a Merge Request (pushing to a source branch, force-pushing to target branch, or if the target branch now contains any commits from the MR) +we fetch the comparison information using `Gitlab::Git::Compare`, which fetches `base` and `head` data using Gitaly and diff between them through +`Gitlab::Git::Diff.between` (which uses _Gitaly_ if it's enabled, otherwise _Rugged_). +The diffs fetching process _limits_ single file diff sizes and the overall size of the whole diff through a series of constant values. Raw diff files are +then persisted on `merge_request_diff_files` table. + +Even though diffs higher than 10kb are collapsed (`Gitlab::Git::Diff::COLLAPSE_LIMIT`), we still keep them on Postgres. However, diff files over _safety limits_ +(see the [Diff limits section](#diff-limits)) are _not_ persisted. + +In order to present diffs information on the Merge Request diffs page, we: + +1. Fetch all diff files from database `merge_request_diff_files` +2. Fetch the _old_ and _new_ file blobs in batch to: + 1. Highlight old and new file content + 2. Know which viewer it should use for each file (text, image, deleted, etc) + 3. Know if the file content changed + 4. Know if it was stored externally + 5. Know if it had storage errors +3. If the diff file is cacheable (text-based), it's cached on Redis +using `Gitlab::Diff::FileCollection::MergeRequestDiff` + +## Diff limits + +As explained above, we limit single diff files and the size of the whole diff. There are scenarios where we collapse the diff file, +and cases where the diff file is not presented at all, and the user is guided to the Blob view. Here we'll go into details about +these limits. + +### Diff collection limits + +Limits that act onto all diff files collection. Files number, lines number and files size are considered. + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:safe_max_files] = Gitlab::Git::DiffCollection::DEFAULT_LIMITS[:max_files] = 100 +``` + +File diffs will be collapsed (but be expandable) if 100 files have already been rendered. + + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:safe_max_lines] = Gitlab::Git::DiffCollection::DEFAULT_LIMITS[:max_lines] = 5000 +``` + +File diffs will be collapsed (but be expandable) if 5000 lines have already been rendered. + + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:safe_max_bytes] = Gitlab::Git::DiffCollection.collection_limits[:safe_max_files] * 5.kilobytes = 500.kilobytes +``` + +File diffs will be collapsed (but be expandable) if 500 kilobytes have already been rendered. + + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:max_files] = Commit::DIFF_HARD_LIMIT_FILES = 1000 +``` + +No more files will be rendered at all if 1000 files have already been rendered. + + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:max_lines] = Commit::DIFF_HARD_LIMIT_LINES = 50000 +``` + +No more files will be rendered at all if 50,000 lines have already been rendered. + +```ruby +Gitlab::Git::DiffCollection.collection_limits[:max_bytes] = Gitlab::Git::DiffCollection.collection_limits[:max_files] * 5.kilobytes = 5000.kilobytes +``` + +No more files will be rendered at all if 5 megabytes have already been rendered. + + +### Individual diff file limits + +Limits that act onto each diff file of a collection. Files number, lines number and files size are considered. + +```ruby +Gitlab::Git::Diff::COLLAPSE_LIMIT = 10.kilobytes +``` + +File diff will be collapsed (but be expandable) if it is larger than 10 kilobytes. + +```ruby +Gitlab::Git::Diff::SIZE_LIMIT = 100.kilobytes +``` + +File diff will not be rendered if it's larger than 100 kilobytes. + + +```ruby +Commit::DIFF_SAFE_LINES = Gitlab::Git::DiffCollection::DEFAULT_LIMITS[:max_lines] = 5000 +``` + +File diff will be suppressed (technically different from collapsed, but behaves the same, and is expandable) if it has more than 5000 lines. + +## Viewers + +Diff Viewers, which can be found on `models/diff_viewer/*` are classes used to map metadata about each type of Diff File. It has information +whether it's a binary, which partial should be used to render it or which File extensions this class accounts for. + +`DiffViewer::Base` validates _blobs_ (old and new versions) content, extension and file type in order to check if it can be rendered. + diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 41e3412c7ff..5da015ca557 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -4,7 +4,7 @@ The documentation style guide defines the markup structure used in GitLab documentation. Check the [documentation guidelines](writing_documentation.md) for general development instructions. -Check the GitLab hanbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). +Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). ## Text @@ -19,7 +19,7 @@ Check the GitLab hanbook for the [writing styles guidelines](https://about.gitla - Unless there's a logical reason not to, add documents in alphabetical order - Write in US English - Use [single spaces][] instead of double spaces -- Jump a line between different markups (e.g., after every paragraph, hearder, list, etc) +- Jump a line between different markups (e.g., after every paragraph, header, list, etc) - Capitalize "G" and "L" in GitLab - Capitalize feature, products, and methods names. E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. @@ -157,6 +157,39 @@ below. Otherwise, leave this mention out. +### Product badges + +When a feature is available in EE-only tiers, add the corresponding tier according to the +feature availability: + +- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` +- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` +- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` +- For GitLab Core and GitLab.com Free: `**[CORE]**` + +To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the +keyword "only": + +- For GitLab Starter: `**[STARTER ONLY]**` +- For GitLab Premium: `**[PREMIUM ONLY]**` +- For GitLab Ultimate: `**[ULTIMATE ONLY]**` +- For GitLab Core: `**[CORE ONLY]**` + +The tier should be ideally added to headers, so that the full badge will be displayed. +But it can be also mentioned from paragraphs, list items, and table cells. For these cases, +the tier mention will be represented by an orange question mark. +E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. + +The absence of tiers' mentions mean that the feature is available in GitLab Core, +GitLab.com Free, and higher tiers. + +#### How it works + +Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), +the special markup `**[STARTER]**` will generate a `span` element to trigger the +badges and tooltips (`<span class="badge-trigger starter">`). When the keyword +"only" is added, the corresponding GitLab.com badge will not be displayed. + ### GitLab Restart There are many cases that a restart/reconfigure of GitLab is required. To diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 287143d6255..4873090a2d4 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -279,7 +279,7 @@ end ``` In `lib/gitlab/visibility_level.rb` this method is used to return the -allowed visibilty levels: +allowed visibility levels: ```ruby def levels_for_user(user = nil) diff --git a/doc/development/emails.md b/doc/development/emails.md index 4dbf064fd75..73cac82caf0 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -74,6 +74,24 @@ See the [Rails guides] for more info. 1. Reply by email should now be working. +## Email namespace + +If you need to implement a new feature which requires a new email handler, follow these rules: + + - You must choose a namespace. The namespace cannot contain `/` or `+`, and cannot match `\h{16}`. + - If your feature is related to a project, you will append the namespace **after** the project path, separated by a `+` + - If you have different actions in the namespace, you add the actions **after** the namespace separated by a `+`. The action name cannot contain `/` or `+`, , and cannot match `\h{16}`. + - You will register your handlers in `lib/gitlab/email/handler.rb` + +Therefore, these are the only valid formats for an email handler: + + - `path/to/project+namespace` + - `path/to/project+namespace+action` + - `namespace` + - `namespace+action` + +Please note that `path/to/project` is used in GitLab Premium as handler for the Service Desk feature. + --- [Return to Development documentation](README.md) diff --git a/doc/development/fe_guide/development_process.md b/doc/development/fe_guide/development_process.md new file mode 100644 index 00000000000..5504a6421d5 --- /dev/null +++ b/doc/development/fe_guide/development_process.md @@ -0,0 +1,77 @@ +# Frontend Development Process + +You can find more about the organization of the frontend team in the [handbook](https://about.gitlab.com/handbook/frontend/). + +## Development Checklist + +The idea is to remind us about specific topics during the time we build a new feature or start something. This is a common practice in other industries (like pilots) that also use standardised checklists to reduce problems early on. + +Copy the content over to your issue or merge request and if something doesn't apply simply remove it from your current list. + +This checklist is intended to help us during development of bigger features/refactorings, it's not a "use it always and every point always matches" list. + +Please use your best judgement when to use it and please contribute new points through merge requests if something comes to your mind. + +--- + +### Frontend development + +#### Planning development + +- [ ] Check the current set weight of the issue, does it fit your estimate? +- [ ] Are all [departments](https://about.gitlab.com/handbook/engineering/#engineering-teams) that are needed from your perspective already involved in the issue? (For example is UX missing?) +- [ ] Is the specification complete? Are you missing decisions? How about error handling/defaults/edge cases? Take your time to understand the needed implementation and go through its flow. +- [ ] Are all necessary UX specifications available that you will need in order to implement? Are there new UX components/patterns in the designs? Then contact the UI component team early on. How should error messages or validation be handled? +- [ ] **Library usage** Use Vuex as soon as you have even a medium state to manage, use Vue router if you need to have different views internally and want to link from the outside. Check what libraries we already have for which occassions. +- [ ] **Plan your implementation:** + - [ ] **Architecture plan:** Create a plan aligned with GitLab's architecture, how you are going to do the implementation, for example Vue application setup and its components (through [onion skinning](https://gitlab.com/gitlab-org/gitlab-ce/issues/35873#note_39994091)), Store structure and data flow, which existing Vue components can you reuse. Its a good idea to go through your plan with another engineer to refine it. + - [ ] **Backend:** The best way is to kickoff the implementation in a call and discuss with the assigned Backend engineer what you will need from the backend and also when. Can you reuse existing API's? How is the performance with the planned architecture? Maybe create together a JSON mock object to already start with development. + - [ ] **Communication:** It also makes sense to have for bigger features an own slack channel (normally called #f_{feature_name}) and even weekly demo calls with all people involved. + - [ ] **Dependency Plan:** Are there big dependencies in the plan between you and others, then maybe create an execution diagram to show what is blocking which part and the order of the different parts. + - [ ] **Task list:** Create a simple checklist of the subtasks that are needed for the implementation, also consider creating even sub issues. (for example show a comment, delete a comment, update a comment, etc.). This helps you and also everyone else following the implementation +- [ ] **Keep it small** To make it easier for you and also all reviewers try to keep merge requests small and merge into a feature branch if needed. To accomplish that you need to plan that from the start. Different methods are: + - [ ] **Skeleton based plan** Start with an MR that has the skeleton of the components with placeholder content. In following MRs you can fill the components with interactivity. This also makes it easier to spread out development on multiple people. + - [ ] **Cookie Mode** Think about hiding the feature behind a cookie flag if the implementation is on top of existing features + - [ ] **New route** Are you refactoring something big then you might consider adding a new route where you implement the new feature and when finished delete the current route and rename the new one. (for example 'merge_request' and 'new_merge_request') +- [ ] **Setup** Is there any specific setup needed for your implementation (for example a kubernetes cluster)? Then let everyone know if it is not already mentioned where they can find documentation (if it doesn't exist - create it) +- [ ] **Security** Are there any new security relevant implementations? Then please contact the security team for an app security review. If you are not sure ask our [domain expert](https://about.gitlab.com/handbook/frontend/#frontend-domain-experts) + +#### During development + +- [ ] Check off tasks on your created task list to keep everyone updated on the progress +- [ ] [Share your work early with reviewers/maintainers](#share-your-work-early) +- [ ] Share your work with UXer and Product Manager with Screenshots and/or [GIF's](https://about.gitlab.com/handbook/product/making-gifs/). They are easy to create for you and keep them up to date. +- [ ] If you are blocked on something let everyone on the issue know through a comment. +- [ ] Are you unable to work on this issue for a longer period of time, also let everyone know. +- [ ] **Documentation** Update/add docs for the new feature, see `docs/`. Ping one of the documentation experts/reviewers + +#### Finishing development + Review + +- [ ] **Keep it in the scope** Try to focus on the actual scope and avoid a scope creep during review and keep new things to new issues. +- [ ] **Performance** Have you checked performance? For example do the same thing with 500 comments instead of 1. Document the tests and possible findings in the MR so a reviewer can directly see it. +- [ ] Have you tested with a variety of our [supported browsers](../../install/requirements.md#supported-web-browsers)? You can use [browserstack](https://www.browserstack.com/) to be able to access a wide variety of browsers and operating systems. +- [ ] Did you check the mobile view? +- [ ] Check the built webpack bundle (For the report run `WEBPACK_REPORT=true gdk run`, then open `webpack-report/index.html`) if we have unnecessary bloat due to wrong references, including libraries multiple times, etc.. If you need help contact the webpack [domain expert](https://about.gitlab.com/handbook/frontend/#frontend-domain-experts) +- [ ] **Tests** Not only greenfield tests - Test also all bad cases that come to your mind. +- [ ] If you have multiple MR's then also smoke test against the final merge. +- [ ] Are there any big changes on how and especially how frequently we use the API then let production know about it +- [ ] Smoke test of the RC on dev., staging., canary deployments and .com +- [ ] Follow up on issues that came out of the review. Create isssues for discovered edge cases that should be covered in future iterations. + +--- + +### Share your work early +1. Before writing code, ensure your vision of the architecture is aligned with +GitLab's architecture. +1. Add a diagram to the issue and ask a frontend architect in the slack channel `#fe_architectural` about it. + +  + +1. Don't take more than one week between starting work on a feature and +sharing a Merge Request with a reviewer or a maintainer. + +### Vue features +1. Follow the steps in [Vue.js Best Practices](vue.md) +1. Follow the style guide. +1. Only a handful of people are allowed to merge Vue related features. +Reach out to one of Vue experts early in this process. diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index b288ee95722..b469a9c6aef 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -49,7 +49,7 @@ Please use the following function inside JS to render an icon : All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. -To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. The updated files should be tracked in Git as those are referenced. +To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs`. # SVG Illustrations diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 73084d667d4..5a67414e835 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -5,11 +5,15 @@ across GitLab's frontend team. ## Overview -GitLab is built on top of [Ruby on Rails][rails] using [Haml][haml] with -[Hamlit][hamlit]. Be wary of [the limitations that come with using -Hamlit][hamlit-limits]. We also use [SCSS][scss] and plain JavaScript with -modern ECMAScript standards supported through [Babel][babel] and ES module -support through [webpack][webpack]. +GitLab is built on top of [Ruby on Rails][rails] using [Haml][haml] and also a JavaScript based Frontend with [Vue.js][vue]. +Be wary of [the limitations that come with using Hamlit][hamlit-limits]. We also use [SCSS][scss] and plain JavaScript with +modern ECMAScript standards supported through [Babel][babel] and ES module support through [webpack][webpack]. + +### Javascript development + +[Vue.js][vue] is used for particularly advanced, dynamic elements and based on previous iterations [jQuery][jquery] is used in lot of places through the application's JavaScript. + +We also use [Axios][axios] to handle all of our network requests. We also utilize [webpack][webpack] to handle the bundling, minification, and compression of our assets. @@ -18,66 +22,32 @@ Working with our frontend assets requires Node (v6.0 or greater) and Yarn (v1.2 or greater). You can find information on how to install these on our [installation guide][install]. -[jQuery][jquery] is used throughout the application's JavaScript, with -[Vue.js][vue] for particularly advanced, dynamic elements. - -We also use [Axios][axios] to handle all of our network requests. - ### Browser Support For our currently-supported browsers, see our [requirements][requirements]. --- -## Development Process - -### Share your work early -1. Before writing code guarantee your vision of the architecture is aligned with -GitLab's architecture. -1. Add a diagram to the issue and ask a Frontend Architecture about it. - -  - -1. Don't take more than one week between starting work on a feature and -sharing a Merge Request with a reviewer or a maintainer. - -### Vue features -1. Follow the steps in [Vue.js Best Practices](vue.md) -1. Follow the style guide. -1. Only a handful of people are allowed to merge Vue related features. -Reach out to one of Vue experts early in this process. - - ---- +## [Development Process](development_process.md) +How we plan and execute the work on the frontend. ## [Architecture](architecture.md) How we go about making fundamental design decisions in GitLab's frontend team or make changes to our frontend development guidelines. ---- - ## [Testing](../testing_guide/frontend_testing.md) - How we write frontend tests, run the GitLab test suite, and debug test related issues. ---- - ## [Design Patterns](design_patterns.md) Common JavaScript design patterns in GitLab's codebase. ---- - ## [Vue.js Best Practices](vue.md) Vue specific design patterns and practices. ---- - ## [Vuex](vuex.md) Vuex specific design patterns and practices. ---- - ## [Axios](axios.md) Axios specific practices and gotchas. diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 7b5fa6ca42f..04dfe418dbe 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -236,7 +236,7 @@ export class Foo { } ``` -On the other hand, if a class only needs to extend a third party/add event listeners in some specific cases, they should be initialized oustside of the constructor. +On the other hand, if a class only needs to extend a third party/add event listeners in some specific cases, they should be initialized outside of the constructor. 1. Prefer `.map`, `.reduce` or `.filter` over `.forEach` A forEach will most likely cause side effects, it will be mutating the array being iterated. Prefer using `.map`, @@ -310,7 +310,7 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. })); ``` -1. Don not use a singleton for the service or the store +1. Do not use a singleton for the service or the store ```javascript // bad class Store { @@ -328,9 +328,11 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. } } ``` +1. Use `.vue` for Vue templates. Do not use `%template` in HAML. #### Naming -1. **Extensions**: Use `.vue` extension for Vue components. + +1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371]). 1. **Reference Naming**: Use PascalCase for their instances: ```javascript // bad @@ -364,6 +366,8 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. <component my-prop="prop" /> ``` +[#34371]: https://gitlab.com/gitlab-org/gitlab-ce/issues/34371 + #### Alignment 1. Follow these alignment styles for the template method: 1. With more than one attribute, all attributes should be on a new line: diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 3b68fd858cc..62c3a05eb3b 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -440,7 +440,6 @@ need to test the rendered output. [Vue][vue-test] guide's to unit test show us e Refer to [mock axios](axios.md#mock-axios-response-on-tests) - [vue-docs]: http://vuejs.org/guide/index.html [issue-boards]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/assets/javascripts/boards [environments-table]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/assets/javascripts/environments diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md index 7298ffcd54a..d0d74afe7bb 100644 --- a/doc/development/fe_guide/vuex.md +++ b/doc/development/fe_guide/vuex.md @@ -1,5 +1,5 @@ # Vuex -To manage the state of an application you may use [Vuex][vuex-docs]. +To manage the state of an application you should use [Vuex][vuex-docs]. _Note:_ All of the below is explained in more detail in the official [Vuex documentation][vuex-docs]. @@ -115,8 +115,8 @@ create: 1. An action `requestSomething`, to toggle the loading state 1. An action `receiveSomethingSuccess`, to handle the success callback 1. An action `receiveSomethingError`, to handle the error callback -1. An action `fetchSomething` to make the request. - 1. In case your application does more than a `GET` request you can use these as examples: +1. An action `fetchSomething` to make the request. + 1. In case your application does more than a `GET` request you can use these as examples: 1. `PUT`: `createSomething` 2. `POST`: `updateSomething` 3. `DELETE`: `deleteSomething` diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index 34a02bd2c3c..fdbd7f1fa37 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -84,7 +84,7 @@ The `RecordsUploads::Concern` concern will create an `Upload` entry for every fi By including the `ObjectStorage::Concern` in the `GitlabUploader` derived class, you may enable the object storage for this uploader. To enable the object storage in your uploader, you need to either 1) include `RecordsUpload::Concern` and prepend `ObjectStorage::Extension::RecordsUploads` or 2) mount the uploader and create a new field named `<mount>_store`. -The `CarrierWave::Uploader#store_dir` is overriden to +The `CarrierWave::Uploader#store_dir` is overridden to - `GitlabUploader.base_dir` + `GitlabUploader.dynamic_segment` when the store is LOCAL - `GitlabUploader.dynamic_segment` when the store is REMOTE (the bucket name is used to namespace) diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index 26abf967dcf..4f9ca1920a5 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -7,6 +7,67 @@ be replaced by Gitaly API calls. Visit the [Gitaly Migration Board](https://gitlab.com/gitlab-org/gitaly/boards/331341) for current status of the migration. +## Developing new Git features + +Starting with Gitlab 10.8, all new Git features should be developed in +Gitaly. + +> This is a new process that is not clearly defined yet. If you want +to contribute a Git feature and you're getting stuck, reach out to the +Gitaly team or `@jacobvosmaer-gitlab`. + +By 'new feature' we mean any method or class in `lib/gitlab/git` that is +called from outside `lib/gitlab/git`. For new methods that are called +from inside `lib/gitlab/git`, see 'Modifying existing Git features' +below. + +There should be no new code that touches Git repositories via +disk access (e.g. Rugged, `git`, `rm -rf`) anywhere outside +`lib/gitlab/git`. + +The process for adding new Gitaly features is: + +- exploration / prototyping +- design and create a new Gitaly RPC [in gitaly-proto](https://gitlab.com/gitlab-org/gitaly-proto) +- release a new version of gitaly-proto +- write implementation and tests for the RPC [in Gitaly](https://gitlab.com/gitlab-org/gitaly), in Go or Ruby +- release a new version of Gitaly +- write client code in gitlab-ce/ee, gitlab-workhorse or gitlab-shell that calls the new Gitaly RPC + +These steps often overlap. It is possible to use an unreleased version +of Gitaly and gitaly-proto during testing and development. + +- See the [Gitaly repo](https://gitlab.com/gitlab-org/gitaly/blob/master/CONTRIBUTING.md#development-and-testing-with-a-custom-gitaly-proto) for instructions on writing server side code with an unreleased protocol. +- See [below](#running-tests-with-a-locally-modified-version-of-gitaly) for instructions on running gitlab-ce tests with a modified version of Gitaly. +- In GDK run `gdk install` and restart `gdk run` (or `gdk run app`) to use a locally modified Gitaly version for development + +### Gitaly-ruby + +It is possible to implement and test RPC's in Gitaly using Ruby code, +in +[gitaly-ruby](https://gitlab.com/gitlab-org/gitaly/tree/master/ruby). +This should make it easier to contribute for developers who are less +comfortable writing Go code. + +There is documentation for this approach in [the Gitaly +repo](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/ruby_endpoint.md). + +## Modifying existing Git features + +If you modify existing Git features in `lib/gitlab/git` you need to make +sure the changes also work in Gitaly. Because we are still in the +migration process there are a number of subtle pitfalls. Features that +have been migrated have dual implementations (Gitaly and local). The +Gitaly implementation may or may not use a vendored (and therefore +possibly outdated) copy of the local implementation in `lib/gitlab/git`. + +To avoid unexpected problems and conflicts, all changes to +`lib/gitlab/git` need to be approved by a member of the Gitaly team. + +For the time being, while the Gitaly migration is still in progress, +there should be no Enterprise Edition-only Git code in +`lib/gitlab/git`. Also no mixins. + ## Feature Flags Gitaly makes heavy use of [feature flags](feature_flags.md). @@ -99,10 +160,14 @@ end ## Running tests with a locally modified version of Gitaly -Normally, gitlab-ce/ee tests use a local clone of Gitaly in `tmp/tests/gitaly` -pinned at the version specified in GITALY_SERVER_VERSION. If you want -to run tests locally against a modified version of Gitaly you can -replace `tmp/tests/gitaly` with a symlink. +Normally, gitlab-ce/ee tests use a local clone of Gitaly in +`tmp/tests/gitaly` pinned at the version specified in +`GITALY_SERVER_VERSION`. The `GITALY_SERVER_VERSION` file supports +`=my-branch` syntax to use a custom branch in gitlab-org/gitaly. If +you want to run tests locally against a modified version of Gitaly you +can replace `tmp/tests/gitaly` with a symlink. This is much faster +because the `=my-branch` syntax forces a Gitaly re-install each time +you run `rspec`. ```shell rm -rf tmp/tests/gitaly diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 856ef882453..0edcb23c7c5 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -131,6 +131,9 @@ There is also and alternative method to [translate messages from validation erro ### Interpolation +Placeholders in translated text should match the code style of the respective source file. +For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. + - In Ruby/HAML: ```ruby @@ -141,11 +144,19 @@ There is also and alternative method to [translate messages from validation erro ```js import { __, sprintf } from '~/locale'; - sprintf(__('Hello %{username}'), { username: 'Joe' }) => 'Hello Joe' + + sprintf(__('Hello %{username}'), { username: 'Joe' }); // => 'Hello Joe' ``` -The placeholders should match the code style of the respective source file. -For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. + By default, `sprintf` escapes the placeholder values. + If you want to take care of that yourself, you can pass `false` as third argument. + + ```js + import { __, sprintf } from '~/locale'; + + sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }); // => 'This is <strong>bold</strong>' + sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }, false); // => 'This is <strong>bold</strong>' + ``` ### Plurals @@ -259,7 +270,7 @@ If there are merge conflicts in the `gitlab.pot` file, you can delete the file and regenerate it using the same command. Confirm that you are not deleting any strings accidentally by looking over the diff. The command also updates the translation files for each language: `locale/*/gitlab.po` -These changes can be discarded, the languange files will be updated by Crowdin +These changes can be discarded, the language files will be updated by Crowdin automatically. Discard all of them at once like this: diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index cf62314bc29..5185d843ccb 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -23,6 +23,7 @@ are very appreciative of the work done by translators and proofreaders! - Italian - Paolo Falomo - [GitLab](https://gitlab.com/paolofalomo), [Crowdin](https://crowdin.com/profile/paolo.falomo) - Japanese + - Yamana Tokiuji - [GitLab](https://gitlab.com/tokiuji), [Crowdin](https://crowdin.com/profile/yamana) - Korean - Chang-Ho Cha - [GitLab](https://gitlab.com/changho-cha), [Crowdin](https://crowdin.com/profile/zzazang) - Huang Tao - [GitLab](https://gitlab.com/htve), [Crowdin](https://crowdin.com/profile/htve) diff --git a/doc/development/img/state-model-issue.png b/doc/development/img/state-model-issue.png Binary files differdeleted file mode 100644 index ee33b6886c6..00000000000 --- a/doc/development/img/state-model-issue.png +++ /dev/null diff --git a/doc/development/img/state-model-legend.png b/doc/development/img/state-model-legend.png Binary files differdeleted file mode 100644 index 1c121f2588c..00000000000 --- a/doc/development/img/state-model-legend.png +++ /dev/null diff --git a/doc/development/img/state-model-merge-request.png b/doc/development/img/state-model-merge-request.png Binary files differdeleted file mode 100644 index e00da10cac2..00000000000 --- a/doc/development/img/state-model-merge-request.png +++ /dev/null diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 2b4126b43ef..12badbe39b2 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -162,7 +162,7 @@ need for running complex operations to fetch the data. You should use Redis if data should be cached for a certain time period instead of the duration of the transaction. -For example, say you process multiple snippets of text containiner username +For example, say you process multiple snippets of text containing username mentions (e.g. `Hello @alice` and `How are you doing @alice?`). By caching the user objects for every username we can remove the need for running the same query for every mention of `@alice`. diff --git a/doc/development/new_fe_guide/development/components.md b/doc/development/new_fe_guide/development/components.md index 637099d1e83..899efb398cd 100644 --- a/doc/development/new_fe_guide/development/components.md +++ b/doc/development/new_fe_guide/development/components.md @@ -1,3 +1,21 @@ # Components -> TODO: Add content +## Graphs + +We have a lot of graphing libraries in our codebase to render graphs. In an effort to improve maintainability, new graphs should use [D3.js](https://d3js.org/). If a new graph is fairly simple, consider implementing it in SVGs or HTML5 canvas. + +We chose D3 as our library going forward because of the following features: + +* [Tree shaking webpack capabilities.](https://github.com/d3/d3/blob/master/CHANGES.md#changes-in-d3-40) +* [Compatible with vue.js as well as vanilla javascript.](https://github.com/d3/d3/blob/master/CHANGES.md#changes-in-d3-40) + +D3 is very popular across many projects outside of GitLab: + +* [The New York Times](https://archive.nytimes.com/www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html) +* [plot.ly](https://plot.ly/) +* [Droptask](https://www.droptask.com/) + +Within GitLab, D3 has been used for the following notable features + +* [Prometheus graphs](https://docs.gitlab.com/ee/user/project/integrations/prometheus.html) +* Contribution calendars diff --git a/doc/development/object_state_models.md b/doc/development/object_state_models.md deleted file mode 100644 index 623bbf143ef..00000000000 --- a/doc/development/object_state_models.md +++ /dev/null @@ -1,25 +0,0 @@ -# Object state models - -## Diagrams - -[GitLab object state models](https://drive.google.com/drive/u/3/folders/0B5tDlHAM4iZINmpvYlJXcDVqMGc) - ---- - -## Legend - - - ---- - -## Issue - -[`app/models/issue.rb`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/models/issue.rb) - - ---- - -## Merge request - -[`app/models/merge_request.rb`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/models/merge_request.rb) -
\ No newline at end of file diff --git a/doc/development/ordering_table_columns.md b/doc/development/ordering_table_columns.md index 249e70c7b0e..5d00e1f7a0c 100644 --- a/doc/development/ordering_table_columns.md +++ b/doc/development/ordering_table_columns.md @@ -30,7 +30,7 @@ example) at the end. ## Type Sizes -While the PostgreSQL docuemntation +While the PostgreSQL documentation (https://www.postgresql.org/docs/current/static/datatype.html) contains plenty of information we will list the sizes of common types here so it's easier to look them up. Here "word" refers to the word size, which is 4 bytes for a 32 diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index df80cd9f584..7b32e0a47e1 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -63,6 +63,8 @@ writing one](testing_levels.md#consider-not-writing-a-system-test)! Sometimes you may need to debug Capybara tests by observing browser behavior. +#### Live debug + You can pause Capybara and view the website on the browser by using the `live_debug` method in your spec. The current page will be automatically opened in your default browser. @@ -90,6 +92,54 @@ Finished in 34.51 seconds (files took 0.76702 seconds to load) Note: `live_debug` only works on javascript enabled specs. +#### Run `:js` spec in a visible browser + +Run the spec with `CHROME_HEADLESS=0`, e.g.: + +``` +CHROME_HEADLESS=0 bundle exec rspec some_spec.rb +``` + +The test will go by quickly, but this will give you an idea of what's happening. + +You can also add `byebug` or `binding.pry` to pause execution and step through +the test. + +#### Screenshots + +We use the `capybara-screenshot` gem to automatically take a screenshot on +failure. In CI you can download these files as job artifacts. + +Also, you can manually take screenshots at any point in a test by adding the +methods below. Be sure to remove them when they are no longer needed! See +https://github.com/mattheworiordan/capybara-screenshot#manual-screenshots for +more. + +Add `screenshot_and_save_page` in a `:js` spec to screenshot what Capybara +"sees", and save the page source. + +Add `screenshot_and_open_image` in a `:js` spec to screenshot what Capybara +"sees", and automatically open the image. + +### Fast unit tests + +Some classes are well-isolated from Rails and you should be able to test them +without the overhead added by the Rails environment and Bundler's `:default` +group's gem loading. In these cases, you can `require 'fast_spec_helper'` +instead of `require 'spec_helper'` in your test file, and your test should run +really fast since: + +- Gems loading is skipped +- Rails app boot is skipped +- gitlab-shell and Gitaly setup are skipped +- Test repositories setup are skipped + +Note that in some cases, you might have to add some `require_dependency 'foo'` +in your file under test since Rails autoloading is not available in these cases. + +This shouldn't be a problem since explicitely listing dependencies should be +considered a good practice anyway. + ### `let` variables GitLab's RSpec suite has made extensive use of `let` variables to reduce @@ -281,14 +331,13 @@ All fixtures should be be placed under `spec/fixtures/`. RSpec config files are files that change the RSpec config (i.e. `RSpec.configure do |config|` blocks). They should be placed under -`spec/support/config/`. +`spec/support/`. Each file should be related to a specific domain, e.g. -`spec/support/config/capybara.rb`, `spec/support/config/carrierwave.rb`, etc. +`spec/support/capybara.rb`, `spec/support/carrierwave.rb`, etc. -Helpers can be included in the `spec/support/config/rspec.rb` file. If a -helpers module applies only to a certain kind of specs, it should add modifiers -to the `config.include` call. For instance if +If a helpers module applies only to a certain kind of specs, it should add +modifiers to the `config.include` call. For instance if `spec/support/helpers/cycle_analytics_helpers.rb` applies to `:lib` and `type: :model` specs only, you would write the following: @@ -299,6 +348,14 @@ RSpec.configure do |config| end ``` +If a config file only consists of `config.include`, you can add these +`config.include` directly in `spec/spec_helper.rb`. + +For very generic helpers, consider including them in the `spec/support/rspec.rb` +file which is used by the `spec/fast_spec_helper.rb` file. See +[Fast unit tests](#fast-unit-tests) for more details about the +`spec/fast_spec_helper.rb` file. + --- [Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index d10a797a142..21ec926414d 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -67,9 +67,9 @@ and examples in [the `qa/` directory][instance-qa-examples]. ## Where can I ask for help? -You can ask question in the `#qa` channel on Slack (GitLab internal) or you can -find an issue you would like to work on in [the issue tracker][gitlab-qa-issues] -and start a new discussion there. +You can ask question in the `#quality` channel on Slack (GitLab internal) or +you can find an issue you would like to work on in +[the issue tracker][gitlab-qa-issues] and start a new discussion there. [omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab [gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 0c63f51cb45..0d0d511582b 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -62,6 +62,7 @@ describe('.methodName', () => { }); }); ``` + #### Testing promises When testing Promises you should always make sure that the test is asynchronous and rejections are handled. @@ -69,9 +70,9 @@ Your Promise chain should therefore end with a call of the `done` callback and ` ```javascript // Good -it('tests a promise', (done) => { +it('tests a promise', done => { promise - .then((data) => { + .then(data => { expect(data).toBe(asExpected); }) .then(done) @@ -79,10 +80,10 @@ it('tests a promise', (done) => { }); // Good -it('tests a promise rejection', (done) => { +it('tests a promise rejection', done => { promise .then(done.fail) - .catch((error) => { + .catch(error => { expect(error).toBe(expectedError); }) .then(done) @@ -91,48 +92,85 @@ it('tests a promise rejection', (done) => { // Bad (missing done callback) it('tests a promise', () => { - promise - .then((data) => { - expect(data).toBe(asExpected); - }) + promise.then(data => { + expect(data).toBe(asExpected); + }); }); // Bad (missing catch) -it('tests a promise', (done) => { +it('tests a promise', done => { promise - .then((data) => { + .then(data => { expect(data).toBe(asExpected); }) - .then(done) + .then(done); }); // Bad (use done.fail in asynchronous tests) -it('tests a promise', (done) => { +it('tests a promise', done => { promise - .then((data) => { + .then(data => { expect(data).toBe(asExpected); }) .then(done) - .catch(fail) + .catch(fail); }); // Bad (missing catch) -it('tests a promise rejection', (done) => { +it('tests a promise rejection', done => { promise - .catch((error) => { + .catch(error => { expect(error).toBe(expectedError); }) - .then(done) + .then(done); }); ``` -#### Stubbing +#### Stubbing and Mocking + +Jasmine provides useful helpers `spyOn`, `spyOnProperty`, `jasmine.createSpy`, +and `jasmine.createSpyObject` to facilitate replacing methods with dummy +placeholders, and recalling when they are called and the arguments that are +passed to them. These tools should be used liberally, to test for expected +behavior, to mock responses, and to block unwanted side effects (such as a +method that would generate a network request or alter `window.location`). The +documentation for these methods can be found in the [jasmine introduction page](https://jasmine.github.io/2.0/introduction.html#section-Spies). + +Sometimes you may need to spy on a method that is directly imported by another +module. GitLab has a custom `spyOnDependency` method which utilizes +[babel-plugin-rewire](https://github.com/speedskater/babel-plugin-rewire) to +achieve this. It can be used like so: + +```javascript +// my_module.js +import { visitUrl } from '~/lib/utils/url_utility'; + +export default function doSomething() { + visitUrl('/foo/bar'); +} -For unit tests, you should stub methods that are unrelated to the current unit you are testing. -If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely. +// my_module_spec.js +import doSomething from '~/my_module'; -For integration tests, you should stub methods that will effect the stability of the test if they -execute their original behaviour. i.e. Network requests. +describe('my_module', () => { + it('does something', () => { + const visitUrl = spyOnDependency(doSomething, 'visitUrl'); + + doSomething(); + expect(visitUrl).toHaveBeenCalledWith('/foo/bar'); + }); +}); +``` + +Unlike `spyOn`, `spyOnDependency` expects its first parameter to be the default +export of a module who's import you want to stub, rather than an object which +contains a method you wish to stub (if the module does not have a default +export, one is be generated by the babel plugin). The second parameter is the +name of the import you wish to change. The result of the function is a Spy +object which can be treated like any other jasmine spy object. + +Further documentation on the babel rewire pluign API can be found on +[its repository Readme doc](https://github.com/speedskater/babel-plugin-rewire#babel-plugin-rewire). ### Vue.js unit tests @@ -143,8 +181,8 @@ See this [section][vue-test]. `rake karma` runs the frontend-only (JavaScript) tests. It consists of two subtasks: -- `rake karma:fixtures` (re-)generates fixtures -- `rake karma:tests` actually executes the tests +* `rake karma:fixtures` (re-)generates fixtures +* `rake karma:tests` actually executes the tests As long as the fixtures don't change, `rake karma:tests` (or `yarn karma`) is sufficient (and saves you some time). @@ -152,19 +190,41 @@ is sufficient (and saves you some time). ### Live testing and focused testing While developing locally, it may be helpful to keep karma running so that you -can get instant feedback on as you write tests and modify code. To do this -you can start karma with `npm run karma-start`. It will compile the javascript +can get instant feedback on as you write tests and modify code. To do this +you can start karma with `yarn run karma-start`. It will compile the javascript assets and run a server at `http://localhost:9876/` where it will automatically -run the tests on any browser which connects to it. You can enter that url on +run the tests on any browser which connects to it. You can enter that url on multiple browsers at once to have it run the tests on each in parallel. While karma is running, any changes you make will instantly trigger a recompile and retest of the entire test suite, so you can see instantly if you've broken -a test with your changes. You can use [jasmine focused][jasmine-focus] or +a test with your changes. You can use [jasmine focused][jasmine-focus] or excluded tests (with `fdescribe` or `xdescribe`) to get karma to run only the tests you want while you're working on a specific feature, but make sure to remove these directives when you commit your code. +It is also possible to only run karma on specific folders or files by filtering +the run tests via the argument `--filter-spec` or short `-f`: + +```bash +# Run all files +yarn karma-start +# Run specific spec files +yarn karma-start --filter-spec profile/account/components/update_username_spec.js +# Run specific spec folder +yarn karma-start --filter-spec profile/account/components/ +# Run all specs which path contain vue_shared or vie +yarn karma-start -f vue_shared -f vue_mr_widget +``` + +You can also use glob syntax to match files. Remember to put quotes around the +glob otherwise your shell may split it into multiple arguments: + +```bash +# Run all specs named `file_spec` within the IDE subdirectory +yarn karma -f 'spec/javascripts/ide/**/file_spec.js' +``` + ## RSpec feature integration tests Information on setting up and running RSpec integration tests with @@ -176,19 +236,19 @@ Information on setting up and running RSpec integration tests with Similar errors will be thrown if you're using JavaScript features not yet supported by the PhantomJS test runner which is used for both Karma and RSpec -tests. We polyfill some JavaScript objects for older browsers, but some +tests. We polyfill some JavaScript objects for older browsers, but some features are still unavailable: -- Array.from -- Array.first -- Async functions -- Generators -- Array destructuring -- For..Of -- Symbol/Symbol.iterator -- Spread +* Array.from +* Array.first +* Async functions +* Generators +* Array destructuring +* For..Of +* Symbol/Symbol.iterator +* Spread -Until these are polyfilled appropriately, they should not be used. Please +Until these are polyfilled appropriately, they should not be used. Please update this list with additional unsupported features. ### RSpec errors due to JavaScript @@ -223,7 +283,7 @@ end ### Spinach errors due to missing JavaScript NOTE: **Note:** Since we are discouraging the use of Spinach when writing new -feature tests, you shouldn't ever need to use this. This information is kept +feature tests, you shouldn't ever need to use this. This information is kept available for legacy purposes only. In Spinach, the JavaScript driver is enabled differently. In the `*.feature` @@ -243,11 +303,11 @@ Scenario: Developer can approve merge request [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html [jasmine-jquery]: https://github.com/velesin/jasmine-jquery [karma]: http://karma-runner.github.io/ -[vue-test]:https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components -[RSpec]: https://github.com/rspec/rspec-rails#feature-specs -[Capybara]: https://github.com/teamcapybara/capybara -[Karma]: http://karma-runner.github.io/ -[Jasmine]: https://jasmine.github.io/ +[vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components +[rspec]: https://github.com/rspec/rspec-rails#feature-specs +[capybara]: https://github.com/teamcapybara/capybara +[karma]: http://karma-runner.github.io/ +[jasmine]: https://jasmine.github.io/ --- diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index e86c1f5232a..51794f7f4df 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -28,7 +28,7 @@ records should use stubs/doubles as much as possible. | `app/uploaders/` | `spec/uploaders/` | RSpec | | | `app/views/` | `spec/views/` | RSpec | | | `app/workers/` | `spec/workers/` | RSpec | | -| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [Frontent Testing guide](frontend_testing.md) section. | +| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [Frontend Testing guide](frontend_testing.md) section. | ## Integration tests diff --git a/doc/development/testing_guide/testing_rake_tasks.md b/doc/development/testing_guide/testing_rake_tasks.md index 60163f1a230..db8ca87e9f8 100644 --- a/doc/development/testing_guide/testing_rake_tasks.md +++ b/doc/development/testing_guide/testing_rake_tasks.md @@ -9,7 +9,7 @@ At a minimum, requiring the Rake helper will redirect `stdout`, include the runtime task helpers, and include the `RakeHelpers` Spec support module. The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make -executing tasks simple. See `spec/support/rake_helpers.rb` for all available +executing tasks simple. See `spec/support/helpers/rake_helpers.rb` for all available methods. Example: diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md index 012c64be79f..b57520a00e0 100644 --- a/doc/development/ux_guide/components.md +++ b/doc/development/ux_guide/components.md @@ -219,7 +219,7 @@ Blocks are a way to group related information. #### Content blocks -Content blocks (`.content-block`) are the basic grouping of content. They are commonly used in [lists](#lists), and are separated by a botton border. +Content blocks (`.content-block`) are the basic grouping of content. They are commonly used in [lists](#lists), and are separated by a button border.  @@ -281,7 +281,7 @@ Modals are only used for having a conversation and confirmation with the user. T | Modal with 2 actions | Modal with 3 actions | Special confirmation | | --------------------- | --------------------- | -------------------- | -|  |  |  | +|  |  |  | > TODO: Special case for modal. diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index 9d0c62ecc35..b8be8daa157 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -255,7 +255,7 @@ otherwise it will raise a `TypeError`. ## Adding Indexes Adding indexes is an expensive process that blocks INSERT and UPDATE queries for -the duration. When using PostgreSQL one can work arounds this by using the +the duration. When using PostgreSQL one can work around this by using the `CONCURRENTLY` option: ```sql diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index d6a13e7483a..9bca4637830 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -49,7 +49,7 @@ do before. **Use cases**: provide at least two, ideally three, use cases for every major feature. You should answer this question: what can you do with this feature/change? Use cases -are examples of how this feauture or change can be used in real life. +are examples of how this feature or change can be used in real life. Examples: - CE and EE: [Issues](../user/project/issues/index.md#use-cases) diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md index 2a531193adf..c9766040234 100644 --- a/doc/gitlab-basics/command-line-commands.md +++ b/doc/gitlab-basics/command-line-commands.md @@ -71,7 +71,7 @@ rm NAME-OF-FILE ### Remove a directory and all of its contents ``` -rm -rf NAME-OF-DIRECTORY +rm -r NAME-OF-DIRECTORY ``` ### View history in the command line diff --git a/doc/install/README.md b/doc/install/README.md index 9724b56910d..5dadf57ea9a 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -31,8 +31,8 @@ the hardware requirements. - [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/) via [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/) - [Install GitLab on Azure](azure/index.md) - [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md) -- [Install GitLab on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/): video tutorial on -the full process of installing GitLab on Google Container Engine (GKE), pushing an application to GitLab, building the app with GitLab CI/CD, and deploying to production. +- [Install GitLab on Google Kubernetes Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/): video tutorial on +the full process of installing GitLab on Google Kubernetes Engine (GKE), pushing an application to GitLab, building the app with GitLab CI/CD, and deploying to production. - [Install on AWS](https://about.gitlab.com/aws/) - _Testing only!_ [DigitalOcean and Docker Machine](digitaloceandocker.md) - Quickly test any version of GitLab on DigitalOcean using Docker Machine. diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 5c7557ed2b3..e1af086f418 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -91,7 +91,7 @@ Follow the below instructions to ensure you use the most up to date requirements #### Check for InnoDB File-Per-Table Tablespaces -We need to check, enable and maybe convert your existing GitLab DB tables to the [InnoDB File-Per-Table Tablespaces](http://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) as a prerequise for supporting **utfb8mb4 with long indexes** required by recent GitLab databases. +We need to check, enable and maybe convert your existing GitLab DB tables to the [InnoDB File-Per-Table Tablespaces](http://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html) as a prerequisite for supporting **utfb8mb4 with long indexes** required by recent GitLab databases. # Login to MySQL mysql -u root -p diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 3389f0260f9..2691495e0d4 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -2,7 +2,7 @@  -Gettung started with GitLab on a [Google Cloud Platform (GCP)][gcp] instance is quick and easy. +Getting started with GitLab on a [Google Cloud Platform (GCP)][gcp] instance is quick and easy. ## Prerequisites diff --git a/doc/install/installation.md b/doc/install/installation.md index 1abbfd78738..fa5bcfa6f07 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -93,9 +93,9 @@ Is the system packaged Git too old? Remove it and compile from source. # Download and compile from source cd /tmp - curl --remote-name --progress https://www.kernel.org/pub/software/scm/git/git-2.16.2.tar.gz - echo '9acc4339b7a2ab484eea69d705923271682b7058015219cf5a7e6ed8dee5b5fb git-2.16.2.tar.gz' | shasum -a256 -c - && tar -xzf git-2.16.2.tar.gz - cd git-2.16.2/ + curl --remote-name --progress https://www.kernel.org/pub/software/scm/git/git-2.16.3.tar.gz + echo 'dda229e9c73f4fbb7d4324e0d993e11311673df03f73b194c554c2e9451e17cd git-2.16.3.tar.gz' | shasum -a256 -c - && tar -xzf git-2.16.3.tar.gz + cd git-2.16.3/ ./configure make prefix=/usr/local all @@ -133,9 +133,10 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz - echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz - cd ruby-2.3.6 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.7.tar.gz + echo '540996fec64984ab6099e34d2f5820b14904f15a ruby-2.3.7.tar.gz' | shasum -c - && tar xzf ruby-2.3.7.tar.gz + cd ruby-2.3.7 + ./configure --disable-install-rdoc make sudo make install @@ -300,7 +301,7 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-6-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-7-stable gitlab **Note:** You can change `10-6-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! diff --git a/doc/install/kubernetes/gitlab_chart.md b/doc/install/kubernetes/gitlab_chart.md index 84eeacac3fd..429519a92e1 100644 --- a/doc/install/kubernetes/gitlab_chart.md +++ b/doc/install/kubernetes/gitlab_chart.md @@ -1,488 +1,6 @@ # GitLab Helm Chart -> **Note:** -* This chart has been tested on Google Kubernetes Engine and Azure Container Service. +> **Note:** This chart is currently in alpha. -**This chart is deprecated.** For small installations on Kubernetes today, we recommend the beta [`gitlab-omnibus` Helm chart](gitlab_omnibus.md). +The cloud native `gitlab` chart is the next generation Helm chart, currently in alpha, and will replace the [`gitlab-omnibus`](gitlab_omnibus.md) chart. It will support large deployments with horizontal scaling of individual GitLab components. -A new [cloud native GitLab chart](index.md#cloud-native-gitlab-chart) is in development with increased scalability and resilience, among other benefits. The cloud native chart will replace both the `gitlab` and `gitlab-omnibus` charts when available later this year. - -Due to the significant architectural changes, migrating will require backing up data out of this instance and restoring it into the new deployment. For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). - -## Introduction - -The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. This chart requires advanced knowledge of Kubernetes to successfully use. We **strongly recommend** the [gitlab-omnibus](gitlab_omnibus.md) chart. - -This chart includes the following: - -- Deployment using the [gitlab-ce](https://hub.docker.com/r/gitlab/gitlab-ce) or [gitlab-ee](https://hub.docker.com/r/gitlab/gitlab-ee) container image -- ConfigMap containing the `gitlab.rb` contents that configure [Omnibus GitLab](https://docs.gitlab.com/omnibus/settings/configuration.html#configuration-options) -- Persistent Volume Claims for Data, Config, Logs, and Registry Storage -- A Kubernetes service -- Optional Redis deployment using the [Redis Chart](https://github.com/kubernetes/charts/tree/master/stable/redis) (defaults to enabled) -- Optional PostgreSQL deployment using the [PostgreSQL Chart](https://github.com/kubernetes/charts/tree/master/stable/postgresql) (defaults to enabled) -- Optional Ingress (defaults to disabled) - -## Prerequisites - -- _At least_ 3 GB of RAM available on your cluster. 41GB of storage and 2 CPU are also required. -- Kubernetes 1.4+ with Beta APIs enabled -- [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) provisioner support in the underlying infrastructure -- The ability to point a DNS entry or URL at your GitLab install -- The `kubectl` CLI installed locally and authenticated for the cluster -- The [Helm client](https://github.com/kubernetes/helm/blob/master/docs/quickstart.md) installed locally on your machine - -## Configuring GitLab - -Create a `values.yaml` file for your GitLab configuration. See the -[Helm docs](https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/values_files.md) -for information on how your values file will override the defaults. - -The default configuration can always be [found in the `values.yaml`](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab/values.yaml), in the chart repository. - -### Required configuration - -In order for GitLab to function, your config file **must** specify the following: - -- An `externalUrl` that GitLab will be reachable at. - -### Choosing GitLab Edition - -The Helm chart defaults to installing GitLab CE. This can be controlled by setting the `edition` variable in your values. - -Setting `edition` to GitLab Enterprise Edition (EE) in your `values.yaml` - -```yaml -edition: EE - -externalUrl: 'http://gitlab.example.com' -``` - -### Choosing a different GitLab release version - -The version of GitLab installed is based on the `edition` setting (see [section](#choosing-gitlab-edition) above), and -the value of the corresponding helm setting: `ceImage` or `eeImage`. - -```yaml -## GitLab Edition -## ref: https://about.gitlab.com/products/ -## - CE - Community Edition -## - EE - Enterprise Edition - (requires license issued by GitLab Inc) -## -edition: CE - -## GitLab CE image -## ref: https://hub.docker.com/r/gitlab/gitlab-ce/tags/ -## -ceImage: gitlab/gitlab-ce:9.1.2-ce.0 - -## GitLab EE image -## ref: https://hub.docker.com/r/gitlab/gitlab-ee/tags/ -## -eeImage: gitlab/gitlab-ee:9.1.2-ee.0 -``` - -The different images can be found in the [gitlab-ce](https://hub.docker.com/r/gitlab/gitlab-ce/tags/) and [gitlab-ee](https://hub.docker.com/r/gitlab/gitlab-ee/tags/) -repositories on Docker Hub - -> **Note:** -There is no guarantee that other release versions of GitLab, other than what are -used by default in the chart, will be supported by a chart install. - - -### Custom Omnibus GitLab configuration - -In addition to the configuration options provided for GitLab in the Helm Chart, you can also pass any custom configuration -that is valid for the [Omnibus GitLab Configuration](https://docs.gitlab.com/omnibus/settings/configuration.html). - -The setting to pass these values in is `omnibusConfigRuby`. It accepts any valid -Ruby code that could used in the Omnibus `/etc/gitlab/gitlab.rb` file. In -Kubernetes, the contents will be stored in a ConfigMap. - -Example setting: - -```yaml -omnibusConfigRuby: | - unicorn['worker_processes'] = 2; - gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"]; -``` - -### Persistent storage - -By default, persistent storage is enabled for GitLab and the charts it depends -on (Redis and PostgreSQL). - -Components can have their claim size set from your `values.yaml`, and each -component allows you to optionally configure the `storageClass` variable so you -can take advantage of faster drives on your cloud provider. - -Basic configuration: - -```yaml -## Enable persistence using Persistent Volume Claims -## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ -## ref: https://docs.gitlab.com/ce/install/requirements.html#storage -## -persistence: - ## This volume persists generated configuration files, keys, and certs. - ## - gitlabEtc: - enabled: true - size: 1Gi - ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass> - ## Default: volume.alpha.kubernetes.io/storage-class: default - ## - # storageClass: - accessMode: ReadWriteOnce - ## This volume is used to store git data and other project files. - ## ref: https://docs.gitlab.com/omnibus/settings/configuration.html#storing-git-data-in-an-alternative-directory - ## - gitlabData: - enabled: true - size: 10Gi - ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass> - ## Default: volume.alpha.kubernetes.io/storage-class: default - ## - # storageClass: - accessMode: ReadWriteOnce - gitlabRegistry: - enabled: true - size: 10Gi - ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass> - ## Default: volume.alpha.kubernetes.io/storage-class: default - ## - # storageClass: - - postgresql: - persistence: - # storageClass: - size: 10Gi - ## Configuration values for the Redis dependency. - ## ref: https://github.com/kubernetes/charts/blob/master/stable/redis/README.md - ## - redis: - persistence: - # storageClass: - size: 10Gi -``` - ->**Note:** -You can make use of faster SSD drives by adding a [StorageClass] to your cluster -and using the `storageClass` setting in the above config to the name of -your new storage class. - -### Routing - -By default, the GitLab chart uses a service type of `LoadBalancer` which will -result in the GitLab service being exposed externally using your cloud provider's -load balancer. - -This field is configurable in your `values.yml` by setting the top-level -`serviceType` field. See the [Service documentation][kube-srv] for more -information on the possible values. - -#### Ingress routing - -Optionally, you can enable the Chart's ingress for use by an ingress controller -deployed in your cluster. - -To enable the ingress, edit its section in your `values.yaml`: - -```yaml -ingress: - ## If true, gitlab Ingress will be created - ## - enabled: true - - ## gitlab Ingress hostnames - ## Must be provided if Ingress is enabled - ## - hosts: - - gitlab.example.com - - ## gitlab Ingress annotations - ## - annotations: - kubernetes.io/ingress.class: nginx -``` - -You must also provide the list of hosts that the ingress will use. In order for -you ingress controller to work with the GitLab Ingress, you will need to specify -its class in an annotation. - ->**Note:** -The Ingress alone doesn't expose GitLab externally. You need to have a Ingress controller setup to do that. -Setting up an Ingress controller can be done by installing the `nginx-ingress` helm chart. But be sure -to read the [documentation](https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/README.md). ->**Note:** -If you would like to use the Registry, you will also need to ensure your Ingress supports a [sufficiently large request size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size). - -#### Preserving Source IPs - -If you are using the `LoadBalancer` serviceType you may run into issues where user IP addresses in the GitLab -logs, and used in abuse throttling are not accurate. This is due to how Kubernetes uses source NATing on cluster nodes without endpoints. - -See the [Kubernetes documentation](https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typeloadbalancer) for more information. - -To fix this you can add the following service annotation to your `values.yaml` - -```yaml -## For minikube, set this to NodePort, elsewhere use LoadBalancer -## ref: http://kubernetes.io/docs/user-guide/services/#publishing-services---service-types -## -serviceType: LoadBalancer - -## Optional annotations for gitlab service. -serviceAnnotations: - service.beta.kubernetes.io/external-traffic: "OnlyLocal" -``` - ->**Note:** -If you are using the ingress routing, you will likely also need to specify the annotation on the service for the ingress -controller. For `nginx-ingress` you can check the -[configuration documentation](https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/README.md#configuration) -on how to add the annotation to the `controller.service.annotations` array. - ->**Note:** -When using the `nginx-ingress` controller on Google Kubernetes Engine (GKE), and using the `external-traffic` annotation, -you will need to additionally set the `controller.kind` to be DaemonSet. Otherwise only pods running on the same node -as the nginx controller will be able to reach GitLab. This may result in pods within your cluster not being able to reach GitLab. -See the [Kubernetes documentation](https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typeloadbalancer) and -[nginx-ingress configuration documentation](https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/README.md#configuration) -for more information. - -### External database - -You can configure the GitLab Helm chart to connect to an external PostgreSQL -database. - ->**Note:** -This is currently our recommended approach for a Production setup. - -To use an external database, in your `values.yaml`, disable the included -PostgreSQL dependency, then configure access to your database: - -```yaml -dbHost: "<reachable postgres hostname>" -dbPassword: "<password for the user with access to the db>" -dbUsername: "<user with read/write access to the database>" -dbDatabase: "<database name on postgres to connect to for GitLab>" - -postgresql: - # Sets whether the PostgreSQL helm chart is used as a dependency - enabled: false -``` - -Be sure to check the GitLab documentation on how to -[configure the external database](../requirements.md#postgresql-requirements) - -You can also configure the chart to use an external Redis server, but this is -not required for basic production use: - -```yaml -dbHost: "<reachable redis hostname>" -dbPassword: "<password>" - -redis: - # Sets whether the Redis helm chart is used as a dependency - enabled: false -``` - -### Sending email - -By default, the GitLab container will not be able to send email from your cluster. -In order to send email, you should configure SMTP settings in the -`omnibusConfigRuby` section, as per the [GitLab Omnibus documentation](https://docs.gitlab.com/omnibus/settings/smtp.html). - ->**Note:** -Some cloud providers restrict emails being sent out on SMTP, so you will have -to use a SMTP service that is supported by your provider. See this -[Google Cloud Platform page](https://cloud.google.com/compute/docs/tutorials/sending-mail/) -as and example. - -Here is an example configuration for Mailgun SMTP support: - -```yaml -omnibusConfigRuby: | - # This is example config of what you may already have in your omnibusConfigRuby object - unicorn['worker_processes'] = 2; - gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"]; - - # SMTP settings - gitlab_rails['smtp_enable'] = true - gitlab_rails['smtp_address'] = "smtp.mailgun.org" - gitlab_rails['smtp_port'] = 2525 # High port needed for Google Cloud - gitlab_rails['smtp_authentication'] = "plain" - gitlab_rails['smtp_enable_starttls_auto'] = false - gitlab_rails['smtp_user_name'] = "postmaster@mg.your-mail-domain" - gitlab_rails['smtp_password'] = "you-password" - gitlab_rails['smtp_domain'] = "mg.your-mail-domain" -``` - -### HTTPS configuration - -To setup HTTPS access to your GitLab server, first you need to configure the -chart to use the [ingress](#ingress-routing). - -GitLab's config should be updated to support [proxied SSL](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl). - -In addition to having a Ingress Controller deployed and the basic ingress -settings configured, you will also need to specify in the ingress settings -which hosts to use HTTPS for. - -Make sure `externalUrl` now includes `https://` instead of `http://` in its -value, and update the `omnibusConfigRuby` section: - -```yaml -externalUrl: 'https://gitlab.example.com' - -omnibusConfigRuby: | - # This is example config of what you may already have in your omnibusConfigRuby object - unicorn['worker_processes'] = 2; - gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"]; - - # These are the settings needed to support proxied SSL - nginx['listen_port'] = 80 - nginx['listen_https'] = false - nginx['proxy_set_headers'] = { - "X-Forwarded-Proto" => "https", - "X-Forwarded-Ssl" => "on" - } - -ingress: - enabled: true - annotations: - kubernetes.io/ingress.class: nginx - # kubernetes.io/tls-acme: 'true' Annotation used for letsencrypt support - - hosts: - - gitlab.example.com - - ## gitlab Ingress TLS configuration - ## Secrets must be created in the namespace, and is not done for you in this chart - ## - tls: - - secretName: gitlab-tls - hosts: - - gitlab.example.com -``` - -You will need to create the named secret in your cluster, specifying the private -and public certificate pair using the format outlined in the -[ingress documentation](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). - -Alternatively, you can use the `kubernetes.io/tls-acme` annotation, and install -the `kube-lego` chart to your cluster to have Let's Encrypt issue your -certificate. See the [kube-lego documentation](https://github.com/kubernetes/charts/blob/master/stable/kube-lego/README.md) -for more information. - -### Enabling the GitLab Container Registry - -The GitLab Registry is disabled by default but can be enabled by providing an -external URL for it in the configuration. In order for the Registry to be easily -used by GitLab CI and your Kubernetes cluster, you will need to set it up with -a TLS certificate, so these examples will include the ingress settings for that -as well. See the [HTTPS Configuration section](#https-configuration) -for more explanation on some of these settings. - -Example config: - -```yaml -externalUrl: 'https://gitlab.example.com' - -omnibusConfigRuby: | - # This is example config of what you may already have in your omnibusConfigRuby object - unicorn['worker_processes'] = 2; - gitlab_rails['trusted_proxies'] = ["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"]; - - registry_external_url 'https://registry.example.com'; - - # These are the settings needed to support proxied SSL - nginx['listen_port'] = 80 - nginx['listen_https'] = false - nginx['proxy_set_headers'] = { - "X-Forwarded-Proto" => "https", - "X-Forwarded-Ssl" => "on" - } - registry_nginx['listen_port'] = 80 - registry_nginx['listen_https'] = false - registry_nginx['proxy_set_headers'] = { - "X-Forwarded-Proto" => "https", - "X-Forwarded-Ssl" => "on" - } - -ingress: - enabled: true - annotations: - kubernetes.io/ingress.class: nginx - # kubernetes.io/tls-acme: 'true' Annotation used for letsencrypt support - - hosts: - - gitlab.example.com - - registry.example.com - - ## gitlab Ingress TLS configuration - ## Secrets must be created in the namespace, and is not done for you in this chart - ## - tls: - - secretName: gitlab-tls - hosts: - - gitlab.example.com - - registry.example.com -``` - -## Installing GitLab using the Helm Chart -> You may see a temporary error message `SchedulerPredicates failed due to PersistentVolumeClaim is not bound` while storage provisions. Once the storage provisions, the pods will automatically restart. This may take a couple minutes depending on your cloud provider. If the error persists, please review the [prerequisites](#prerequisites) to ensure you have enough RAM, CPU, and storage. - -Add the GitLab Helm repository and initialize Helm: - -```bash -helm repo add gitlab https://charts.gitlab.io -helm init -``` - -Once you [have configured](#configuration) GitLab in your `values.yml` file, -run the following: - -```bash -helm install --namespace <NAMESPACE> --name gitlab -f <CONFIG_VALUES_FILE> gitlab/gitlab -``` - -where: - -- `<NAMESPACE>` is the Kubernetes namespace where you want to install GitLab. -- `<CONFIG_VALUES_FILE>` is the path to values file containing your custom - configuration. See the [Configuration](#configuration) section to create it. - -## Updating GitLab using the Helm Chart - -Once your GitLab Chart is installed, configuration changes and chart updates -should we done using `helm upgrade` - -```bash -helm upgrade --namespace <NAMESPACE> -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab -``` - -where: - -- `<NAMESPACE>` is the Kubernetes namespace where GitLab is installed. -- `<CONFIG_VALUES_FILE>` is the path to values file containing your custom - [configuration] (#configuration). -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing) we called it `gitlab`. - -## Uninstalling GitLab using the Helm Chart - -To uninstall the GitLab Chart, run the following: - -```bash -helm delete --namespace <NAMESPACE> <RELEASE-NAME> -``` - -where: - -- `<NAMESPACE>` is the Kubernetes namespace where GitLab is installed. -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing) we called it `gitlab`. - -[kube-srv]: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types -[storageclass]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storageclasses +Installation instructions and known issues during alpha are available at the [project page](https://gitlab.com/charts/gitlab/).
\ No newline at end of file diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index 1f53e12d5f8..0a093c9ec32 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -50,12 +50,12 @@ Here is a snippet of the important settings: gitlabUrl: http://gitlab.your-domain.com/ ## The Registration Token for adding new Runners to the GitLab Server. This must -## be retreived from your GitLab Instance. +## be retrieved from your GitLab Instance. ## ref: https://docs.gitlab.com/ce/ci/runners/README.html#creating-and-registering-a-runner ## runnerRegistrationToken: "" -## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Set the certsSecretName in order to pass custom certificates for GitLab Runner to use ## Provide resource name for a Kubernetes Secret Object in the same namespace, ## this is used to populate the /etc/gitlab-runner/certs directory ## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates @@ -72,6 +72,18 @@ concurrent: 10 ## checkInterval: 30 +## For RBAC support: +rbac: + create: false + + ## Run the gitlab-bastion container with the ability to deploy/manage containers of jobs + ## cluster-wide or only within namespace + clusterWideAccess: false + + ## Use the following Kubernetes Service Account name if RBAC is disabled in this Helm chart (see rbac.create) + ## + # serviceAccountName: default + ## Configuration for the Pods that that the runner launches for each new job ## runners: @@ -80,7 +92,7 @@ runners: image: ubuntu:16.04 ## Run all containers with the privileged flag enabled - ## This will allow the docker:dind image to run if you need to run Docker + ## This will allow the docker:stable-dind image to run if you need to run Docker ## commands. Please read the docs before turning this on: ## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind ## @@ -116,6 +128,12 @@ runners: ``` +### Enabling RBAC support + +If your cluster has RBAC enabled, you can choose to either have the chart create its own service account or provide one. + +To have the chart create the service account for you, set `rbac.create` to true. + ### Controlling maximum Runner concurrency A single GitLab Runner deployed on Kubernetes is able to execute multiple jobs in parallel by automatically starting additional Runner pods. The [`concurrent` setting](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) controls the maximum number of pods allowed at a single time, and defaults to `10`. @@ -147,7 +165,7 @@ enable privileged mode in `values.yaml`: ```yaml runners: ## Run all containers with the privileged flag enabled - ## This will allow the docker:dind image to run if you need to run Docker + ## This will allow the docker:stable-dind image to run if you need to run Docker ## commands. Please read the docs before turning this on: ## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind ## @@ -190,7 +208,7 @@ You then need to provide the secret's name to the GitLab Runner chart. Add the following to your `values.yaml` ```yaml -## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Set the certsSecretName in order to pass custom certificates for GitLab Runner to use ## Provide resource name for a Kubernetes Secret Object in the same namespace, ## this is used to populate the /etc/gitlab-runner/certs directory ## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index aa9b8777359..7d8b8fc1597 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -10,10 +10,9 @@ should be deployed, upgraded, and configured. ## Chart Overview * **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). -* **[Cloud Native GitLab Chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md)**: The next generation GitLab chart, currently in alpha. Will support large deployments with horizontal scaling of individual GitLab components. +* **[Cloud Native GitLab Chart](https://gitlab.com/charts/gitlab/blob/master/README.md)**: The next generation GitLab chart, currently in alpha. Will support large deployments with horizontal scaling of individual GitLab components. * Other Charts * [GitLab Runner Chart](gitlab_runner_chart.md): For deploying just the GitLab Runner. - * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. ## GitLab-Omnibus Chart (Recommended) @@ -27,7 +26,7 @@ Learn more about the [gitlab-omnibus chart](gitlab_omnibus.md). ## Cloud Native GitLab Chart -GitLab is working towards building a [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into its [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current chart](#gitlab-omnibus-chart-recommended). +GitLab is working towards building a [cloud native GitLab chart](https://gitlab.com/charts/gitlab/blob/master/README.md). A key part of this effort is to isolate each service into its [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current chart](#gitlab-omnibus-chart-recommended). By offering individual containers and charts, we will be able to provide a number of benefits: * Easier horizontal scaling of each service, @@ -37,7 +36,7 @@ By offering individual containers and charts, we will be able to provide a numbe Presently this chart is available in alpha for testing, and not recommended for production use. -Learn more about the [cloud native GitLab chart here ](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) and [here [Video]](https://youtu.be/Z6jWR8Z8dv8). +Learn more about the [cloud native GitLab chart here ](https://gitlab.com/charts/gitlab/blob/master/README.md) and [here [Video]](https://youtu.be/Z6jWR8Z8dv8). ## Other Charts diff --git a/doc/integration/github.md b/doc/integration/github.md index b0d67db8b59..23bb8ef9303 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -69,7 +69,7 @@ GitHub will generate an application ID and secret key for you to use. "name" => "github", "app_id" => "YOUR_APP_ID", "app_secret" => "YOUR_APP_SECRET", - "url" => "https://github.com/", + "url" => "https://github.example.com/", "args" => { "scope" => "user:email" } } ] @@ -125,7 +125,7 @@ For omnibus package: "name" => "github", "app_id" => "YOUR_APP_ID", "app_secret" => "YOUR_APP_SECRET", - "url" => "https://github.com/", + "url" => "https://github.example.com/", "verify_ssl" => false, "args" => { "scope" => "user:email" } } diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md index e0fc1bb801f..8611d4f7315 100644 --- a/doc/integration/shibboleth.md +++ b/doc/integration/shibboleth.md @@ -43,7 +43,7 @@ exclude shibboleth URLs from rewriting, add "RewriteCond %{REQUEST_URI} !/Shibbo RequestHeader set X_FORWARDED_PROTO 'https' ``` -1. Edit /etc/gitlab/gitlab.rb configuration file, your shibboleth attributes should be in form of "HTTP_ATTRIBUTE" and you should addjust them to your need and environment. Add any other configuration you need. +1. Edit /etc/gitlab/gitlab.rb configuration file, your shibboleth attributes should be in form of "HTTP_ATTRIBUTE" and you should adjust them to your need and environment. Add any other configuration you need. File should look like this: ``` diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index bbd2d214fe4..785cc32d590 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -498,6 +498,13 @@ more of the following options: Read what the [backup timestamp is about](#backup-timestamp). - `force=yes` - Does not ask if the authorized_keys file should get regenerated and assumes 'yes' for warning that database tables will be removed. +If you are restoring into directories that are mountpoints you will need to make +sure these directories are empty before attempting a restore. Otherwise GitLab +will attempt to move these directories before restoring the new data and this +would cause an error. + +Read more on [configuring NFS mounts](../administration/high_availability/nfs.md) + ### Restore for installation from source ``` diff --git a/doc/security/img/outbound_requests_section.png b/doc/security/img/outbound_requests_section.png Binary files differnew file mode 100644 index 00000000000..95c9c6ee771 --- /dev/null +++ b/doc/security/img/outbound_requests_section.png diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index faabc53ce72..a573445ab5b 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -2,12 +2,19 @@ If you have non-GitLab web services running on your GitLab server or within its local network, these may be vulnerable to exploitation via Webhooks. -With [Webhooks](../user/project/integrations/webhooks.md), you and your project masters and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. +With [Webhooks](../user/project/integrations/webhooks.md), you and your project masters and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. Things get hairy, however, when a Webhook is set up with a URL that doesn't point to an external, but to an internal service, that may do something completely unintended when the webhook is triggered and the POST request is sent. Because Webhook requests are made by the GitLab server itself, these have complete access to everything running on the server (http://localhost:123) or within the server's local network (http://192.168.1.12:345), even if these services are otherwise protected and inaccessible from the outside world. -If a web service does not require authentication, Webhooks can be used to trigger destructive commands by getting the GitLab server to make POST requests to endpoints like "http://localhost:123/some-resource/delete". +If a web service does not require authentication, Webhooks can be used to trigger destructive commands by getting the GitLab server to make POST requests to endpoints like "http://localhost:123/some-resource/delete". -To prevent this type of exploitation from happening, make sure that you are aware of every web service GitLab could potentially have access to, and that all of these are set up to require authentication for every potentially destructive command. Enabling authentication but leaving a default password is not enough. +To prevent this type of exploitation from happening, starting with GitLab 10.6, all Webhook requests to the current GitLab instance server address and/or in a private network will be forbidden by default. That means that all requests made to 127.0.0.1, ::1 and 0.0.0.0, as well as IPv4 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and IPv6 site-local (ffc0::/10) addresses won't be allowed. + +This behavior can be overridden by enabling the option *"Allow requests to the local network from hooks and services"* in the *"Outbound requests"* section inside the Admin area under **Settings** (`/admin/application_settings`): + + + +>**Note:** +*System hooks* are exempt from this protection because they are set up by admins. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index aa14a39e4c9..b71e9bf3000 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -196,7 +196,7 @@ This is really useful for integrating repositories to secured, shared Continuous Integration (CI) services or other shared services. GitLab administrators can set up the Global Shared Deploy key in GitLab and add the private key to any shared systems. Individual repositories opt into -exposing their repsitory using these keys when a project masters (or higher) +exposing their repository using these keys when a project masters (or higher) authorizes a Global Shared Deploy key to be used with their project. Global Shared Keys can provide greater security compared to Per-Project Deploy @@ -224,7 +224,7 @@ if there is at least one Global Deploy Key configured. CAUTION: **Warning:** Defining Global Deploy Keys does not expose any given repository via -the key until that respository adds the Global Deploy Key to their project. +the key until that repository adds the Global Deploy Key to their project. In this way the Global Deploy Keys enable access by other systems, but do not implicitly give any access just by setting them up. diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index e88b787187c..882ddf4d2c5 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -10,8 +10,30 @@ applications. ## Overview With Auto DevOps, the software development process becomes easier to set up -as every project can have a complete workflow from build to deploy and monitoring, -with minimal to zero configuration. +as every project can have a complete workflow from verification to monitoring +without needing to configure anything. Just push your code and GitLab takes +care of everything else. This makes it easier to start new projects and brings +consistency to how applications are set up throughout a company. + +## Comparison to application platforms and PaaS + +Auto DevOps provides functionality described by others as an application +platform or as a Platform as a Service (PaaS). It takes inspiration from the +innovative work done by [Heroku](https://www.heroku.com/) and goes beyond it +in a couple of ways: + +1. Auto DevOps works with any Kubernetes cluster, you're not limited to running + on GitLab's infrastructure (note that many features also work without Kubernetes). +1. There is no additional cost (no markup on the infrastructure costs), and you + can use a self-hosted Kubernetes cluster or Containers as a Service on any + public cloud (for example [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/)). +1. Auto DevOps has more features including security testing, performance testing, + and code quality testing. +1. It offers an incremental graduation path. If you need advanced customizations + you can start modifying the templates without having to start over on a + completely different platform. + +## Features Comprised of a set of stages, Auto DevOps brings these best practices to your project in an easy and automatic way: @@ -113,6 +135,11 @@ and `1.2.3.4` is the IP address of your load balancer; generally NGINX ([see prerequisites](#prerequisites)). How to set up the DNS record is beyond the scope of this document; you should check with your DNS provider. +Alternatively you can use free public services like [xip.io](http://xip.io) or +[nip.io](http://nip.io) which provide automatic wildcard DNS without any +configuration. Just set the Auto DevOps base domain to `1.2.3.4.xip.io` or +`1.2.3.4.nip.io`. + Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). @@ -383,12 +410,12 @@ into your project to enable staging and canary deployments, and more. ### Custom buildpacks If the automatic buildpack detection fails for your project, or if you want to -use a custom buildpack, you can override the buildpack using a project variable -or a `.buildpack` file in your project: +use a custom buildpack, you can override the buildpack(s) using a project variable +or a `.buildpacks` file in your project: - **Project variable** - Create a project variable `BUILDPACK_URL` with the URL of the buildpack to use. -- **`.buildpack` file** - Add a file in your project's repo called `.buildpack` +- **`.buildpacks` file** - Add a file in your project's repo called `.buildpacks` and add the URL of the buildpack to use on a line in the file. If you want to use multiple buildpacks, you can enter them in, one on each line. @@ -455,17 +482,20 @@ The following variables can be used for setting up the Auto DevOps domain, providing a custom Helm chart, or scaling your application. PostgreSQL can be also be customized, and you can easily use a [custom buildpack](#custom-buildpacks). -| **Variable** | **Description** | -| ------------ | --------------- | -| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-domain); by default set automatically by the [Auto DevOps setting](#enabling-auto-devops). | -| `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). | -| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment; defaults to 1. | -| `CANARY_PRODUCTION_REPLICAS`| The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. | -| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | -| `POSTGRES_USER` | The PostgreSQL user; defaults to `user`. Set it to use a custom username. | -| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | -| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | -| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142`| +| **Variable** | **Description** | +| ------------ | --------------- | +| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-domain); by default set automatically by the [Auto DevOps setting](#enabling-auto-devops). | +| `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). | +| `REPLICAS` | The number of replicas to deploy; defaults to 1. | +| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. This takes precedence over `REPLICAS`; defaults to 1. | +| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html); defaults to 1 | +| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | +| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | +| `POSTGRES_USER` | The PostgreSQL user; defaults to `user`. Set it to use a custom username. | +| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | +| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | +| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142` | +| `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | TIP: **Tip:** Set up the replica variables using a @@ -496,8 +526,9 @@ The general rule is: `TRACK_ENV_REPLICAS`. Where: That way, you can define your own `TRACK_ENV_REPLICAS` variables with which you will be able to scale the pod's replicas easily. -In the example below, the environment's name is `qa` which would result in -looking for the `QA_REPLICAS` environment variable: +In the example below, the environment's name is `qa` and it deploys the track +`foo` which would result in looking for the `FOO_QA_REPLICAS` environment +variable: ```yaml QA testing: @@ -505,11 +536,11 @@ QA testing: environment: name: qa script: - - deploy qa + - deploy foo ``` -If, in addition, there was also a `track: foo` defined in the application's Helm -chart, like: +The track `foo` being referenced would also need to be defined in the +application's Helm chart, like: ```yaml replicaCount: 1 @@ -531,7 +562,21 @@ service: internalPort: 5000 ``` -then the environment variable would be `FOO_QA_REPLICAS`. +#### Deploy policy for staging and production environments + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/160) +in GitLab 10.8. + +The normal behavior of Auto DevOps is to use Continuous Deployment, pushing +automatically to the `production` environment every time a new pipeline is run +on the default branch. However, there are cases where you might want to use a +staging environment and deploy to production manually. For this scenario, the +`STAGING_ENABLED` environment variable was introduced. + +If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to +`1` as a secret variable), then the application will be automatically deployed +to a `staging` environment, and a `production_manual` job will be created for +you when you're ready to manually deploy to production. ## Currently supported languages diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index a9ccbf5a085..945d6a578b0 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -89,7 +89,7 @@ A [copy](https://git-scm.com/docs/git-clone) of a repository stored on your mach ### Code Review -Examination of a progam's code. The main aim is to maintain high quality standards of code that is being shipped. Merge requests [serve as a code review tool](https://about.gitlab.com/2014/09/29/gitlab-flow/) in GitLab. +Examination of a program's code. The main aim is to maintain high quality standards of code that is being shipped. Merge requests [serve as a code review tool](https://about.gitlab.com/2014/09/29/gitlab-flow/) in GitLab. ### Code Snippet diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 47ccd0e6dbc..f340164b882 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -354,11 +354,11 @@ add the following script to the User Data section: - mount -a -t nfs - sudo gitlab-ctl reconfigure -On the security group section we can chosse our existing +On the security group section we can choose our existing `gitlab-ec2-security-group` group which has already been tested. After this is launched we are able to start creating our Auto Scaling -Group. Start by giving it a name and assinging it our VPC and private +Group. Start by giving it a name and assigning it our VPC and private subnets. We also want to always start with two instances and if you scroll down to Advanced Details we can choose to receive traffic from ELBs. Lets enable that option and select our ELB. We also want to use the ELB's diff --git a/doc/university/support/README.md b/doc/university/support/README.md index 25d5fe351ca..d1d5db6bbcd 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -163,7 +163,7 @@ Some tickets need specific knowledge or a deep understanding of a particular com - Aim to have a good understanding of the problems that customers are facing - Aim to have gained experience in scheduling and participating in calls with customers -- Aim to have a good understanding of ticket flow through Zendesk and how to interat with our various channels +- Aim to have a good understanding of ticket flow through Zendesk and how to interact with our various channels ### Stage 4 diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index a882bf0eb48..9b8a8db58e2 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -27,7 +27,7 @@ project. ### Short Story of Git -- 1991-2002: The Linux kernel was being maintaned by sharing archived files +- 1991-2002: The Linux kernel was being maintained by sharing archived files and patches. - 2002: The Linux kernel project began using a DVCS called BitKeeper - 2005: BitKeeper revoked the free-of-charge status and Git was created diff --git a/doc/university/training/topics/tags.md b/doc/university/training/topics/tags.md index ab48d52d3c3..6333ceedbd7 100644 --- a/doc/university/training/topics/tags.md +++ b/doc/university/training/topics/tags.md @@ -9,7 +9,7 @@ comments: false - Useful for marking deployments and releases - Annotated tags are an unchangeable part of Git history - Soft/lightweight tags can be set and removed at will -- Many projects combine an anotated release tag with a stable branch +- Many projects combine an annotated release tag with a stable branch - Consider setting deployment/release tags automatically ---------- diff --git a/doc/university/training/user_training.md b/doc/university/training/user_training.md index 90e1d2ba5e8..dccb6cbf071 100644 --- a/doc/university/training/user_training.md +++ b/doc/university/training/user_training.md @@ -279,7 +279,7 @@ See GitLab merge requests for examples: - Useful for marking deployments and releases - Annotated tags are an unchangeable part of Git history - Soft/lightweight tags can be set and removed at will -- Many projects combine an anotated release tag with a stable branch +- Many projects combine an annotated release tag with a stable branch - Consider setting deployment/release tags automatically --- diff --git a/doc/update/10.6-to-10.7.md b/doc/update/10.6-to-10.7.md new file mode 100644 index 00000000000..4a76ae14d2e --- /dev/null +++ b/doc/update/10.6-to-10.7.md @@ -0,0 +1,361 @@ +--- +comments: false +--- + +# From 10.6 to 10.7 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz +echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz +cd ruby-2.3.6 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab utilizes [webpack](http://webpack.js.org) to compile frontend assets. +This requires a minimum version of node v6.0.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v6.0.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + +GitLab also requires the use of yarn `>= v1.2.0` to manage JavaScript +dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all --prune +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-7-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-7-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-6-stable:config/gitlab.yml.example origin/10-7-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-6-stable:lib/support/nginx/gitlab-ssl origin/10-7-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-6-stable:lib/support/nginx/gitlab origin/10-7-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-7-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-7-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-6-stable:lib/support/init.d/gitlab.default.example origin/10-7-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.5) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.5 to 10.6](10.5-to-10.6.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-7-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-7-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md new file mode 100644 index 00000000000..7c9e5bf882e --- /dev/null +++ b/doc/user/admin_area/settings/email.md @@ -0,0 +1,5 @@ +# Email + +## Custom logo + +The logo in the header of some emails can be customized, see the [logo customization section](../../../customization/branded_page_and_email_header.md). diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 603b826e7f2..26329f20339 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -1,7 +1,7 @@ # Sign-up restrictions You can block email addresses of specific domains, or whitelist only some -specifc domains via the **Application Settings** in the Admin area. +specific domains via the **Application Settings** in the Admin area. >**Note**: These restrictions are only applied during sign-up. An admin is able to add add a user through the admin panel with a disallowed domain. Also diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index 633f16a617c..3d38588a9ed 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -32,9 +32,15 @@ When you choose to allow only one of the protocols, a couple of things will happ On top of these UI restrictions, GitLab will deny all Git actions on the protocol not selected. +CAUTION: **Important:** +Starting with [GitLab 10.7][ce-18021], HTTP(s) protocol will be allowed for +git clone/fetch requests done by GitLab Runner from CI/CD Jobs, even if +_Only SSH_ was selected. + > **Note:** Please keep in mind that disabling an access protocol does not actually - block access to the server itself. The ports used for the protocol, be it SSH or - HTTP, will still be accessible. What GitLab does is restrict access on the - application level. +block access to the server itself. The ports used for the protocol, be it SSH or +HTTP, will still be accessible. What GitLab does is restrict access on the +application level. [ce-4696]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4696 +[ce-18021]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18021 diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index eacfe2baa27..159109e8954 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -14,6 +14,10 @@ The comment area supports [Markdown] and [quick actions]. One can edit their own comment at any time, and anyone with [Master access level][permissions] or higher can also edit a comment made by someone else. +You could also reply to the notification email in order to reply to a comment, +provided that [Reply by email] is configured by your GitLab admin. This also +supports [Markdown] and [quick actions] as if replied from the web. + Apart from the standard comments, you also have the option to create a comment in the form of a resolvable or threaded discussion. @@ -283,3 +287,4 @@ edit existing comments. Non-team members are restricted from adding or editing c [markdown]: ../markdown.md [quick actions]: ../project/quick_actions.md [permissions]: ../permissions.md +[Reply by email]: ../../administration/reply_by_email.md diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index d5f77191938..7baccb796c6 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -72,15 +72,23 @@ The maximum size your Git repository is allowed to be including LFS. ## Shared Runners Shared Runners on GitLab.com run in [autoscale mode] and powered by -DigitalOcean. Autoscaling means reduced waiting times to spin up builds, -and isolated VMs for each project, thus maximizing security. +Google Cloud Platform and DigitalOcean. Autoscaling means reduced +waiting times to spin up CI/CD jobs, and isolated VMs for each project, +thus maximizing security. They're free to use for public open source projects and limited to 2000 CI minutes per month per group for private projects. Read about all [GitLab.com plans](https://about.gitlab.com/pricing/). -All your builds run on 2GB (RAM) ephemeral instances, with CoreOS and the latest -Docker Engine installed. The default region of the VMs is NYC. +In case of DigitalOcean based Runners, all your CI/CD jobs run on ephemeral +instances with 2GB of RAM, CoreOS and the latest Docker Engine installed. +Instances provide 2 vCPUs and 60GB of SSD disk space. The default region of the +VMs is NYC1. + +In case of Google Cloud Platform based Runners, all your CI/CD jobs run on +ephemeral instances with 3.75GB of RAM, CoreOS and the latest Docker Engine +installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default +region of the VMs is US East1. Below are the shared Runners settings. @@ -88,52 +96,116 @@ Below are the shared Runners settings. | ----------- | ----------------- | ---------- | | [GitLab Runner] | [Runner versions dashboard][ci_version_dashboard] | - | | Executor | `docker+machine` | - | -| Default Docker image | `ruby:2.1` | - | +| Default Docker image | `ruby:2.5` | - | | `privileged` (run [Docker in Docker]) | `true` | `false` | -[ci_version_dashboard]: https://monitor.gitlab.net/dashboard/db/ci?refresh=5m&orgId=1&panelId=12&fullscreen&from=now-1h&to=now&var-runner_type=All&var-cache_server=All&var-gl_monitor_fqdn=postgres-01.db.prd.gitlab.com&var-has_minutes=yes&var-hanging_droplets_cleaner=All&var-droplet_zero_machines_cleaner=All&var-runner_job_failure_reason=All&theme=light +[ci_version_dashboard]: https://monitor.gitlab.net/dashboard/db/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light ### `config.toml` The full contents of our `config.toml` are: +**DigitalOcean** + ```toml +concurrent = X +check_interval = 1 +metrics_server = "X" +sentry_dsn = "X" + [[runners]] name = "docker-auto-scale" - limit = X request_concurrency = X - url = "https://gitlab.com/ci" + url = "https://gitlab.com/" token = "SHARED_RUNNER_TOKEN" executor = "docker+machine" environment = [ "DOCKER_DRIVER=overlay2" ] + limit = X [runners.docker] - image = "ruby:2.1" + image = "ruby:2.5" privileged = true [runners.machine] - IdleCount = 40 + IdleCount = 20 IdleTime = 1800 + OffPeakPeriods = ["* * * * * sat,sun *"] + OffPeakTimezone = "UTC" + OffPeakIdleCount = 5 + OffPeakIdleTime = 1800 MaxBuilds = 1 + MachineName = "srm-%s" MachineDriver = "digitalocean" - MachineName = "machine-%s-digital-ocean-2gb" MachineOptions = [ - "digitalocean-image=coreos-stable", + "digitalocean-image=X", "digitalocean-ssh-user=core", - "digitalocean-access-token=DIGITAL_OCEAN_ACCESS_TOKEN", "digitalocean-region=nyc1", - "digitalocean-size=2gb", + "digitalocean-size=s-2vcpu-2gb", "digitalocean-private-networking", - "digitalocean-userdata=/etc/gitlab-runner/cloudinit.sh", - "engine-registry-mirror=http://IP_TO_OUR_REGISTRY_MIRROR" + "digitalocean-tags=shared_runners,gitlab_com", + "engine-registry-mirror=http://INTERNAL_IP_OF_OUR_REGISTRY_MIRROR", + "digitalocean-access-token=DIGITAL_OCEAN_ACCESS_TOKEN", ] [runners.cache] Type = "s3" - ServerAddress = "IP_TO_OUR_CACHE_SERVER" + BucketName = "runner" + Insecure = true + Shared = true + ServerAddress = "INTERNAL_IP_OF_OUR_CACHE_SERVER" AccessKey = "ACCESS_KEY" SecretKey = "ACCESS_SECRET_KEY" +``` + +**Google Cloud Platform** + +```toml +concurrent = X +check_interval = 1 +metrics_server = "X" +sentry_dsn = "X" + +[[runners]] + name = "docker-auto-scale" + request_concurrency = X + url = "https://gitlab.com/" + token = "SHARED_RUNNER_TOKEN" + executor = "docker+machine" + environment = [ + "DOCKER_DRIVER=overlay2" + ] + limit = X + [runners.docker] + image = "ruby:2.5" + privileged = true + [runners.machine] + IdleCount = 20 + IdleTime = 1800 + OffPeakPeriods = ["* * * * * sat,sun *"] + OffPeakTimezone = "UTC" + OffPeakIdleCount = 5 + OffPeakIdleTime = 1800 + MaxBuilds = 1 + MachineName = "srm-%s" + MachineDriver = "google" + MachineOptions = [ + "google-project=PROJECT", + "google-disk-size=25", + "google-machine-type=n1-standard-1", + "google-username=core", + "google-tags=gitlab-com,srm", + "google-use-internal-ip", + "google-zone=us-east1-d", + "google-machine-image=PROJECT/global/images/IMAGE", + "engine-registry-mirror=http://INTERNAL_IP_OF_OUR_REGISTRY_MIRROR" + ] + [runners.cache] + Type = "s3" BucketName = "runner" + Insecure = true Shared = true + ServerAddress = "INTERNAL_IP_OF_OUR_CACHE_SERVER" + AccessKey = "ACCESS_KEY" + SecretKey = "ACCESS_SECRET_KEY" ``` ## Sidekiq diff --git a/doc/user/group/img/groups.png b/doc/user/group/img/groups.png Binary files differindex 6211f999d5e..3173ddce7ff 100644 --- a/doc/user/group/img/groups.png +++ b/doc/user/group/img/groups.png diff --git a/doc/user/group/img/new_group_from_groups.png b/doc/user/group/img/new_group_from_groups.png Binary files differindex baf34244cb2..9c5dd7ebd8b 100644 --- a/doc/user/group/img/new_group_from_groups.png +++ b/doc/user/group/img/new_group_from_groups.png diff --git a/doc/user/group/img/new_group_from_other_pages.png b/doc/user/group/img/new_group_from_other_pages.png Binary files differindex 014a7088af2..77427224447 100644 --- a/doc/user/group/img/new_group_from_other_pages.png +++ b/doc/user/group/img/new_group_from_other_pages.png diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 88efddbfba8..88f4bb2ee04 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -245,10 +245,7 @@ To enable this feature, navigate to the group settings page. Select  -#### Member Lock - -> Available in [GitLab Starter](https://about.gitlab.com/products/) and -[GitLab.com Bronze](https://about.gitlab.com/gitlab-com/). +#### Member Lock **[STARTER]** With **Member Lock** it is possible to lock membership in project to the level of members in group. @@ -259,8 +256,8 @@ Learn more about [Member Lock](https://docs.gitlab.com/ee/user/group/index.html# - **Projects**: view all projects within that group, add members to each project, access each project's settings, and remove any project from the same screen. -- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) -and [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html#push-rules) to your group (Push Rules is available in [GitLab Starter](https://about.gitlab.com/products/).) +- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) to your group. +- **Push rules**: configure [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html#push-rules) to your group. **[STARTER]** - **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html#audit-events) -for the group (GitLab admins only, available in [GitLab Starter][ee]). +for the group. **[STARTER ONLY]** - **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 2a982344e5f..02f8ef08117 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -55,7 +55,7 @@ first group being the name of the distro and subsequent groups split like: Another example of GitLab as a company would be the following: - Organization Group - GitLab - - Category Subroup - Marketing + - Category Subgroup - Marketing - (project) Design - (project) General - Category Subgroup - Software diff --git a/doc/user/index.md b/doc/user/index.md index 43b6fd53b91..2494df46f1c 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -56,7 +56,7 @@ With GitLab Enterprise Edition, you can also: [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#merge-request-approvals), [Multiple Assignees for Issues](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html), and [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) -- Create formal relashionships between issues with [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) +- Create formal relationships between issues with [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) - Use [Burndown Charts](https://docs.gitlab.com/ee/user/project/milestones/burndown_charts.html) to track progress during a sprint or while working on a new version of their software. - Leverage [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html) with [Advanced Global Search](https://docs.gitlab.com/ee/user/search/advanced_global_search.html) and [Advanced Syntax Search](https://docs.gitlab.com/ee/user/search/advanced_search_syntax.html) for faster, more advanced code search across your entire GitLab instance - [Authenticate users with Kerberos](https://docs.gitlab.com/ee/integration/kerberos.html) diff --git a/doc/user/permissions.md b/doc/user/permissions.md index a520279c29e..a9ba2a51242 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -15,6 +15,10 @@ GitLab [administrators](../README.md#administrator-documentation) receive all pe To add or import a user, you can follow the [project members documentation](../user/project/members/index.md). +## Principles behind permissions + +See our [product handbook on permissions](https://about.gitlab.com/handbook/product#permissions-in-gitlab) + ## Project members permissions The following table depicts the various user permission levels in a project. diff --git a/doc/user/profile/active_sessions.md b/doc/user/profile/active_sessions.md new file mode 100644 index 00000000000..5119c0e30d0 --- /dev/null +++ b/doc/user/profile/active_sessions.md @@ -0,0 +1,20 @@ +# Active Sessions + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17867) +> in GitLab 10.8. + +GitLab lists all devices that have logged into your account. This allows you to +review the sessions and revoke any of it that you don't recognize. + +## Listing all active sessions + +1. On the upper right corner, click on your avatar and go to your **Settings**. +1. Navigate to the **Active Sessions** tab. + + + +## Revoking a session + +1. Navigate to your [profile's](#profile-settings) **Settings > Active Sessions**. +1. Click on **Revoke** besides a session. The current session cannot be + revoked, as this would sign you out of GitLab. diff --git a/doc/user/profile/img/active_sessions_list.png b/doc/user/profile/img/active_sessions_list.png Binary files differnew file mode 100644 index 00000000000..76a52220bcd --- /dev/null +++ b/doc/user/profile/img/active_sessions_list.png diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index ab16f8d14c1..91cdef8d1dd 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -39,6 +39,7 @@ From there, you can: - Manage [SSH keys](../../ssh/README.md#ssh) to access your account via SSH - Manage your [preferences](preferences.md#syntax-highlighting-theme) to customize your own GitLab experience +- [View your active sessions](active_sessions.md) and revoke any of them if necessary - Access your audit log, a security log of important events involving your account ## Changing your username diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md new file mode 100644 index 00000000000..c4e59444ef7 --- /dev/null +++ b/doc/user/project/badges.md @@ -0,0 +1,73 @@ +# Badges + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/41174) +in GitLab 10.7. + +Badges are a unified way to present condensed pieces of information about your +projects. They consist of a small image and additionally a URL that the image +points to. Examples for badges can be the [pipeline status], [test coverage], +or ways to contact the project maintainers. + + + +## Project badges + +Badges can be added to a project and will then be visible on the project's overview page. +If you find that you have to add the same badges to several projects, you may want to add them at the [group level](#group-badges). + +To add a new badge to a project: + +1. Navigate to your project's **Settings > Badges**. +1. Under "Link", enter the URL that the badges should point to and under + "Badge image URL" the URL of the image that should be displayed. +1. Submit the badge by clicking the **Add badge** button. + +After adding a badge to a project, you can see it in the list below the form. +You can edit it by clicking on the pen icon next to it or to delete it by +clicking on the trash icon. + +Badges associated with a group can only be edited or deleted on the +[group level](#group-badges). + +## Group badges + +Badges can be added to a group and will then be visible on every project's +overview page that's under that group. In this case, they cannot be edited or +deleted on the project level. If you need to have individual badges for each +project, consider adding them on the [project level](#project-badges) or use +[placeholders](#placeholders). + +To add a new badge to a group: + +1. Navigate to your group's **Settings > Project Badges**. +1. Under "Link", enter the URL that the badges should point to and under + "Badge image URL" the URL of the image that should be displayed. +1. Submit the badge by clicking the **Add badge** button. + +After adding a badge to a group, you can see it in the list below the form. +You can edit the badge by clicking on the pen icon next to it or to delete it +by clicking on the trash icon. + +Badges directly associated with a project can be configured on the +[project level](#project-badges). + +## Placeholders + +The URL a badge points to, as well as the image URL, can contain placeholders +which will be evaluated when displaying the badge. The following placeholders +are available: + +- `%{project_path}`: Path of a project including the parent groups +- `%{project_id}`: Database ID associated with a project +- `%{default_branch}`: Default branch name configured for a project's repository +- `%{commit_sha}`: ID of the most recent commit to the default branch of a + project's repository + +## API + +You can also configure badges via the GitLab API. As in the settings, there is +a distinction between endpoints for badges on the +[project level](../../api/project_badges.md) and [group level](../../api/group_badges.md). + +[pipeline status]: pipelines/settings.md#pipeline-status-badge +[test coverage]: pipelines/settings.md#test-coverage-report-badge diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 716787532fc..edb875bc7e6 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -238,6 +238,7 @@ work. The default environment scope is `*`, which means all jobs, regardless of their environment, will use that cluster. Each scope can only be used by a single cluster in a project, and a validation error will occur if otherwise. +Also, jobs that don't have an environment keyword set will not be able to access any cluster. --- diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 394aa9209e4..9c5e3509046 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -115,15 +115,16 @@ and [Using the GitLab Container Registry documentation](../../ci/docker/using_do ## Using with private projects -> [Introduced][ce-11845] in GitLab 9.3. +> Personal Access tokens were [introduced][ce-11845] in GitLab 9.3. +> Project Deploy Tokens were [introduced][ce-17894] in GitLab 10.7 If a project is private, credentials will need to be provided for authorization. -The preferred way to do this, is by using [personal access tokens][pat]. -The minimal scope needed is `read_registry`. +The preferred way to do this, is either by using a [personal access tokens][pat] or a [project deploy token][pdt]. +The minimal scope needed for both of them is `read_registry`. Example of using a personal access token: ``` -docker login registry.example.com -u <your_username> -p <your_personal_access_token> +docker login registry.example.com -u <your_username> -p <your_access_token> ``` ## Troubleshooting the GitLab Container Registry @@ -270,5 +271,7 @@ Once the right permissions were set, the error will go away. [ce-4040]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4040 [ce-11845]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845 +[ce-17894]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17894 [docker-docs]: https://docs.docker.com/engine/userguide/intro/ [pat]: ../profile/personal_access_tokens.md +[pdt]: ../project/deploy_tokens/index.md diff --git a/doc/user/project/deploy_tokens/img/deploy_tokens.png b/doc/user/project/deploy_tokens/img/deploy_tokens.png Binary files differnew file mode 100644 index 00000000000..7e2d67a3120 --- /dev/null +++ b/doc/user/project/deploy_tokens/img/deploy_tokens.png diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md new file mode 100644 index 00000000000..7a8b3c75690 --- /dev/null +++ b/doc/user/project/deploy_tokens/index.md @@ -0,0 +1,86 @@ +# Deploy Tokens + +> [Introduced][ce-17894] in GitLab 10.7. + +Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. + +Please note, that the expiration of deploy tokens happens on the date you define, +at midnight UTC and that they can be only managed by [masters](https://docs.gitlab.com/ee/user/permissions.html). + +## Creating a Deploy Token + +You can create as many deploy tokens as you like from the settings of your project: + +1. Log in to your GitLab account. +1. Go to the project you want to create Deploy Tokens for. +1. Go to **Settings** > **Repository** +1. Click on "Expand" on **Deploy Tokens** section +1. Choose a name and optionally an expiry date for the token. +1. Choose the [desired scopes](#limiting-scopes-of-a-deploy-token). +1. Click on **Create deploy token**. +1. Save the deploy token somewhere safe. Once you leave or refresh + the page, **you won't be able to access it again**. + + + +## Revoking a deploy token + +At any time, you can revoke any deploy token by just clicking the +respective **Revoke** button under the 'Active deploy tokens' area. + +## Limiting scopes of a deploy token + +Deploy tokens can be created with two different scopes that allow various +actions that a given token can perform. The available scopes are depicted in +the following table. + +| Scope | Description | +| ----- | ----------- | +| `read_repository` | Allows read-access to the repository through `git clone` | +| `read_registry` | Allows read-access to [container registry] images if a project is private and authorization is required. | + +## Usage + +### Git clone a repository + +To download a repository using a Deploy Token, you just need to: + +1. Create a Deploy Token with `read_repository` as a scope. +2. Take note of your `username` and `token` +3. `git clone` the project using the Deploy Token: + + +```bash +git clone http://<username>:<deploy_token>@gitlab.example.com/tanuki/awesome_project.git +``` + +Just replace `<username>` and `<deploy_token>` with the proper values + +### Read container registry images + +To read the container registry images, you'll need to: + +1. Create a Deploy Token with `read_registry` as a scope. +2. Take note of your `username` and `token` +3. Log in to GitLab’s Container Registry using the deploy token: + +``` +docker login registry.example.com -u <username> -p <deploy_token> +``` + +Just replace `<username>` and `<deploy_token>` with the proper values. Then you can simply +pull images from your Container Registry. + +### GitLab Deploy Token + +> [Introduced][ce-18414] in GitLab 10.8. + +There's a special case when it comes to Deploy Tokens, if a user creates one +named `gitlab-deploy-token`, the name and token of the Deploy Token will be +automatically exposed to the CI/CD jobs as environment variables: `CI_DEPLOY_USER` and +`CI_DEPLOY_PASSWORD`, respectively. + +[ce-17894]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17894 +[ce-11845]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845 +[ce-18414]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18414 +[container registry]: ../container_registry.md diff --git a/doc/user/project/img/project_overview_badges.png b/doc/user/project/img/project_overview_badges.png Binary files differnew file mode 100644 index 00000000000..3067a7dfa13 --- /dev/null +++ b/doc/user/project/img/project_overview_badges.png diff --git a/doc/user/project/index.md b/doc/user/project/index.md index f94e93dd7d8..557375a1da9 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -17,7 +17,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Issue tracker](issues/index.md): Discuss implementations with your team within issues - [Issue Boards](issue_board.md): Organize and prioritize your workflow - - [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) (**Starter/Premium**): Allow your teams to create their own workflows (Issue Boards) for the same project + - [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards): Allow your teams to create their own workflows (Issue Boards) for the same project **[STARTER]** - [Repositories](repository/index.md): Host your code in a fully integrated platform - [Branches](repository/branches/index.md): use Git branching strategies to @@ -27,10 +27,11 @@ integrated platform - [Protected tags](protected_tags.md): Control over who has permission to create tags, and prevent accidental update or deletion - [Signing commits](gpg_signed_commits/index.md): use GPG to sign your commits + - [Deploy tokens](deploy_tokens/index.md): Manage project-based deploy tokens that allow permanent access to the repository and Container Registry. - [Merge Requests](merge_requests/index.md): Apply your branching strategy and get reviewed by your team - - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) (**Starter/Premium**): Ask for approval before - implementing a change + - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html): Ask for approval before + implementing a change **[STARTER]** - [Fix merge conflicts from the UI](merge_requests/resolve_conflicts.md): Your Git diff tool right from GitLab's UI - [Review Apps](../../ci/review_apps/index.md): Live preview the results @@ -44,6 +45,7 @@ and time spent on templates for issue and merge request description fields for your project - [Slash commands (quick actions)](quick_actions.md): Textual shortcuts for common actions on issues or merge requests +- [Web IDE](web_ide/index.md) **GitLab CI/CD:** @@ -73,6 +75,7 @@ website with GitLab Pages - [Cycle Analytics](cycle_analytics.md): Review your development lifecycle - [Syntax highlighting](highlighting.md): An alternative to customize your code blocks, overriding GitLab's default choice of language +- [Badges](badges.md): Badges for the project overview ### Project's integrations diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index 731291ebe84..6fc083170b6 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -15,8 +15,8 @@ in the table below. Once you have configured and enabled Custom Issue Tracker Service you'll see a link on the GitLab project pages that takes you to that custom issue tracker. - ## Referencing issues -Issues are referenced with `#<ID>`, where `<ID>` is a number (example `#143`). -So with the example above, `#143` would refer to `https://customissuetracker.com/project-name/143`.
\ No newline at end of file +- Issues are referenced with `ANYTHING-<ID>`, where `ANYTHING` can be any string and `<ID>` is a number used in the target project of the custom integration (example `PROJECT-143`). +- `ANYTHING` is a placeholder to differentiate against GitLab issues, which are referenced with `#<ID>`. You can use a project name or project key to replace it for example. +- So with the example above, `PROJECT-143` would refer to `https://customissuetracker.com/project-name/143`.
\ No newline at end of file diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index 34a0b97a171..bf6c0dc0e7e 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -6,7 +6,7 @@ GitLab has support for automatically detecting and monitoring AWS resources, sta ## Requirements -The [Prometheus service](../prometheus/index.md) must be enabled. +The [Prometheus service](../prometheus.md) must be enabled. ## Metrics supported diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index 518018e5839..cd398f7c0fd 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -5,7 +5,7 @@ GitLab has support for automatically detecting and monitoring HAProxy. This is p ## Requirements -The [Prometheus service](../prometheus/index.md) must be enabled. +The [Prometheus service](../prometheus.md) must be enabled. ## Metrics supported diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md index f09ecf9ff2d..96a22316265 100644 --- a/doc/user/project/integrations/prometheus_library/metrics.md +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -1,4 +1,5 @@ # Prometheus Metrics library + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). Currently supported exporters are: @@ -15,7 +16,7 @@ We have tried to surface the most important metrics for each exporter, and will GitLab retrieves performance data from the configured Prometheus server, and attempts to identifying the presence of known metrics. Once identified, GitLab then needs to be able to map the data to a particular environment. In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do that, -GitLab uses the defined queries and fills in the environment specific variables. Typically this involves looking for the [$CI_ENVIRONMENT_SLUG](https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables), but may also include other information such as the project's Kubernetes namespace. Each search query is defined in the [exporter specific documentation](#prometheus-metrics-library). +GitLab uses the defined queries and fills in the environment specific variables. Typically this involves looking for the [$CI_ENVIRONMENT_SLUG](../../../../ci/variables/README.md#predefined-variables-environment-variables), but may also include other information such as the project's Kubernetes namespace. Each search query is defined in the [exporter specific documentation](#prometheus-metrics-library). ## Adding to the library diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index 7fb8369d3c1..fea3231006b 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -6,7 +6,7 @@ GitLab has support for automatically detecting and monitoring NGINX. This is pro ## Requirements -The [Prometheus service](../prometheus/index.md) must be enabled. +The [Prometheus service](../prometheus.md) must be enabled. ## Metrics supported diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 49b34c82ae6..590b1c4275a 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -6,7 +6,7 @@ GitLab has support for automatically detecting and monitoring the Kubernetes NGI ## Requirements -[Prometheus integration](../prometheus/index.md) must be active. +[Prometheus integration](../prometheus.md) must be active. ## Metrics supported @@ -27,7 +27,7 @@ For other deployments, there is [some configuration](#manually-setting-up-nginx- ### About managed NGINX Ingress deployments -NGINX Ingress is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress). NGINX Ingress will be [externally reachable via the Load Balancer's IP](https://docs.gitlab.com/ce/user/project/clusters/index.html#getting-the-external-ip-address). +NGINX Ingress is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress). NGINX Ingress will be [externally reachable via the Load Balancer's IP](../../clusters/index.md#getting-the-external-ip-address). NGINX is configured for Prometheus monitoring, by setting: * `enable-vts-status: "true"`, to export Prometheus metrics @@ -51,4 +51,4 @@ Managing these settings depends on how NGINX ingress has been deployed. If you h In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do this, GitLab will search for metrics with appropriate labels. In this case, the `upstream` label must be of the form `<KUBE_NAMESPACE>-<CI_ENVIRONMENT_SLUG>-*`. -If you have used [Auto Deploy](https://docs.gitlab.com/ee/ci/autodeploy/index.html) to deploy your app, this format will be used automatically and metrics will be detected with no action on your part. +If you have used [Auto Deploy](../../../../topics/autodevops/index.md#auto-deploy) to deploy your app, this format will be used automatically and metrics will be detected with no action on your part. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index b4a842f33d6..7eab825fa32 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -240,8 +240,7 @@ Issue Board, that is create/delete lists and drag issues around. >Introduced in GitLab 10.6 Group issue board is analogous to project-level issue board and it is accessible at the group -navigation level. A group-level issue board allows you to view all issues from all projects in that group -(currently, it does not see issues from projects in subgroups). Similarly, you can only filter by group labels for these +navigation level. A group-level issue board allows you to view all issues from all projects in that group or descendant subgroups. Similarly, you can only filter by group labels for these boards. When updating milestones and labels for an issue through the sidebar update mechanism, again only group-level objects are available. diff --git a/doc/user/project/issues/closing_issues.md b/doc/user/project/issues/closing_issues.md index dcfa5ff59b2..1d88745af9f 100644 --- a/doc/user/project/issues/closing_issues.md +++ b/doc/user/project/issues/closing_issues.md @@ -48,12 +48,12 @@ link to each other, but the MR will NOT close the issue(s) when merged. ## From the Issue Board -You can close an issue from [Issue Boards](../issue_board.md) by draging an issue card +You can close an issue from [Issue Boards](../issue_board.md) by dragging an issue card from its list and dropping into **Closed**.  -## Customizing the issue closing patern +## Customizing the issue closing pattern Alternatively, a GitLab **administrator** can -[customize the issue closing patern](../../../administration/issue_closing_pattern.md). +[customize the issue closing pattern](../../../administration/issue_closing_pattern.md). diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index cc8988be36b..786d1c81b1b 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -60,4 +60,4 @@ or simply link both issue and merge request as described in the ### Close an issue by merging a merge request -To [close an issue when a merge request is merged](closing_issues.md#via-merge-request), use the [automatic issue closing patern](automatic_issue_closing.md). +To [close an issue when a merge request is merged](closing_issues.md#via-merge-request), use the [automatic issue closing pattern](automatic_issue_closing.md). diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index e0c405353ce..1bf8b776c2e 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -35,5 +35,9 @@ Due dates also appear in your [todos list](../../../workflow/todos.md).  +The day before an open issue is due, an email will be sent to all participants +of the issue. Both the due date and the day before are calculated using the +server's timezone. + [ce-3614]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3614 [permissions]: ../../permissions.md#project diff --git a/doc/user/project/issues/issues_functionalities.md b/doc/user/project/issues/issues_functionalities.md index f2ca6a6822e..e9903b01c82 100644 --- a/doc/user/project/issues/issues_functionalities.md +++ b/doc/user/project/issues/issues_functionalities.md @@ -28,7 +28,7 @@ Comments and system notes also appear automatically in response to various actio #### 2. Todos - Add todo: add that issue to your [GitLab Todo](../../../workflow/todos.html) list -- Mark done: mark that issue as done (reflects on the Todo list) +- Mark todo as done: mark that issue as done (reflects on the Todo list) #### 3. Assignee @@ -41,10 +41,7 @@ it's reassigned to someone else to take it from there. if a user is not member of that project, it can only be assigned to them if they created the issue themselves. -##### 3.1. Multiple Assignees - -> Available in [GitLab Starter](https://about.gitlab.com/products/) and -[GitLab.com Bronze](https://about.gitlab.com/gitlab-com/). +##### 3.1. Multiple Assignees **[STARTER]** Often multiple people likely work on the same issue together, which can especially be difficult to track in large teams @@ -89,10 +86,7 @@ but they are immediately available to all projects in the group. > **Tip:** if the label doesn't exist yet, when you click **Edit**, it opens a dropdown menu from which you can select **Create new label**. -#### 8. Weight - -> Available in [GitLab Starter](https://about.gitlab.com/products/) and -[GitLab.com Bronze](https://about.gitlab.com/gitlab-com/). +#### 8. Weight **[STARTER]** - Attribute a weight (in a 0 to 9 range) to that issue. Easy to complete should weight 1 and very hard to complete should weight 9. @@ -158,7 +152,7 @@ know you like it without spamming them. These text fields also fully support [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). -#### 17. Comment, start a discusion, or comment and close +#### 17. Comment, start a discussion, or comment and close Once you wrote your comment, you can either: diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index a89a1206170..914898ea2ea 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -9,8 +9,7 @@ Labels allow you to categorize issues or merge requests using descriptive titles In GitLab, you can create project and group labels: - **Project labels** can be assigned to issues or merge requests in that project only. -- **Group labels** can be assigned to any issue or merge request of any project in that group or subgroup. -- In the [future](https://gitlab.com/gitlab-org/gitlab-ce/issues/40915), you will be able to assign group labels to issues and merge reqeusts of projects in [subgroups](../group/subgroups/index.md). +- **Group labels** can be assigned to any issue or merge request of any project in that group or any subgroups of the group. ## Creating labels diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 3640d236db4..a6c0fd49c45 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -32,10 +32,10 @@ With GitLab merge requests, you can: With **[GitLab Enterprise Edition][ee]**, you can also: -- View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) (available only in GitLab Premium) -- Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers (available in GitLab Starter) -- [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history (available in GitLab Starter) -- Analyze the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Starter) +- View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) **[PREMIUM]** +- Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers **[STARTER]** +- [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history **[STARTER]** +- Analyze the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) **[STARTER]** ## Use cases @@ -43,7 +43,7 @@ A. Consider you are a software developer working in a team: 1. You checkout a new branch, and submit your changes through a merge request 1. You gather feedback from your team -1. You work on the implementation optimizing code with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Starter) +1. You work on the implementation optimizing code with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) **[STARTER]** 1. You build and test your changes with GitLab CI/CD 1. You request the approval from your manager 1. Your manager pushes a commit with his final review, [approves the merge request](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds) (Merge Request Approvals are available in GitLab Starter) @@ -56,7 +56,7 @@ B. Consider you're a web developer writing a webpage for your company's: 1. You gather feedback from your reviewers 1. Your changes are previewed with [Review Apps](../../../ci/review_apps/index.md) 1. You request your web designers for their implementation -1. You request the [approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your manager (available in GitLab Starter) +1. You request the [approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your manager **[STARTER]** 1. Once approved, your merge request is [squashed and merged](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html), and [deployed to staging with GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) (Squash and Merge is available in GitLab Starter) 1. Your production team [cherry picks](#cherry-pick-changes) the merge commit into production diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 10e6321eb82..64bb33be547 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -10,7 +10,7 @@ Milestones allow you to organize issues and merge requests into a cohesive group - **Project milestones** can be assigned to issues or merge requests in that project only. - **Group milestones** can be assigned to any issue or merge request of any project in that group. -- In the [future](https://gitlab.com/gitlab-org/gitlab-ce/issues/36862), you will be able to assign group milestones to issues and merge reqeusts of projects in [subgroups](../../group/subgroups/index.md). +- In the [future](https://gitlab.com/gitlab-org/gitlab-ce/issues/36862), you will be able to assign group milestones to issues and merge requests of projects in [subgroups](../../group/subgroups/index.md). ## Creating milestones diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 430fe3af1f8..61af1d2ab27 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -70,7 +70,7 @@ In case you want to point a root domain (`example.com`) to your GitLab Pages site, deployed to `namespace.gitlab.io`, you need to log into your domain's admin control panel and add a DNS `A` record pointing your domain to Pages' server IP address. For projects on -GitLab.com, this IP is `52.167.214.135`. For projects leaving in +GitLab.com, this IP is `52.167.214.135`. For projects living in other GitLab instances (CE or EE), please contact your sysadmin asking for this information (which IP address is Pages server running on your instance). diff --git a/doc/user/project/pages/getting_started_part_two.md b/doc/user/project/pages/getting_started_part_two.md index 2274cac8ace..556bf1db116 100644 --- a/doc/user/project/pages/getting_started_part_two.md +++ b/doc/user/project/pages/getting_started_part_two.md @@ -50,14 +50,14 @@ created for the steps below. 1. [Fork a sample project](../../../gitlab-basics/fork-project.md) from the [Pages group](https://gitlab.com/pages) 1. Trigger a build (push a change to any file) 1. As soon as the build passes, your website will have been deployed with GitLab Pages. Your website URL will be available under your project's **Settings** > **Pages** -1. Optionally, remove the fork relationship by navigating to your project's **Settings** > expanding **Advanced settings** and scrolling down to **Remove fork relashionship**: +1. Optionally, remove the fork relationship by navigating to your project's **Settings** > expanding **Advanced settings** and scrolling down to **Remove fork relationship**: -  +  To turn a **project website** forked from the Pages group into a **user/group** website, you'll need to: - Rename it to `namespace.gitlab.io`: navigate to project's **Settings** > expand **Advanced settings** > and scroll down to **Rename repository** -- Adjust your SSG's [base URL](#urls-and-baseurls) to from `"project-name"` to `""`. This setting will be at a different place for each SSG, as each of them have their own structure and file tree. Most likelly, it will be in the SSG's config file. +- Adjust your SSG's [base URL](#urls-and-baseurls) to from `"project-name"` to `""`. This setting will be at a different place for each SSG, as each of them have their own structure and file tree. Most likely, it will be in the SSG's config file. > **Notes:** > diff --git a/doc/user/project/pages/img/remove_fork_relashionship.png b/doc/user/project/pages/img/remove_fork_relationship.png Binary files differindex 67c45491f08..67c45491f08 100644 --- a/doc/user/project/pages/img/remove_fork_relashionship.png +++ b/doc/user/project/pages/img/remove_fork_relationship.png diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index a65aa758198..a97ce84b861 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -1,23 +1,22 @@ # GitLab Pages -With GitLab Pages you can host your website at no cost. - -Your files live in a GitLab project's [repository](../repository/index.md), -from which you can deploy [static websites](#explore-gitlab-pages). -GitLab Pages supports all static site generators (SSGs). +With GitLab Pages it's easy to publish your project website. GitLab Pages is a hosting service for static websites, at no additional cost. ## Getting Started -Follow the steps below to get your website live. They shouldn't take more than -5 minutes to complete: +[Create a project from scratch](getting_started_part_two.md#create-a-project-from-scratch) +to get you started quickly, or, +alternatively, start from an existing project as follows: -- 1. [Fork](../../../gitlab-basics/fork-project.md#how-to-fork-a-project) an [example project](https://gitlab.com/pages) -- 2. Change a file to trigger a GitLab CI/CD pipeline -- 3. Visit your project's **Settings > Pages** to see your **website link**, and click on it. Bam! Your website is live. +- 1. [Fork](../../../gitlab-basics/fork-project.md#how-to-fork-a-project) an [example project](https://gitlab.com/pages): +by forking a project, you create a copy of the codebase you're forking from to start from a template instead of starting from scratch. +- 2. Change a file to trigger a GitLab CI/CD pipeline: GitLab CI/CD will build and deploy your site to GitLab Pages. +- 3. Visit your project's **Settings > Pages** to see your **website link**, and click on it. Bam! Your website is live! :) _Further steps (optional):_ -- 4. Remove the [fork relationship](getting_started_part_two.md#fork-a-project-to-get-started-from) (_You don't need the relationship unless you intent to contribute back to the example project you forked from_). +- 4. Remove the [fork relationship](getting_started_part_two.md#fork-a-project-to-get-started-from) +(_You don't need the relationship unless you intent to contribute back to the example project you forked from_). - 5. Make it a [user/group website](getting_started_part_one.md#user-and-group-websites) **Watch a video with the steps above: https://www.youtube.com/watch?v=TWqh9MtT4Bg** @@ -27,14 +26,23 @@ _Advanced options:_ - [Use a custom domain](getting_started_part_three.md#adding-your-custom-domain-to-gitlab-pages) - Apply [SSL/TLS certification](getting_started_part_three.md#ssl-tls-certificates) to your custom domain -## Explore GitLab Pages +## How Does It Work? With GitLab Pages you can create [static websites](getting_started_part_one.md#what-you-need-to-know-before-getting-started) -for your GitLab projects, groups, or user accounts. You can use any static -website generator: Jekyll, Middleman, Hexo, Hugo, Pelican, you name it! +for your GitLab projects, groups, or user accounts. + +It supports plain static content, such as HTML, and **all** [static site generators (SSGs)](https://about.gitlab.com/2016/06/03/ssg-overview-gitlab-pages-part-1-dynamic-x-static/), such as Jekyll, Middleman, Hexo, Hugo, and Pelican. + Connect as many custom domains as you like and bring your own TLS certificate to secure them. +Your files live in a project [repository](../repository/index.md) on GitLab. +[GitLab CI](../../../ci/README.md) picks up those files and makes them available at, typically, +`http://<username>.gilab.io/<projectname>`. Please read through the docs on +[GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain) for more info. + +## Explore GitLab Pages + Read the following tutorials to know more about: - [Static websites and GitLab Pages domains](getting_started_part_one.md): Understand what is a static website, and how GitLab Pages default domains work diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 6cead7b9961..14f2e522f01 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -106,7 +106,7 @@ If you want to auto-cancel all pending non-HEAD pipelines on branch, when new pipeline will be created (after your git push or manually from UI), check **Auto-cancel pending pipelines** checkbox and save the changes. -## Badges +## Pipeline Badges In the pipelines settings page you can find pipeline status and test coverage badges for your project. The latest successful pipeline will be used to read diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 442fc978284..2f4ed3493c2 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -38,6 +38,7 @@ do. | `/award :emoji:` | Toggle award for :emoji: | | `/board_move ~column` | Move issue to column on the board | | `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | -| `/move path/to/project` | Moves issue to another project | -| `/tableflip` | Append the comment with `(╯°□°)╯︵ ┻━┻` | -| `/shrug` | Append the comment with `¯\＿(ツ)＿/¯` |
\ No newline at end of file +| `/move path/to/project` | Moves issue to another project | +| `/tableflip` | Append the comment with `(╯°□°)╯︵ ┻━┻` | +| `/shrug` | Append the comment with `¯\＿(ツ)＿/¯` | +| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request | diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 08805a4dc99..a06ecc3220f 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -1,6 +1,6 @@ # Reducing the repository size using Git -A GitLab Entrerprise Edition administrator can set a [repository size limit][admin-repo-size] +A GitLab Enterprise Edition administrator can set a [repository size limit][admin-repo-size] which will prevent you to exceed it. When a project has reached its size limit, you will not be able to push to it, diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index dedf102fc37..2c90f4b4413 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -31,7 +31,8 @@ with all their related data and be moved into a new GitLab instance. | GitLab version | Import/Export version | | ---------------- | --------------------- | -| 10.4 to current | 0.2.2 | +| 10.8 to current | 0.2.3 | +| 10.4 | 0.2.2 | | 10.3 | 0.2.1 | | 10.0 | 0.2.0 | | 9.4.0 | 0.1.8 | @@ -57,11 +58,11 @@ The following items will be exported: - Project configuration including web hooks and services - Issues with comments, merge requests with diffs and comments, labels, milestones, snippets, and other project entities +- LFS objects The following items will NOT be exported: - Build traces and artifacts -- LFS objects - Container registry images - CI variables - Any encrypted tokens diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 888dd0e143a..c9d2f8dc32d 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -34,7 +34,7 @@ Set up your project's merge request settings: - Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.html)). - Merge request [description templates](../description_templates.md#description-templates). -- Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals), _available in [GitLab Starter](https://about.gitlab.com/products/)_. +- Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals). **[STARTER]** - Enable [merge only of pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). - Enable [merge only when all discussions are resolved](../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-discussions-are-resolved). @@ -57,15 +57,20 @@ Here you can run housekeeping, archive, rename, transfer, or remove a project. NOTE: **Note:** Only project Owners and Admin users have the [permissions] to archive a project. -An archived project will be hidden by default in the project listings. +Archiving a project makes it read-only for all users and indicates that it is +no longer actively maintained. Projects that have been archived can also be +unarchived. + +When a project is archived, the repository, issues, merge requests and all +other features are read-only. Archived projects are also hidden +in project listings. + +To archive a project: 1. Navigate to your project's **Settings > General > Advanced settings**. -1. Under "Archive project", hit the **Archive project** button. +1. In the Archive project section, click the **Archive project** button. 1. Confirm the action when asked to. -An archived project can be fully restored and will therefore retain its -repository and all associated resources whilst in an archived state. - #### Renaming a repository NOTE: **Note:** diff --git a/doc/user/project/web_ide/img/commit_changes.png b/doc/user/project/web_ide/img/commit_changes.png Binary files differnew file mode 100644 index 00000000000..b6fcbf699aa --- /dev/null +++ b/doc/user/project/web_ide/img/commit_changes.png diff --git a/doc/user/project/web_ide/img/enable_web_ide.png b/doc/user/project/web_ide/img/enable_web_ide.png Binary files differnew file mode 100644 index 00000000000..196baa82ad2 --- /dev/null +++ b/doc/user/project/web_ide/img/enable_web_ide.png diff --git a/doc/user/project/web_ide/img/open_web_ide.png b/doc/user/project/web_ide/img/open_web_ide.png Binary files differnew file mode 100644 index 00000000000..d1192daf506 --- /dev/null +++ b/doc/user/project/web_ide/img/open_web_ide.png diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md new file mode 100644 index 00000000000..b7064b83c4e --- /dev/null +++ b/doc/user/project/web_ide/index.md @@ -0,0 +1,33 @@ +# Web IDE + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab-ee/issues/4539) [GitLab Ultimate][ee] 10.4. +> [Brought to GitLab Core](https://gitlab.com/gitlab-org/gitlab-ce/issues/44157) in 10.7. + +The Web IDE makes it faster and easier to contribute changes to your projects +by providing an advanced editor with commit staging. + +## Open the Web IDE + +The Web IDE can be opened when viewing a file, from the repository file list, +and from merge requests. + + + +## Commit changes + +Changed files are shown on the right in the commit panel. All changes are +automatically staged. To commit your changes, add a commit message and click +the 'Commit Button'. + + + +## Comparing changes + +Before you commit your changes, you can compare them with the previous commit +by switching to the review mode or selecting the file from the staged files +list. + +An additional review mode is available when you open a merge request, which +shows you a preview of the merge request diff if you commit your changes. + +[ee]: https://about.gitlab.com/pricing/ diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 2b23c494dc4..4f1b96b775c 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -96,7 +96,7 @@ On the field **Filter by name**, type the project or group name you want to find will filter them for you as you type. You can also look for the projects you starred (**Starred projects**), and **Explore** all -public and internal projects available in GitLab.com, from which you can filter by visibitily, +public and internal projects available in GitLab.com, from which you can filter by visibility, through **Trending**, best rated with **Most starts**, or **All** of them. You can also sort them by **Name**, **Last created**, **Oldest created**, **Last updated**, diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 377eee69c11..0d592a6d43e 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -243,4 +243,12 @@ GitLab checks files to detect LFS pointers on push. If LFS pointers are detected Verify that LFS in installed locally and consider a manual push with `git lfs push --all`. -If you are storing LFS files outside of GitLab you can disable LFS on the project by settting `lfs_enabled: false` with the [projects api](../../api/projects.md#edit-project). +If you are storing LFS files outside of GitLab you can disable LFS on the project by setting `lfs_enabled: false` with the [projects api](../../api/projects.md#edit-project). + +### Hosting LFS objects externally + +It is possible to host LFS objects externally by setting a custom LFS url with `git config -f .lfsconfig lfs.url https://example.com/<project>.git/info/lfs`. + +Because GitLab verifies the existence of objects referenced by LFS pointers, push will fail when LFS is enabled for the project. + +LFS can be disabled from the [Project settings](../../user/project/settings/index.md). diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index c4095ee0f69..f1501c81b27 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -86,6 +86,7 @@ In most of the below cases, the notification will be sent to: | Close issue | | | Reassign issue | The above, plus the old assignee | | Reopen issue | | +| Due issue | Participants and Custom notification level with this event selected | | New merge request | | | Push to merge request | Participants and Custom notification level with this event selected | | Reassign merge request | The above, plus the old assignee | @@ -96,15 +97,14 @@ In most of the below cases, the notification will be sent to: | Failed pipeline | The author of the pipeline | | Successful pipeline | The author of the pipeline, if they have the custom notification setting for successful pipelines set | - In addition, if the title or description of an Issue or Merge Request is changed, notifications will be sent to any **new** mentions by `@username` as if they had been mentioned in the original text. -You won't receive notifications for Issues, Merge Requests or Milestones -created by yourself. You will only receive automatic notifications when -somebody else comments or adds changes to the ones that you've created or -mentions you. +You won't receive notifications for Issues, Merge Requests or Milestones created +by yourself (except when an issue is due). You will only receive automatic +notifications when somebody else comments or adds changes to the ones that +you've created or mentions you. ### Email Headers @@ -122,7 +122,7 @@ Notification emails include headers that provide extra content about the notific | X-GitLab-NotificationReason | The reason for being notified. "mentioned", "assigned", etc | #### X-GitLab-NotificationReason -This header holds the reason for the notification to have been sent out, +This header holds the reason for the notification to have been sent out, where reason can be `mentioned`, `assigned`, `own_activity`, etc. Only one reason is sent out according to its priority: - `own_activity` @@ -130,7 +130,7 @@ Only one reason is sent out according to its priority: - `mentioned` The reason in this header will also be shown in the footer of the notification email. For example an email with the -reason `assigned` will have this sentence in the footer: +reason `assigned` will have this sentence in the footer: `"You are receiving this email because you have been assigned an item on {configured GitLab hostname}"` **Note: Only reasons listed above have been implemented so far** diff --git a/doc/workflow/todos.md b/doc/workflow/todos.md index e612646cfbc..f13d29884d4 100644 --- a/doc/workflow/todos.md +++ b/doc/workflow/todos.md @@ -92,9 +92,9 @@ corresponding **Done** button, and it will disappear from your Todo list.  A Todo can also be marked as done from the issue or merge request sidebar using -the "Mark done" button. +the "Mark todo as done" button. - + You can mark all your Todos as done at once by clicking on the **Mark all as done** button. |
