From f7edc65899aa2d360d26215b13212f681b8f0b95 Mon Sep 17 00:00:00 2001 From: Jonathon Reinhart Date: Fri, 22 Sep 2017 01:57:13 +0000 Subject: Update installation.md with receive.advertisePushOptions See https://gitlab.com/gitlab-org/gitlab-ce/issues/33061#note_37473342 --- doc/install/installation.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 200cd94f43c..7eec3132393 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -367,6 +367,9 @@ sudo usermod -aG redis git # Enable packfile bitmaps sudo -u git -H git config --global repack.writeBitmaps true + + # Enable push options + sudo -u git -H git config --global receive.advertisePushOptions true # Configure Redis connection settings sudo -u git -H cp config/resque.yml.example config/resque.yml -- cgit v1.2.1 From fc8bbf3b49ccd0e082c594167797f0b8eedf14a1 Mon Sep 17 00:00:00 2001 From: Ben Bodenmiller Date: Tue, 7 Nov 2017 23:33:10 +0000 Subject: formatting cleanup, swamp details, unicorn defaults --- doc/install/requirements.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 7bf126eec5d..baecf9455b0 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -80,13 +80,13 @@ errors during usage. - 256GB RAM supports up to 32,000 users - More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/) -We recommend having at least 2GB of swap on your server, even if you currently have +We recommend having at least [2GB of swap on your server](https://askubuntu.com/a/505344/310789), even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring if your available memory changes. We also recommend [configuring the kernel's swappiness setting](https://askubuntu.com/a/103916) to a low value like `10` to make the most of your RAM while still having the swap available when needed. -Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. +Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as `top` or `htop`) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. ## Database @@ -146,7 +146,7 @@ So for a machine with 2 cores, 3 unicorn workers is ideal. For all machines that have 2GB and up we recommend a minimum of three unicorn workers. If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping. -To change the Unicorn workers when you have the Omnibus package please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings). +To change the Unicorn workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings). ## Redis and Sidekiq -- cgit v1.2.1 From 08eaa59359ff1afc8ecac5b942d0f5b355cb8df4 Mon Sep 17 00:00:00 2001 From: Tim Zallmann Date: Fri, 3 Nov 2017 09:44:00 +0100 Subject: Added + Updated Document for Vue Component --- doc/development/fe_guide/icons.md | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index cef62618a3c..382c7311905 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -4,15 +4,17 @@ We are using SVG Icons in GitLab with a SVG Sprite, due to this the icons are on ### Usage in HAML/Rails -To use a sprite Icon in HAML or Rails we use a specific helper function : +To use a sprite Icon in HAML or Rails we use a specific helper function : `sprite_icon(icon_name, size: nil, css_class: '')` -**icon_name** Use the icon_name that you can find in the SVG Sprite (Overview is available under `/assets/sprite.symbol.html`). +**icon_name** Use the icon_name that you can find in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). + **size (optional)** Use one of the following sizes : 16,24,32,48,72 (this will be translated into a `s16` class) + **css_class (optional)** If you want to add additional css classes -**Example** +**Example** `= sprite_icon('issues', size: 72, css_class: 'icon-danger')` @@ -20,9 +22,27 @@ To use a sprite Icon in HAML or Rails we use a specific helper function : `` +### Usage in Vue + +We have a special Vue component for our sprite icons in `\vue_shared\components\icon.vue`. + +Sample usage : + +`` + +**name** Name of the Icon in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). + +**size (optional)** Number value for the size which is then mapped to a specific CSS class (Available Sizes: 8,12,16,18,24,32,48,72 are mapped to `sXX` css classes) + +**css-classes (optional)** Additional CSS Classes to add to the svg tag. + ### Usage in HTML/JS -Please use the following function inside JS to render an icon : +Please use the following function inside JS to render an icon : `gl.utils.spriteIcon(iconName)` ## Adding a new icon to the sprite -- cgit v1.2.1 From 245d0daddef23427d9b1aa206dbedf5b9cc89b34 Mon Sep 17 00:00:00 2001 From: Tim Zallmann Date: Tue, 14 Nov 2017 20:08:51 +0100 Subject: Added info about tracking --- doc/development/fe_guide/icons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index 382c7311905..b288ee95722 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -49,7 +49,7 @@ Please use the following function inside JS to render an icon : All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. -To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. +To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. The updated files should be tracked in Git as those are referenced. # SVG Illustrations -- cgit v1.2.1 From 555f261d73aa7d5765f0d57fc9773898d3212ee9 Mon Sep 17 00:00:00 2001 From: Lee Matos Date: Thu, 16 Nov 2017 15:49:41 +0000 Subject: Update HA README.md to clarify GitLab support does not troubleshoot DRBD. --- doc/administration/high_availability/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 4d3be0ab8f6..a88e67bfeb5 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -53,7 +53,9 @@ or in different cloud availability zones. > **Note:** GitLab recommends against choosing this HA method because of the complexity of managing DRBD and crafting automatic failover. This is - *compatible* with GitLab, but not officially *supported*. + *compatible* with GitLab, but not officially *supported*. If you are + an EE customer, support will help you with GitLab related problems, but if the + root cause is identified as DRBD, we will not troubleshoot further. Components/Servers Required: 2 servers/virtual machines (one active/one passive) -- cgit v1.2.1 From 644a2941c0903b7c67e356998b8f3a39143b54e0 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 16 Nov 2017 09:44:06 +0100 Subject: Add info about the installed apps in the Cluster docs --- .../project/clusters/img/cluster-applications.png | Bin 39115 -> 0 bytes doc/user/project/clusters/index.md | 22 +++++++++++---------- 2 files changed, 12 insertions(+), 10 deletions(-) delete mode 100644 doc/user/project/clusters/img/cluster-applications.png (limited to 'doc') diff --git a/doc/user/project/clusters/img/cluster-applications.png b/doc/user/project/clusters/img/cluster-applications.png deleted file mode 100644 index 7c82d19297e..00000000000 Binary files a/doc/user/project/clusters/img/cluster-applications.png and /dev/null differ diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 27b4b49c207..7f7afc27aa0 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -56,7 +56,7 @@ To add a new cluster: don't have to fill it in; by leaving it blank, GitLab will create one for you. 1. Click the **Create cluster** button. -After a few moments your cluster should be created. If something goes wrong, +After a few moments, your cluster should be created. If something goes wrong, you will be notified. Now, you can proceed to [enable the Cluster integration](#enabling-or-disabling-the-cluster-integration). @@ -73,6 +73,17 @@ You can now start using your Kubernetes cluster for your deployments. To disable the Cluster integration, follow the same procedure. +## Installing applications + +GitLab provides a one-click install for various applications which will be +added directly to your configured cluster. Those applications are needed for +review apps and deployments. + +| Application | GitLab version | Description | +| ----------- | :------------: | ----------- | +| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress acts as a web proxy for your web applications and is useful if you want to use Auto DevOps or deploy your own apps. | + ## Removing the Cluster integration NOTE: **Note:** @@ -88,12 +99,3 @@ To remove the Cluster integration from your project, simply click on the and [add a cluster](#adding-a-cluster) again. [permissions]: ../../permissions.md - -## Installing applications - -GitLab provides a one-click install for -[Helm Tiller](https://docs.helm.sh/) and -[Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) -which will be added directly to your configured cluster. - -![Cluster application settings](img/cluster-applications.png) -- cgit v1.2.1 From b7a01e08b268e6d63910ce2facdbbd41f946614b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 16 Nov 2017 14:07:57 +0100 Subject: Add info on creating your own k8s cluster from the cluster page --- doc/user/project/clusters/index.md | 53 +++++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 21 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 7f7afc27aa0..fda8505e898 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,14 +1,15 @@ -# Connecting GitLab with GKE +# Connecting GitLab with a Kubernetes cluster > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. CAUTION: **Warning:** The Cluster integration is currently in **Beta**. -Connect your project to Google Container Engine (GKE) in a few steps. - With a cluster associated to your project, you can use Review Apps, deploy your -applications, run your pipelines, and much more in an easy way. +applications, run your pipelines, and much more, in an easy way. + +Connect your project to Google Container Engine (GKE) or your own Kubernetes +cluster in a few steps. NOTE: **Note:** The Cluster integration will eventually supersede the @@ -30,31 +31,41 @@ prerequisites must be met: - You must have Master [permissions] in order to be able to access the **Cluster** page. -If all of the above requirements are met, you can proceed to add a new cluster. +If all of the above requirements are met, you can proceed to add a new GKE +cluster. ## Adding a cluster NOTE: **Note:** You need Master [permissions] and above to add a cluster. +There are two options when adding a new cluster; either use Google Container +Engine (GKE) or provide the credentials to your own Kubernetes cluster. + To add a new cluster: -1. Navigate to your project's **CI/CD > Cluster** page. -1. Connect your Google account if you haven't done already by clicking the - "Sign-in with Google" button. -1. Fill in the requested values: - - **Cluster name** (required) - The name you wish to give the cluster. - - **GCP project ID** (required) - The ID of the project you created in your GCP - console that will host the Kubernetes cluster. This must **not** be confused - with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). - - **Zone** - The zone under which the cluster will be created. Read more about - [the available zones](https://cloud.google.com/compute/docs/regions-zones/). - - **Number of nodes** - The number of nodes you wish the cluster to have. - - **Machine type** - The machine type of the Virtual Machine instance that - the cluster will be based on. Read more about [the available machine types](https://cloud.google.com/compute/docs/machine-types). - - **Project namespace** - The unique namespace for this project. By default you - don't have to fill it in; by leaving it blank, GitLab will create one for you. -1. Click the **Create cluster** button. +1. Navigate to your project's **CI/CD > Cluster** page +1. If you want to let GitLab create a cluster on GKE for you, go through the + following steps, otherwise skip to the next one. + 1. Click on **Create with GKE** + 1. Connect your Google account if you haven't done already by clicking the + **Sign in with Google** button + 1. Fill in the requested values: + - **Cluster name** (required) - The name you wish to give the cluster. + - **GCP project ID** (required) - The ID of the project you created in your GCP + console that will host the Kubernetes cluster. This must **not** be confused + with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). + - **Zone** - The zone under which the cluster will be created. Read more about + [the available zones](https://cloud.google.com/compute/docs/regions-zones/). + - **Number of nodes** - The number of nodes you wish the cluster to have. + - **Machine type** - The machine type of the Virtual Machine instance that + the cluster will be based on. Read more about [the available machine types](https://cloud.google.com/compute/docs/machine-types). + - **Project namespace** - The unique namespace for this project. By default you + don't have to fill it in; by leaving it blank, GitLab will create one for you. +1. If you want to use your own Kubernetes cluster, click on + **Add an existing cluster** and fill in the details as described in the + [Kubernetes integration](../integrations/kubernetes.md) documentation. +1. Finally, click the **Create cluster** button After a few moments, your cluster should be created. If something goes wrong, you will be notified. -- cgit v1.2.1 From ee7ecc815ee8abadf3f9be665e50bba439c0dd3a Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 16 Nov 2017 15:34:21 +0100 Subject: Add Ingress description from official docs --- doc/user/project/clusters/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index fda8505e898..e4d12079d1c 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -93,7 +93,7 @@ review apps and deployments. | Application | GitLab version | Description | | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress acts as a web proxy for your web applications and is useful if you want to use Auto DevOps or deploy your own apps. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use Auto DevOps or deploy your own web apps. | ## Removing the Cluster integration -- cgit v1.2.1 From 3b9773fc3d1942370f9787a0384d94dfa35920b3 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 16 Nov 2017 18:10:16 +0100 Subject: Change to Google Kubernetes Cluster and add internal links --- doc/user/project/clusters/index.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index e4d12079d1c..01949f5cd23 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -8,7 +8,7 @@ The Cluster integration is currently in **Beta**. With a cluster associated to your project, you can use Review Apps, deploy your applications, run your pipelines, and much more, in an easy way. -Connect your project to Google Container Engine (GKE) or your own Kubernetes +Connect your project to Google Kubernetes Engine (GKE) or your own Kubernetes cluster in a few steps. NOTE: **Note:** @@ -39,7 +39,7 @@ cluster. NOTE: **Note:** You need Master [permissions] and above to add a cluster. -There are two options when adding a new cluster; either use Google Container +There are two options when adding a new cluster; either use Google Kubernetes Engine (GKE) or provide the credentials to your own Kubernetes cluster. To add a new cluster: @@ -55,14 +55,14 @@ To add a new cluster: - **GCP project ID** (required) - The ID of the project you created in your GCP console that will host the Kubernetes cluster. This must **not** be confused with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). - - **Zone** - The zone under which the cluster will be created. Read more about - [the available zones](https://cloud.google.com/compute/docs/regions-zones/). + - **Zone** - The [zone](https://cloud.google.com/compute/docs/regions-zones/) + under which the cluster will be created. - **Number of nodes** - The number of nodes you wish the cluster to have. - - **Machine type** - The machine type of the Virtual Machine instance that - the cluster will be based on. Read more about [the available machine types](https://cloud.google.com/compute/docs/machine-types). + - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) + of the Virtual Machine instance that the cluster will be based on. - **Project namespace** - The unique namespace for this project. By default you don't have to fill it in; by leaving it blank, GitLab will create one for you. -1. If you want to use your own Kubernetes cluster, click on +1. If you want to use your own existing Kubernetes cluster, click on **Add an existing cluster** and fill in the details as described in the [Kubernetes integration](../integrations/kubernetes.md) documentation. 1. Finally, click the **Create cluster** button @@ -88,12 +88,12 @@ To disable the Cluster integration, follow the same procedure. GitLab provides a one-click install for various applications which will be added directly to your configured cluster. Those applications are needed for -review apps and deployments. +[Review Apps](../../../ci/review_apps/index.md) and [deployments](../../../ci/environments.md). | Application | GitLab version | Description | | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use Auto DevOps or deploy your own web apps. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | ## Removing the Cluster integration -- cgit v1.2.1 From bacf671a335297e61ad4c470cde49ce4d3fcc009 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 16 Nov 2017 18:17:27 +0100 Subject: Move installation of apps higher --- doc/user/project/clusters/index.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 01949f5cd23..cf0c7c109a8 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -70,7 +70,19 @@ To add a new cluster: After a few moments, your cluster should be created. If something goes wrong, you will be notified. -Now, you can proceed to [enable the Cluster integration](#enabling-or-disabling-the-cluster-integration). +You can now proceed to install some pre-defined applications and then +enable the Cluster integration. + +## Installing applications + +GitLab provides a one-click install for various applications which will be +added directly to your configured cluster. Those applications are needed for +[Review Apps](../../../ci/review_apps/index.md) and [deployments](../../../ci/environments.md). + +| Application | GitLab version | Description | +| ----------- | :------------: | ----------- | +| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | ## Enabling or disabling the Cluster integration @@ -84,17 +96,6 @@ You can now start using your Kubernetes cluster for your deployments. To disable the Cluster integration, follow the same procedure. -## Installing applications - -GitLab provides a one-click install for various applications which will be -added directly to your configured cluster. Those applications are needed for -[Review Apps](../../../ci/review_apps/index.md) and [deployments](../../../ci/environments.md). - -| Application | GitLab version | Description | -| ----------- | :------------: | ----------- | -| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | - ## Removing the Cluster integration NOTE: **Note:** -- cgit v1.2.1 From 8922b7b781f241d9afd77f15fad0cfcab14d5205 Mon Sep 17 00:00:00 2001 From: Michael Kozono Date: Thu, 16 Nov 2017 18:03:21 +0000 Subject: Update database_debugging.md --- doc/development/database_debugging.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index 4acfbef3020..e1c785d85fa 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -9,7 +9,6 @@ An easy first step is to search for your error in Slack or google "GitLab Date: Thu, 16 Nov 2017 19:17:31 +0000 Subject: Update database_debugging.md --- doc/development/database_debugging.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index e1c785d85fa..50eb8005b44 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -9,17 +9,24 @@ An easy first step is to search for your error in Slack or google "GitLab Date: Fri, 17 Nov 2017 10:57:57 +0000 Subject: Add dropdowns documentation --- doc/development/fe_guide/dropdowns.md | 38 +++++++++++++++++++++++++++++++++++ doc/development/fe_guide/index.md | 2 ++ 2 files changed, 40 insertions(+) create mode 100644 doc/development/fe_guide/dropdowns.md (limited to 'doc') diff --git a/doc/development/fe_guide/dropdowns.md b/doc/development/fe_guide/dropdowns.md new file mode 100644 index 00000000000..e1660ac5caa --- /dev/null +++ b/doc/development/fe_guide/dropdowns.md @@ -0,0 +1,38 @@ +# Dropdowns + + +## How to style a bootstrap dropdown +1. Use the HTML structure provided by the [docs][bootstrap-dropdowns] +1. Add a specific class to the top level `.dropdown` element + + + ```Haml + .dropdown.my-dropdown + %button{ type: 'button', data: { toggle: 'dropdown' }, 'aria-haspopup': true, 'aria-expanded': false } + %span.dropdown-toggle-text + Toggle Dropdown + = icon('chevron-down') + + %ul.dropdown-menu + %li + %a + item! + ``` + + Or use the helpers + ```Haml + .dropdown.my-dropdown + = dropdown_toggle('Toogle!', { toggle: 'dropdown' }) + = dropdown_content + %li + %a + item! + ``` + +1. Include the mixin in CSS + + ```SCSS + @include new-style-dropdown('.my-dropdown '); + ``` + +[bootstrap-dropdowns]: https://getbootstrap.com/docs/3.3/javascript/#dropdowns diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 8f956681693..73a03c07812 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -77,6 +77,8 @@ Vue resource specific practices and gotchas. ## [Icons](icons.md) How we use SVG for our Icons. +## [Dropdowns](dropdowns.md) +How we use dropdowns. --- ## Style Guides -- cgit v1.2.1 From 1ab95055e7fdbc8b110d0df731ec082798d22f93 Mon Sep 17 00:00:00 2001 From: Mark Fletcher Date: Fri, 17 Nov 2017 14:04:20 +0000 Subject: Document when Groups API `/subgroups` endpoint introduced --- doc/api/groups.md | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'doc') diff --git a/doc/api/groups.md b/doc/api/groups.md index 6a6e94195a7..c1b5737c247 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -82,6 +82,8 @@ GET /groups?custom_attributes[key]=value&custom_attributes[other_key]=other_valu ## List a groups's subgroups +> [Introduced][ce-15142] in GitLab 10.3. + Get a list of visible direct subgroups in this group. When accessed without authentication, only public groups are returned. @@ -513,3 +515,5 @@ And to switch pages add: ``` /groups?per_page=100&page=2 ``` + +[ce-15142]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15142 -- cgit v1.2.1 From 4d367dd40037500fd7b96059fa300cbe2ce28e85 Mon Sep 17 00:00:00 2001 From: Sean McGivern Date: Fri, 17 Nov 2017 16:02:10 +0000 Subject: Add computed update docs for update_column_in_batches --- doc/development/migration_style_guide.md | 38 +++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 9b8ab5da74e..a235dd74909 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -198,7 +198,43 @@ end Keep in mind that this operation can easily take 10-15 minutes to complete on larger installations (e.g. GitLab.com). As a result you should only add default -values if absolutely necessary. +values if absolutely necessary. There is a RuboCop cop that will fail if this +method is used on some tables that are very large on GitLab.com, which would +cause other issues. + +## Updating an existing column + +To update an existing column to a particular value, you can use +`update_column_in_batches` (`add_column_with_default` uses this internally to +fill in the default value). This will split the updates into batches, so we +don't update too many rows at in a single statement. + +This updates the column `foo` in the `projects` table to 10, where `some_column` +is `'hello'`: + +```ruby +update_column_in_batches(:projects, :foo, 10) do |table, query| + query.where(table[:some_column].eq('hello')) +end +``` + +To perform a computed update, the value can be wrapped in `Arel.sql`, so Arel +treats it as an SQL literal. The below example is the same as the one above, but +the value is set to the product of the `bar` and `baz` columns: + +```ruby +update_value = Arel.sql('bar * baz') + +update_column_in_batches(:projects, :foo, update_value) do |table, query| + query.where(table[:some_column].eq('hello')) +end +``` + +Like `add_column_with_default`, there is a RuboCop cop to detect usage of this +on large tables. In the case of `update_column_in_batches`, it may be acceptable +to run on a large table, as long as it is only updating a small subset of the +rows in the table, but do not ignore that without validating on the GitLab.com +staging environment - or asking someone else to do so for you - beforehand. ## Integer column type -- cgit v1.2.1 From 5cf5832096311d152953c558bb588f4b65208a5f Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Fri, 17 Nov 2017 20:30:59 -0800 Subject: Remove executable bit from images for docs lint This was causing a docs lint failure in https://gitlab.com/gitlab-org/gitlab-ee/-/jobs/40901316. --- doc/user/discussions/img/image_resolved_discussion.png | Bin doc/user/discussions/img/onion_skin_view.png | Bin doc/user/discussions/img/swipe_view.png | Bin doc/user/discussions/img/two_up_view.png | Bin 4 files changed, 0 insertions(+), 0 deletions(-) mode change 100755 => 100644 doc/user/discussions/img/image_resolved_discussion.png mode change 100755 => 100644 doc/user/discussions/img/onion_skin_view.png mode change 100755 => 100644 doc/user/discussions/img/swipe_view.png mode change 100755 => 100644 doc/user/discussions/img/two_up_view.png (limited to 'doc') diff --git a/doc/user/discussions/img/image_resolved_discussion.png b/doc/user/discussions/img/image_resolved_discussion.png old mode 100755 new mode 100644 diff --git a/doc/user/discussions/img/onion_skin_view.png b/doc/user/discussions/img/onion_skin_view.png old mode 100755 new mode 100644 diff --git a/doc/user/discussions/img/swipe_view.png b/doc/user/discussions/img/swipe_view.png old mode 100755 new mode 100644 diff --git a/doc/user/discussions/img/two_up_view.png b/doc/user/discussions/img/two_up_view.png old mode 100755 new mode 100644 -- cgit v1.2.1 From b001487d94a131fa698d7feb1be8d9168c877e55 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Fri, 10 Nov 2017 17:14:54 -0800 Subject: Document how to troubleshoot internal API calls [ci skip] iFoo --- doc/administration/troubleshooting/debug.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) (limited to 'doc') diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index be538ea250a..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) -- cgit v1.2.1 From e0bb6985d12ce7dd10e1e7d9de0b77e1c756f57b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Elan=20Ruusam=C3=A4e?= Date: Sun, 19 Nov 2017 11:25:28 +0000 Subject: schedules.md: fix only-and-except anchor link --- doc/user/project/pipelines/schedules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index 9ad15a12c3c..eac706be3a7 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -44,7 +44,7 @@ GitLab CI so that they can be used in your `.gitlab-ci.yml` file. To configure that a job can be executed only when the pipeline has been scheduled (or the opposite), you can use -[only and except](../../../ci/yaml/README.md#only-and-except) configuration keywords. +[only and except](../../../ci/yaml/README.md#only-and-except-simplified) configuration keywords. ``` job:on-schedule: -- cgit v1.2.1 From 71c4f75d881c3b41e028fc1f3ab9f293156e3a8f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Mon, 20 Nov 2017 09:48:14 +0100 Subject: Fix a wrong anchor to job permissions doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Rémy Coutable --- doc/ci/git_submodules.md | 2 +- doc/user/project/new_ci_build_permissions_model.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index c83d3f6f248..286f3dee665 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -8,7 +8,7 @@ with the use of [SSH keys](ssh_keys/README.md). - With GitLab 8.12 onward, your permissions are used to evaluate what a CI job can access. More information about how this system works can be found in the - [Jobs permissions model](../user/permissions.md#jobs-permissions). + [Jobs permissions model](../user/permissions.md#job-permissions). - The HTTP(S) Git protocol [must be enabled][gitpro] in your GitLab instance. ## Configuring the `.gitmodules` file diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index 271adee7da1..17dcd152363 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -230,7 +230,7 @@ test: - docker run $CI_REGISTRY/group/other-project:latest ``` -[job permissions]: ../permissions.md#jobs-permissions +[job permissions]: ../permissions.md#job-permissions [comment]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22484#note_16648302 [ext]: ../permissions.md#external-users [gitsub]: ../../ci/git_submodules.md -- cgit v1.2.1 From 9400ed3b3e3bf1c1ce20b9748f4b6e4c0e5b5db7 Mon Sep 17 00:00:00 2001 From: Filipa Lacerda Date: Mon, 20 Nov 2017 09:57:08 +0000 Subject: Use axios instead of vue resource - step 1 --- doc/development/fe_guide/axios.md | 68 ++++++++++++++++++++++++++++++ doc/development/fe_guide/index.md | 4 +- doc/development/fe_guide/vue.md | 70 ++++++------------------------- doc/development/fe_guide/vue_resource.md | 72 -------------------------------- 4 files changed, 83 insertions(+), 131 deletions(-) create mode 100644 doc/development/fe_guide/axios.md delete mode 100644 doc/development/fe_guide/vue_resource.md (limited to 'doc') diff --git a/doc/development/fe_guide/axios.md b/doc/development/fe_guide/axios.md new file mode 100644 index 00000000000..962fe3dcec9 --- /dev/null +++ b/doc/development/fe_guide/axios.md @@ -0,0 +1,68 @@ +# Axios +We use [axios][axios] to communicate with the server in Vue applications and most new code. + +In order to guarantee all defaults are set you *should not use `axios` directly*, you should import `axios` from `axios_utils`. + +## CSRF token +All our request require a CSRF token. +To guarantee this token is set, we are importing [axios][axios], setting the token, and exporting `axios` . + +This exported module should be used instead of directly using `axios` to ensure the token is set. + +## Usage +```javascript + import axios from '~/lib/utils/axios_utils'; + + axios.get(url) + .then((response) => { + // `data` is the response that was provided by the server + const data = response.data; + + // `headers` the headers that the server responded with + // All header names are lower cased + const paginationData = response.headers; + }) + .catch(() => { + //handle the error + }); +``` + +## Mock axios response on tests + +To help us mock the responses we need we use [axios-mock-adapter][axios-mock-adapter] + + +```javascript + import axios from '~/lib/utils/axios_utils'; + import MockAdapter from 'axios-mock-adapter'; + + let mock; + beforeEach(() => { + // This sets the mock adapter on the default instance + mock = new MockAdapter(axios); + // Mock any GET request to /users + // arguments for reply are (status, data, headers) + mock.onGet('/users').reply(200, { + users: [ + { id: 1, name: 'John Smith' } + ] + }); + }); + + afterEach(() => { + mock.reset(); + }); +``` + +### Mock poll requests on tests with axios + +Because polling function requires an header object, we need to always include an object as the third argument: + +```javascript + mock.onGet('/users').reply(200, { foo: 'bar' }, {}); +``` + +[axios]: https://github.com/axios/axios +[axios-instance]: #creating-an-instance +[axios-interceptors]: https://github.com/axios/axios#interceptors +[axios-mock-adapter]: https://github.com/ctimmerm/axios-mock-adapter diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 73a03c07812..72cb557d054 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -71,8 +71,8 @@ Vue specific design patterns and practices. --- -## [Vue Resource](vue_resource.md) -Vue resource specific practices and gotchas. +## [Axios](axios.md) +Axios specific practices and gotchas. ## [Icons](icons.md) How we use SVG for our Icons. diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index f88f0753687..6e9f18dd1c3 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -178,16 +178,13 @@ itself, please read this guide: [State Management][state-management] The Service is a class used only to communicate with the server. It does not store or manipulate any data. It is not aware of the store or the components. -We use [vue-resource][vue-resource-repo] to communicate with the server. -Refer to [vue resource](vue_resource.md) for more details. +We use [axios][axios] to communicate with the server. +Refer to [axios](axios.md) for more details. -Vue Resource should only be imported in the service file. +Axios instance should only be imported in the service file. ```javascript - import Vue from 'vue'; - import VueResource from 'vue-resource'; - - Vue.use(VueResource); + import axios from 'javascripts/lib/utils/axios_utils'; ``` ### End Result @@ -230,15 +227,14 @@ export default class Store { } // service.js -import Vue from 'vue'; -import VueResource from 'vue-resource'; -import 'vue_shared/vue_resource_interceptor'; - -Vue.use(VueResource); +import axios from 'javascripts/lib/utils/axios_utils' export default class Service { constructor(options) { - this.todos = Vue.resource(endpoint.todosEndpoint); + this.todos = axios.create({ + baseURL: endpoint.todosEndpoint + }); + } getTodos() { @@ -477,50 +473,8 @@ The main return value of a Vue component is the rendered output. In order to tes need to test the rendered output. [Vue][vue-test] guide's to unit test show us exactly that: ### Stubbing API responses -[Vue Resource Interceptors][vue-resource-interceptor] allow us to add a interceptor with -the response we need: - - ```javascript - // Mock the service to return data - const interceptor = (request, next) => { - next(request.respondWith(JSON.stringify([{ - title: 'This is a todo', - body: 'This is the text' - }]), { - status: 200, - })); - }; +Refer to [mock axios](axios.md#mock-axios-response-on-tests) - beforeEach(() => { - Vue.http.interceptors.push(interceptor); - }); - - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, interceptor); - }); - - it('should do something', (done) => { - setTimeout(() => { - // Test received data - done(); - }, 0); - }); - ``` - -1. Headers interceptor -Refer to [this section](vue.md#headers) - -1. Use `$.mount()` to mount the component - -```javascript -// bad -new Component({ - el: document.createElement('div') -}); - -// good -new Component().$mount(); -``` ## Vuex To manage the state of an application you may use [Vuex][vuex-docs]. @@ -721,7 +675,6 @@ describe('component', () => { [component-system]: https://vuejs.org/v2/guide/#Composing-with-Components [state-management]: https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch [one-way-data-flow]: https://vuejs.org/v2/guide/components.html#One-Way-Data-Flow -[vue-resource-interceptor]: https://github.com/pagekit/vue-resource/blob/develop/docs/http.md#interceptors [vue-test]: https://vuejs.org/v2/guide/unit-testing.html [issue-boards-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/boards/services/board_service.js.es6 [flux]: https://facebook.github.io/flux @@ -729,3 +682,6 @@ describe('component', () => { [vuex-structure]: https://vuex.vuejs.org/en/structure.html [vuex-mutations]: https://vuex.vuejs.org/en/mutations.html [vuex-testing]: https://vuex.vuejs.org/en/testing.html +[axios]: https://github.com/axios/axios +[axios-interceptors]: https://github.com/axios/axios#interceptors + diff --git a/doc/development/fe_guide/vue_resource.md b/doc/development/fe_guide/vue_resource.md deleted file mode 100644 index c376c5c32bf..00000000000 --- a/doc/development/fe_guide/vue_resource.md +++ /dev/null @@ -1,72 +0,0 @@ -# Vue Resouce -In Vue applications we use [vue-resource][vue-resource-repo] to communicate with the server. - -## HTTP Status Codes - -### `.json()` -When making a request to the server, you will most likely need to access the body of the response. -Use `.json()` to convert. Because `.json()` returns a Promise the follwoing structure should be used: - - ```javascript - service.get('url') - .then(resp => resp.json()) - .then((data) => { - this.store.storeData(data); - }) - .catch(() => new Flash('Something went wrong')); - ``` - - -When using `Poll` (`app/assets/javascripts/lib/utils/poll.js`), the `successCallback` needs to handle `.json()` as a Promise: - ```javascript - successCallback: (response) => { - return response.json().then((data) => { - // handle the response - }); - } - ``` - -### 204 -Some endpoints - usually `delete` endpoints - return `204` as the success response. -When handling `204 - No Content` responses, we cannot use `.json()` since it tries to parse the non-existant body content. - -When handling `204` responses, do not use `.json`, otherwise the promise will throw an error and will enter the `catch` statement: - -```javascript - Vue.http.delete('path') - .then(() => { - // success! - }) - .catch(() => { - // handle error - }) -``` - -## Headers -Headers are being parsed into a plain object in an interceptor. -In Vue-resource 1.x `headers` object was changed into an `Headers` object. In order to not change all old code, an interceptor was added. - -If you need to write a unit test that takes the headers in consideration, you need to include an interceptor to parse the headers after your test interceptor. -You can see an example in `spec/javascripts/environments/environment_spec.js`: - ```javascript - import { headersInterceptor } from './helpers/vue_resource_helper'; - - beforeEach(() => { - Vue.http.interceptors.push(myInterceptor); - Vue.http.interceptors.push(headersInterceptor); - }); - - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, myInterceptor); - Vue.http.interceptors = _.without(Vue.http.interceptors, headersInterceptor); - }); - ``` - -## CSRF token -We use a Vue Resource interceptor to manage the CSRF token. -`app/assets/javascripts/vue_shared/vue_resource_interceptor.js` holds all our common interceptors. -Note: You don't need to load `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` -since it's already being loaded by `common_vue.js`. - - -[vue-resource-repo]: https://github.com/pagekit/vue-resource -- cgit v1.2.1 From 236feb08bc6b40aa67d498727c86c4178815424b Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Mon, 20 Nov 2017 12:29:11 +0100 Subject: Update pipeline status badge details docs --- doc/ci/yaml/README.md | 2 +- doc/user/project/pipelines/settings.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 6ad70707594..f40d2c5e347 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -124,7 +124,7 @@ stages: 1. First, all jobs of `build` are executed in parallel. 1. If all jobs of `build` succeed, the `test` jobs are executed in parallel. 1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel. -1. If all jobs of `deploy` succeed, the commit is marked as `success`. +1. If all jobs of `deploy` succeed, the commit is marked as `passed`. 1. If any of the previous jobs fails, the commit is marked as `failed` and no jobs of further stage are executed. diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 56f58fd755a..daa5463d680 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -115,10 +115,12 @@ pages. Depending on the status of your job, a badge can have the following values: +- pending - running -- success +- passed - failed - skipped +- canceled - unknown You can access a pipeline status badge image using the following link: -- cgit v1.2.1 From c82ae26565fdbffd205476929a2c08c9448a1365 Mon Sep 17 00:00:00 2001 From: Sean McGivern Date: Mon, 20 Nov 2017 16:09:56 +0000 Subject: Clarify wording of protected branch settings for the default branch No-one is allowed to force push to a protected branch, or delete it. That's correct in the documentation, but was wrong in the drop-down. --- doc/api/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/settings.md b/doc/api/settings.md index 4e24e4bbfc3..b27220f57f4 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -79,7 +79,7 @@ PUT /application/settings | `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `2`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and masters can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but masters can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | -- cgit v1.2.1 From faa57d785bb3807be9f0974390bd999bac4dcc6b Mon Sep 17 00:00:00 2001 From: DJ Mountney Date: Mon, 20 Nov 2017 11:20:51 -0800 Subject: Switch to using gettext:compile instead of gettext:pack in the install Docs This skips the step that updates the .pot files with changes in the codebase, which can take extra time, and isn't required during install. --- doc/install/installation.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 4efe911b778..88000f4c7a9 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -513,8 +513,7 @@ Check if GitLab and its environment are configured correctly: ### Compile GetText PO files - sudo -u git -H bundle exec rake gettext:pack RAILS_ENV=production - sudo -u git -H bundle exec rake gettext:po_to_json RAILS_ENV=production + sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production ### Compile Assets -- cgit v1.2.1 From 129036b2cf787f30264a99745cb87125b4b925e4 Mon Sep 17 00:00:00 2001 From: Victor Wu Date: Tue, 21 Nov 2017 15:21:50 +0000 Subject: Custom issue tracker --- doc/integration/external-issue-tracker.md | 1 + .../project/integrations/custom_issue_tracker.md | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) create mode 100644 doc/user/project/integrations/custom_issue_tracker.md (limited to 'doc') diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 372e1909330..075feaeead9 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -22,6 +22,7 @@ Visit the links below for details: - [Redmine](../user/project/integrations/redmine.md) - [Jira](../user/project/integrations/jira.md) - [Bugzilla](../user/project/integrations/bugzilla.md) +- [Custom Issue Tracker](../user/project/integrations/custom_issue_tracker.md) ### Service Template diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md new file mode 100644 index 00000000000..757522c2ae3 --- /dev/null +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -0,0 +1,20 @@ +# Custom Issue Tracker Service + +To enable the Custom Issue Tracker integration in a project, navigate to the +[Integrations page](project_services.md#accessing-the-project-services), click +the **Customer Issue Tracker** service, and fill in the required details on the page as described +in the table below. + +| Field | Description | +| ----- | ----------- | +| `title` | A title for the issue tracker (to differentiate between instances, for example) | +| `description` | A name for the issue tracker (to differentiate between instances, for example) | +| `project_url` | Currently unused. Will be changed in a future release. | +| `issues_url` | The URL to the issue in the issue tracker project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. For example, `https://customissuetracker.com/project-name/:id`. | +| `new_issue_url` | Currently unused. Will be changed in a future release. | + + +## Referencing issues + +Issues are referenced with `#`, where `` is a number (example `#143`). +So with the example above, `#143` would refer to `https://customissuetracker.com/project-name/143`. \ No newline at end of file -- cgit v1.2.1 From 0b9e1e16626eff4cd8ae43ce47ec0f965beaf843 Mon Sep 17 00:00:00 2001 From: Daniel Juarez Date: Tue, 21 Nov 2017 15:47:58 +0000 Subject: Skip confirmation user api --- doc/api/users.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/api/users.md b/doc/api/users.md index aa711090af1..478d747a50d 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -297,6 +297,7 @@ Parameters: - `location` (optional) - User's location - `admin` (optional) - User is admin - true or false (default) - `can_create_group` (optional) - User can create groups - true or false +- `skip_reconfirmation` (optional) - Skip reconfirmation - true or false (default) - `external` (optional) - Flags the user as external - true or false(default) - `avatar` (optional) - Image file for user's avatar -- cgit v1.2.1 From 131e74d10dafbf2b781ab5d5517e42a18e20a587 Mon Sep 17 00:00:00 2001 From: "Vitaliy @blackst0ne Klachkov" Date: Wed, 22 Nov 2017 14:12:04 +1100 Subject: Add support of Mermaid --- doc/user/markdown.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) (limited to 'doc') diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 454988b9b80..fb61e360996 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -368,6 +368,37 @@ _Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._ >**Note:** This also works for the asciidoctor `:stem: latexmath`. For details see the [asciidoctor user manual][asciidoctor-manual]. +### Mermaid + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#mermaid + +It is possible to generate diagrams and flowcharts from text using [Mermaid][mermaid]. + +In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. + +Example: + + ```mermaid + graph TD; + A-->B; + A-->C; + B-->D; + C-->D; + ``` + +Becomes: + +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` + +For details see the [Mermaid official page][mermaid]. + ## Standard Markdown ### Headers @@ -814,6 +845,7 @@ A link starting with a `/` is relative to the wiki root. [^2]: This is my awesome footnote. [markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md +[mermaid]: https://mermaidjs.github.io/ "Mermaid website" [rouge]: http://rouge.jneen.net/ "Rouge website" [redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" [katex]: https://github.com/Khan/KaTeX "KaTeX website" -- cgit v1.2.1 From 52b6cbcb9df7e2c968da3e9a9779500cacfac465 Mon Sep 17 00:00:00 2001 From: Stan Hu Date: Mon, 20 Nov 2017 09:18:15 -0800 Subject: Add QUERY_RECORDER_DEBUG environment variable to improve performance debugging --- doc/development/query_recorder.md | 46 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) (limited to 'doc') diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md index e0127aaed4c..12e90101139 100644 --- a/doc/development/query_recorder.md +++ b/doc/development/query_recorder.md @@ -22,6 +22,52 @@ As an example you might create 5 issues in between counts, which would cause the > **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. +## Finding the source of the query + +It may be useful to identify the source of the queries by looking at the call backtrace. +To enable this, run the specs with the `QUERY_RECORDER_DEBUG` environment variable set. For example: + +``` +QUERY_RECORDER_DEBUG=1 bundle exec rspec spec/requests/api/projects_spec.rb +``` + +This will log calls to QueryRecorder into the `test.log`. For example: + +``` +QueryRecorder SQL: SELECT COUNT(*) FROM "issues" WHERE "issues"."deleted_at" IS NULL AND "issues"."project_id" = $1 AND ("issues"."state" IN ('opened')) AND "issues"."confidential" = $2 + --> /home/user/gitlab/gdk/gitlab/spec/support/query_recorder.rb:19:in `callback' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:127:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `block in finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `each' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/fanout.rb:46:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:36:in `finish' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications/instrumenter.rb:25:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:601:in `exec_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql_adapter.rb:585:in `execute_and_clear' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `block in select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:83:in `cache_sql' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `select_all' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:270:in `execute_simple_calculation' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:227:in `perform_calculation' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:133:in `calculate' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/relation/calculations.rb:48:in `count' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:20:in `uncached_count' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:12:in `block in count' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:299:in `block in fetch' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:585:in `block in save_block_result_to_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:547:in `block in instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/notifications.rb:166:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:547:in `instrument' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:584:in `save_block_result_to_cache' + --> /home/user/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/activesupport-4.2.8/lib/active_support/cache.rb:299:in `fetch' + --> /home/user/gitlab/gdk/gitlab/app/services/base_count_service.rb:12:in `count' + --> /home/user/gitlab/gdk/gitlab/app/models/project.rb:1296:in `open_issues_count' +``` + ## See also - [Bullet](profiling.md#Bullet) For finding `N+1` query problems -- cgit v1.2.1 From 0b22169fc33854f32699ad9fe5f9a9dc98318f2f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 22 Nov 2017 17:14:08 +0100 Subject: Cross link default branch docs to issue closing pattern docs --- doc/user/project/issues/automatic_issue_closing.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/user/project/issues/automatic_issue_closing.md b/doc/user/project/issues/automatic_issue_closing.md index 402a2a3c727..b9607243c8a 100644 --- a/doc/user/project/issues/automatic_issue_closing.md +++ b/doc/user/project/issues/automatic_issue_closing.md @@ -12,8 +12,9 @@ in the project's default branch. If a commit message or merge request description contains a sentence matching a certain regular expression, all issues referenced from the matched text will -be closed. This happens when the commit is pushed to a project's **default** -branch, or when a commit or merge request is merged into it. +be closed. This happens when the commit is pushed to a project's +[**default** branch](../repository/branches/index.md#default-branch), or when a +commit or merge request is merged into it. ## Default closing pattern value -- cgit v1.2.1 From 3f39d787953263d8102437a1f9838548d94eac43 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 23 Nov 2017 11:22:33 +0100 Subject: Fix the redirect location wording Closes https://gitlab.com/gitlab-com/gitlab-docs/issues/142 --- doc/administration/repository_storages.md | 4 +--- doc/development/doc_styleguide.md | 6 +++--- doc/workflow/importing/README.md | 2 +- doc/workflow/importing/import_projects_from_bitbucket.md | 2 +- doc/workflow/importing/import_projects_from_fogbugz.md | 2 +- doc/workflow/importing/import_projects_from_gitea.md | 2 +- doc/workflow/importing/import_projects_from_github.md | 2 +- doc/workflow/importing/import_projects_from_gitlab_com.md | 2 +- doc/workflow/importing/migrating_from_svn.md | 2 +- 9 files changed, 11 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/doc/administration/repository_storages.md b/doc/administration/repository_storages.md index 9d41ba77f34..cf6de15743f 100644 --- a/doc/administration/repository_storages.md +++ b/doc/administration/repository_storages.md @@ -1,3 +1 @@ -# Repository storages - -This document was moved to a [new location](repository_storage_paths.md). +This document was moved to [another location](repository_storage_paths.md). diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 0e4ffbd7910..aaa7032cadb 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -303,10 +303,10 @@ GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the Documentation team beforehand. If you indeed need to change a document's location, do NOT remove the old -document, but rather put a text in it that points to the new location, like: +document, but rather replace all of its contents with a new line: ``` -This document was moved to [path/to/new_doc.md](path/to/new_doc.md). +This document was moved to [another location](path/to/new_doc.md). ``` where `path/to/new_doc.md` is the relative path to the root directory `doc/`. @@ -320,7 +320,7 @@ For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to 1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: ``` - This document was moved to [administration/lfs.md](../../administration/lfs.md). + This document was moved to [another location](../../administration/lfs.md). ``` 1. Find and replace any occurrences of the old location with the new one. diff --git a/doc/workflow/importing/README.md b/doc/workflow/importing/README.md index f753708ad89..e0a445920b4 100644 --- a/doc/workflow/importing/README.md +++ b/doc/workflow/importing/README.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/index.md). +This document was moved to [another location](../../user/project/import/index.md). diff --git a/doc/workflow/importing/import_projects_from_bitbucket.md b/doc/workflow/importing/import_projects_from_bitbucket.md index 248c3990372..ec9a11f390e 100644 --- a/doc/workflow/importing/import_projects_from_bitbucket.md +++ b/doc/workflow/importing/import_projects_from_bitbucket.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/bitbucket.md). +This document was moved to [another location](../../user/project/import/bitbucket.md). diff --git a/doc/workflow/importing/import_projects_from_fogbugz.md b/doc/workflow/importing/import_projects_from_fogbugz.md index 050746e2b4d..876eb0434f0 100644 --- a/doc/workflow/importing/import_projects_from_fogbugz.md +++ b/doc/workflow/importing/import_projects_from_fogbugz.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/fogbugz.md). +This document was moved to [another location](../../user/project/import/fogbugz.md). diff --git a/doc/workflow/importing/import_projects_from_gitea.md b/doc/workflow/importing/import_projects_from_gitea.md index cb90c490b0f..8b55b6c23eb 100644 --- a/doc/workflow/importing/import_projects_from_gitea.md +++ b/doc/workflow/importing/import_projects_from_gitea.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/gitea.md). +This document was moved to [another location](../../user/project/import/gitea.md). diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md index 13639feaa04..72dfe5403c3 100644 --- a/doc/workflow/importing/import_projects_from_github.md +++ b/doc/workflow/importing/import_projects_from_github.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/github.md). +This document was moved to [another location](../../user/project/import/github.md). diff --git a/doc/workflow/importing/import_projects_from_gitlab_com.md b/doc/workflow/importing/import_projects_from_gitlab_com.md index df4c180919a..3256088c014 100644 --- a/doc/workflow/importing/import_projects_from_gitlab_com.md +++ b/doc/workflow/importing/import_projects_from_gitlab_com.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/gitlab_com.md). +This document was moved to [another location](../../user/project/import/gitlab_com.md). diff --git a/doc/workflow/importing/migrating_from_svn.md b/doc/workflow/importing/migrating_from_svn.md index 81df3fbcdbb..32a75a6c6af 100644 --- a/doc/workflow/importing/migrating_from_svn.md +++ b/doc/workflow/importing/migrating_from_svn.md @@ -1 +1 @@ -This document was moved to a [new location](../../user/project/import/svn.md). +This document was moved to [another location](../../user/project/import/svn.md). -- cgit v1.2.1 From 07e42c5c425b5a781398fea2aaf7eeb8a825724f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Thu, 23 Nov 2017 13:17:10 +0100 Subject: Clarify the docs review process and mention the supported repos --- doc/development/writing_documentation.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 68ba3dd2da3..b6def7ef541 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -152,12 +152,23 @@ CE and EE. ## Previewing the changes live If you want to preview the doc changes of your merge request live, you can use -the manual `review-docs-deploy` job in your merge request. +the manual `review-docs-deploy` job in your merge request. You will need at +least Master permissions to be able to run it and is currently enabled for the +following projects: + +- https://gitlab.com/gitlab-org/gitlab-ce +- https://gitlab.com/gitlab-org/gitlab-ee + +NOTE: **Note:** +You will need to push a branch to those repositories, it doesn't work for forks. TIP: **Tip:** If your branch contains only documentation changes, you can use [special branch names](#testing) to avoid long running pipelines. +In the mini pipeline graph, you should see an `>>` icon. Clicking on it will +reveal the `review-docs-deploy` job. Hit the play button for the job to start. + ![Manual trigger a docs build](img/manual_build_docs.png) This job will: -- cgit v1.2.1 From 257fd5713485a05460a9170190100643199a7e48 Mon Sep 17 00:00:00 2001 From: Markus Koller Date: Thu, 23 Nov 2017 13:16:14 +0000 Subject: Allow password authentication to be disabled entirely --- doc/administration/auth/ldap.md | 6 ++++++ doc/api/settings.md | 7 ++++--- 2 files changed, 10 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index ad903aef896..6b5a0f139c5 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -30,6 +30,12 @@ immediately block all access. >**Note**: GitLab EE supports a configurable sync time, with a default of one hour. +## Git password authentication + +LDAP-enabled users can always authenticate with Git using their GitLab username +or email and LDAP password, even if password authentication for Git is disabled +in the application settings. + ## Configuration To enable LDAP integration you need to add your LDAP server settings in diff --git a/doc/api/settings.md b/doc/api/settings.md index b27220f57f4..22fb2baa8ec 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -25,7 +25,7 @@ Example response: "id" : 1, "default_branch_protection" : 2, "restricted_visibility_levels" : [], - "password_authentication_enabled" : true, + "password_authentication_enabled_for_web" : true, "after_sign_out_path" : null, "max_attachment_size" : 10, "user_oauth_applications" : true, @@ -117,7 +117,8 @@ PUT /application/settings | `metrics_port` | integer | no | The UDP port to use for connecting to InfluxDB | | `metrics_sample_interval` | integer | yes (if `metrics_enabled` is `true`) | The sampling interval in seconds. | | `metrics_timeout` | integer | yes (if `metrics_enabled` is `true`) | The amount of seconds after which InfluxDB will time out. | -| `password_authentication_enabled` | boolean | no | Enable authentication via a GitLab account password. Default is `true`. | +| `password_authentication_enabled_for_web` | boolean | no | Enable authentication for the web interface via a GitLab account password. Default is `true`. | +| `password_authentication_enabled_for_git` | boolean | no | Enable authentication for Git over HTTP(S) via a GitLab account password. Default is `true`. | | `performance_bar_allowed_group_id` | string | no | The group that is allowed to enable the performance bar | | `performance_bar_enabled` | boolean | no | Allow enabling the performance bar | | `plantuml_enabled` | boolean | no | Enable PlantUML integration. Default is `false`. | @@ -165,7 +166,7 @@ Example response: "id": 1, "default_projects_limit": 100000, "signup_enabled": true, - "password_authentication_enabled": true, + "password_authentication_enabled_for_web": true, "gravatar_enabled": true, "sign_in_text": "", "created_at": "2015-06-12T15:51:55.432Z", -- cgit v1.2.1 From 10c2ba7dbbae15cde019135fe89b20dbd793dadf Mon Sep 17 00:00:00 2001 From: Gabriel Mazetto Date: Sat, 18 Nov 2017 08:12:37 +0100 Subject: Changelog and Documentation for storage migration of project attachments --- doc/administration/raketasks/storage.md | 97 ++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index bac8fa4bd9d..6ec5baeb6e3 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -1,10 +1,43 @@ # Repository Storage Rake Tasks This is a collection of rake tasks you can use to help you list and migrate -existing projects from Legacy storage to the new Hashed storage type. +existing projects and attachments associated with it from Legacy storage to +the new Hashed storage type. You can read more about the storage types [here][storage-types]. +## Migrate existing projects to Hashed storage + +Before migrating your existing projects, you should +[enable hashed storage][storage-migration] for the new projects as well. + +This task will schedule all your existing projects and attachments associated with it to be migrated to the +**Hashed** storage type: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:migrate_to_hashed +``` + +**Source Installation** + +```bash +rake gitlab:storage:migrate_to_hashed + +``` + +You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. +There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** + +After it reaches zero, you can confirm every project has been migrated by running the commands bellow. +If you find it necessary, you can run this migration script again to schedule missing projects. + +Any error or warning will be logged in the sidekiq's log file. + +You only need the `gitlab:storage:migrate_to_hashed` rake task to migrate your repositories, but we have additional +commands below that helps you inspect projects and attachments in both legacy and hashed storage. + ## List projects on Legacy storage To have a simple summary of projects using **Legacy** storage: @@ -73,35 +106,73 @@ rake gitlab:storage:list_hashed_projects ``` -## Migrate existing projects to Hashed storage +## List attachments on Legacy storage -Before migrating your existing projects, you should -[enable hashed storage][storage-migration] for the new projects as well. +To have a simple summary of project attachments using **Legacy** storage: -This task will schedule all your existing projects to be migrated to the -**Hashed** storage type: +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:legacy_attachments +``` + +**Source Installation** + +```bash +rake gitlab:storage:legacy_attachments + +``` + +------ + +To list project attachments using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:migrate_to_hashed +gitlab-rake gitlab:storage:list_legacy_attachments ``` **Source Installation** ```bash -rake gitlab:storage:migrate_to_hashed +rake gitlab:storage:list_legacy_attachments ``` -You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. -There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** +## List attachments on Hashed storage -After it reaches zero, you can confirm every project has been migrated by running the commands above. -If you find it necessary, you can run this migration script again to schedule missing projects. +To have a simple summary of project attachments using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:hashed_attachments +``` -Any error or warning will be logged in the sidekiq log file. +**Source Installation** + +```bash +rake gitlab:storage:hashed_attachments + +``` + +------ + +To list project attachments using **Hashed** storage: + +**Omnibus Installation** +```bash +gitlab-rake gitlab:storage:list_hashed_attachments +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_hashed_attachments + +``` [storage-types]: ../repository_storage_types.md [storage-migration]: ../repository_storage_types.md#how-to-migrate-to-hashed-storage -- cgit v1.2.1 From 392cc887097bfd0c28b547700b9a67c32cf14e69 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Fri, 17 Nov 2017 02:43:55 +0100 Subject: Add new API endpoint - get a namespace by ID --- doc/api/namespaces.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) (limited to 'doc') diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 5c0bebbaeb0..d06b0d31c67 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -89,3 +89,35 @@ Example response: } ] ``` + +## Get namespace by ID + +Get a namespace by ID. + +``` +GET /namespaces/:id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | ID of the namespace | + +Example request: + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/namespaces/2 +``` + +Example response: + +```json +{ + "id": 2, + "name": "group1", + "path": "group1", + "kind": "group", + "full_path": "group1", + "parent_id": "null", + "members_count_with_descendants": 2 +} +``` -- cgit v1.2.1 From c69e9d28c0b86951d3e5f85bda8b1f85f4502416 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 10 Nov 2017 10:45:38 +0100 Subject: Add article on autoscaling GitLab Runner on AWS --- doc/articles/runner_autoscale_aws/index.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/articles/runner_autoscale_aws/index.md (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md new file mode 100644 index 00000000000..25af5e3d02b --- /dev/null +++ b/doc/articles/runner_autoscale_aws/index.md @@ -0,0 +1,15 @@ +--- +last_updated: 2017-11-10 +--- + +# Autoscaling GitLab Runner on AWS + +## Introduction + +## Installation and configuration + +## Save money with AWS spot instances + +### Caveats of spot instances + +## Conclusion -- cgit v1.2.1 From 61eed644888b90306c716cee2beea25a022be10e Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 10 Nov 2017 15:51:02 +0100 Subject: Add skeleton --- doc/articles/index.md | 1 + doc/articles/runner_autoscale_aws/index.md | 115 ++++++++++++++++++++++++++++- 2 files changed, 115 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/articles/index.md b/doc/articles/index.md index 798d4cbf4ff..bb941fc0023 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -26,6 +26,7 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | Article title | Category | Publishing date | | :------------ | :------: | --------------: | +| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Tutorial | 2017-11-10 | | [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 25af5e3d02b..9e4bfe06bce 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -4,12 +4,125 @@ last_updated: 2017-11-10 # Autoscaling GitLab Runner on AWS +GitLab Runner has the ability to autoscale which means automatically spawning +new machines on demand. This proves very useful in situations where you +don't use your Runners 24/7 and want to have a cost-effective and scalable +solution. + ## Introduction +In this tutorial, we'll explore how to properly configure a GitLab Runner in +AWS that will serve as the bastion where it will spawn new Docker machines on +demand. + ## Installation and configuration -## Save money with AWS spot instances +The bastion will not run any jobs itself. + +Edit `/etc/gitlab-runner/config.toml`: + +```toml +concurrent = 3 +check_interval = 0 + +[[runners]] + name = "gitlab-aws-autoscaler" + url = "" + token = "" + executor = "docker+machine" + environment = ["GODEBUG=netdns=cgo"] + output_limit = 16384 + limit = 4 + [runners.docker] + image = "ruby:2.1" + privileged = true + disable_cache = false + volumes = ["/cache"] + extra_hosts = ["gitlab.thehumangeo.com:", "nexus.thehumangeo.com:"] + [runners.cache] + Type = "s3" + ServerAddress = "s3.amazonaws.com" + AccessKey = "" + SecretKey = "" + BucketName = "" + BucketLocation = "us-east-1" + Shared = true + [runners.machine] + IdleCount = 1 + IdleTime = 1800 + MaxBuilds = 100 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" + OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] + OffPeakIdleCount = 0 + OffPeakIdleTime = 1200 + MachineOptions = [ + "amazonec2-access-key=XXXX", + "amazonec2-secret-key=XXXX", + "amazonec2-region=us-east-1", + "amazonec2-vpc-id=vpc-xxxxx", + "amazonec2-subnet-id=subnet-xxxxx", + "amazonec2-use-private-address=true", + "amazonec2-tags=Name,gitlab-runner-autoscale", + "amazonec2-security-group=docker-machine-scaler", + "amazonec2-instance-type=m4.2xlarge", + "amazonec2-ssh-user=ubuntu", + "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", + "amazonec2-ami=ami-996372fd", + "amazonec2-zone=a", + "amazonec2-root-size=32", + ] +``` + +## Cutting costs with AWS spot instances + +> +Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity. +Since Spot instances are often available at a discount compared to On-Demand +pricing, you can significantly reduce the cost of running your applications, +grow your application’s compute capacity and throughput for the same budget, +and enable new types of cloud computing applications. + +- https://aws.amazon.com/ec2/spot/ +- https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html +- https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/ + + +In `/etc/gitlab-runner/config.toml` under the `MachineOptions` section: + +```toml + MachineOptions = [ + "amazonec2-request-spot-instance=true", + "amazonec2-spot-price=0.03", + "amazonec2-block-duration-minutes=180" + ] +``` ### Caveats of spot instances +If the spot price raises, the auto-scale Runner would fail to create new machines. + +This eventually eats 60 requests and then AWS won't accept any more. Then once +the spot price is acceptable, you are locked out for a bit because the call amount +limit is exceeded. + +You can use the following command in the bastion machine to see the Docker Machines +state: + +```sh +docker-machine ls -q --filter state=Error --format "{{.NAME}}" +``` + +NOTE: **Note:** +Follow [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) +for more information. + ## Conclusion + +Using the autoscale feature of GitLab Runner can save you both time and money. +Using the spot instances that AWS provides can save you even more. + +You can read the following user cases from which this tutorial was influenced: + +- [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html) +- [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/) -- cgit v1.2.1 From 2343d3a5a4914dca8c0c65f9e96edaafc963b344 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 13 Nov 2017 15:25:46 +0100 Subject: Clear up sections --- doc/articles/runner_autoscale_aws/index.md | 73 ++++++++++++++++++++++-------- 1 file changed, 53 insertions(+), 20 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 9e4bfe06bce..79e8c329ec9 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -4,10 +4,11 @@ last_updated: 2017-11-10 # Autoscaling GitLab Runner on AWS -GitLab Runner has the ability to autoscale which means automatically spawning -new machines on demand. This proves very useful in situations where you -don't use your Runners 24/7 and want to have a cost-effective and scalable -solution. +One of the biggest advantages of GitLab Runner is its ability to automatically +spin up and down VMs to make sure your builds get processed immediately. It's a +great feature, and if used correctly, it can be extremely useful in situations +where you don't use your Runners 24/7 and want to have a cost-effective and +scalable solution. ## Introduction @@ -15,9 +16,35 @@ In this tutorial, we'll explore how to properly configure a GitLab Runner in AWS that will serve as the bastion where it will spawn new Docker machines on demand. -## Installation and configuration +In addition, we'll make use of [Amazon's EC2 Spot instances](https://aws.amazon.com/ec2/spot/) +which will greatly reduce the costs of the Runner instances while still using +quite powerful autoscaling machines. -The bastion will not run any jobs itself. +## Prerequisites + +The first step is to install GitLab Runner in an EC2 instance that will serve +as the bastion to spawning new machines. This doesn't have to be a powerful +machine since it will not run any jobs itself, a `t2.micro` instance will do. +This machine will be a dedicated host since we need it always up and running, +thus it will be the only standard cost. + +NOTE: **Note:** +For the bastion instance, choose a distribution that both Docker and GitLab +Runner support, for example either Ubuntu, Debian, CentOS or RHEL will work fine. + +Install the prerequisites: + +1. Log in your server +1. [Install GitLab Runner from the official GitLab repository](https://docs.gitlab.com/runner/install/linux-repository.html) +1. [Install Docker](https://docs.docker.com/engine/installation/#server) +1. [Install Docker Machine](https://docs.docker.com/machine/install-machine/) + +You can now move on to the most important part, configuring GitLab Runner. + +## Configuring GitLab Runner to use the AWS machine driver + +Before configuring the GitLab Runner, you need to first register it, so that +it connects with your GitLab instance. Edit `/etc/gitlab-runner/config.toml`: @@ -30,15 +57,12 @@ check_interval = 0 url = "" token = "" executor = "docker+machine" - environment = ["GODEBUG=netdns=cgo"] - output_limit = 16384 limit = 4 [runners.docker] - image = "ruby:2.1" + image = "alpine" privileged = true disable_cache = false volumes = ["/cache"] - extra_hosts = ["gitlab.thehumangeo.com:", "nexus.thehumangeo.com:"] [runners.cache] Type = "s3" ServerAddress = "s3.amazonaws.com" @@ -68,13 +92,17 @@ check_interval = 0 "amazonec2-instance-type=m4.2xlarge", "amazonec2-ssh-user=ubuntu", "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", - "amazonec2-ami=ami-996372fd", "amazonec2-zone=a", "amazonec2-root-size=32", ] ``` -## Cutting costs with AWS spot instances +Under `MachineOptions` you can add anything that the [AWS Docker Machine driver +supports](https://docs.docker.com/machine/drivers/aws/#options). + +## Cutting down costs with Amazon EC2 Spot instances + +As described by Amazon: > Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity. @@ -83,24 +111,29 @@ pricing, you can significantly reduce the cost of running your applications, grow your application’s compute capacity and throughput for the same budget, and enable new types of cloud computing applications. -- https://aws.amazon.com/ec2/spot/ -- https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html -- https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/ - - In `/etc/gitlab-runner/config.toml` under the `MachineOptions` section: ```toml MachineOptions = [ "amazonec2-request-spot-instance=true", "amazonec2-spot-price=0.03", - "amazonec2-block-duration-minutes=180" + "amazonec2-block-duration-minutes=60" ] ``` -### Caveats of spot instances +With this configuration, Docker Machines are created on Spot instances with a +maximum bid price of $0.03 per hour and the duration of the Spot instance is +capped at 60 minutes. + +To learn more about Amazon EC2 Spot instances, visit the following links: + +- https://aws.amazon.com/ec2/spot/ +- https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html +- https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/ + +### Caveats of Spot instances -If the spot price raises, the auto-scale Runner would fail to create new machines. +If the Spot price raises, the auto-scale Runner would fail to create new machines. This eventually eats 60 requests and then AWS won't accept any more. Then once the spot price is acceptable, you are locked out for a bit because the call amount -- cgit v1.2.1 From 801f6ac42921819ab854d3277d89dd126b9efc78 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Mon, 13 Nov 2017 18:28:49 +0100 Subject: Add Runner registration info --- doc/articles/runner_autoscale_aws/index.md | 94 ++++++++++++++++++++++++++++-- 1 file changed, 89 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 79e8c329ec9..fe1b4b2c897 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -39,14 +39,26 @@ Install the prerequisites: 1. [Install Docker](https://docs.docker.com/engine/installation/#server) 1. [Install Docker Machine](https://docs.docker.com/machine/install-machine/) +Before configuring the GitLab Runner, you need to first register it, so that +it connects with your GitLab instance: + +1. [Obtain a Runner token](../../ci/runners/README.md) +1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux) +1. When asked the executor type, enter `docker+machine` + +TIP: **Tip:** +If you want every user in your instance to be able to use the autoscaled Runners, +register the Runner as a shared one. + You can now move on to the most important part, configuring GitLab Runner. ## Configuring GitLab Runner to use the AWS machine driver -Before configuring the GitLab Runner, you need to first register it, so that -it connects with your GitLab instance. +Now that the Runner is registered, you need to edit its configuration file and +add the required options for the AWS machine driver. -Edit `/etc/gitlab-runner/config.toml`: +Here's a full example of `/etc/gitlab-runner/config.toml`. Open it with your +editor and edit as you see fit: ```toml concurrent = 3 @@ -97,8 +109,80 @@ check_interval = 0 ] ``` -Under `MachineOptions` you can add anything that the [AWS Docker Machine driver -supports](https://docs.docker.com/machine/drivers/aws/#options). +Let's break it down to pieces. + +- Global section + + ```toml + concurrent = 3 + check_interval = 0 + ``` + +- `[[runners]]` + + ```toml + [[runners]] + name = "gitlab-aws-autoscaler" + url = "" + token = "" + executor = "docker+machine" + limit = 4 + ``` + +- `[runners.docker]` + + ```toml + [runners.docker] + image = "alpine" + privileged = true + disable_cache = false + volumes = ["/cache"] + ``` + +- `[runners.cache]` + + ```toml + [runners.cache] + Type = "s3" + ServerAddress = "s3.amazonaws.com" + AccessKey = "" + SecretKey = "" + BucketName = "" + BucketLocation = "us-east-1" + Shared = true + ``` + +- `[runners.machine]` + + ```toml + [runners.machine] + IdleCount = 1 + IdleTime = 1800 + MaxBuilds = 100 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" + OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] + OffPeakIdleCount = 0 + OffPeakIdleTime = 1200 + MachineOptions = [ + "amazonec2-access-key=XXXX", + "amazonec2-secret-key=XXXX", + "amazonec2-region=us-east-1", + "amazonec2-vpc-id=vpc-xxxxx", + "amazonec2-subnet-id=subnet-xxxxx", + "amazonec2-use-private-address=true", + "amazonec2-tags=Name,gitlab-runner-autoscale", + "amazonec2-security-group=docker-machine-scaler", + "amazonec2-instance-type=m4.2xlarge", + "amazonec2-ssh-user=ubuntu", + "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", + "amazonec2-zone=a", + "amazonec2-root-size=32", + ] + ``` + + Under `MachineOptions` you can add anything that the [AWS Docker Machine driver + supports](https://docs.docker.com/machine/drivers/aws/#options). ## Cutting down costs with Amazon EC2 Spot instances -- cgit v1.2.1 From e992471026a27841dcfe74ccbfd3122ffde3bfc1 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 14 Nov 2017 09:51:41 +0100 Subject: Add runners.cache section --- doc/articles/runner_autoscale_aws/index.md | 162 ++++++++++++++++------------- 1 file changed, 90 insertions(+), 72 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index fe1b4b2c897..40622aaac15 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -111,78 +111,96 @@ check_interval = 0 Let's break it down to pieces. -- Global section - - ```toml - concurrent = 3 - check_interval = 0 - ``` - -- `[[runners]]` - - ```toml - [[runners]] - name = "gitlab-aws-autoscaler" - url = "" - token = "" - executor = "docker+machine" - limit = 4 - ``` - -- `[runners.docker]` - - ```toml - [runners.docker] - image = "alpine" - privileged = true - disable_cache = false - volumes = ["/cache"] - ``` - -- `[runners.cache]` - - ```toml - [runners.cache] - Type = "s3" - ServerAddress = "s3.amazonaws.com" - AccessKey = "" - SecretKey = "" - BucketName = "" - BucketLocation = "us-east-1" - Shared = true - ``` - -- `[runners.machine]` - - ```toml - [runners.machine] - IdleCount = 1 - IdleTime = 1800 - MaxBuilds = 100 - MachineDriver = "amazonec2" - MachineName = "gitlab-docker-machine-%s" - OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] - OffPeakIdleCount = 0 - OffPeakIdleTime = 1200 - MachineOptions = [ - "amazonec2-access-key=XXXX", - "amazonec2-secret-key=XXXX", - "amazonec2-region=us-east-1", - "amazonec2-vpc-id=vpc-xxxxx", - "amazonec2-subnet-id=subnet-xxxxx", - "amazonec2-use-private-address=true", - "amazonec2-tags=Name,gitlab-runner-autoscale", - "amazonec2-security-group=docker-machine-scaler", - "amazonec2-instance-type=m4.2xlarge", - "amazonec2-ssh-user=ubuntu", - "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", - "amazonec2-zone=a", - "amazonec2-root-size=32", - ] - ``` - - Under `MachineOptions` you can add anything that the [AWS Docker Machine driver - supports](https://docs.docker.com/machine/drivers/aws/#options). +### Global section + +```toml +concurrent = 3 +check_interval = 0 +``` + +### `[[runners]]` + +```toml +[[runners]] + name = "gitlab-aws-autoscaler" + url = "" + token = "" + executor = "docker+machine" + limit = 4 +``` + +### `[runners.docker]` + +```toml + [runners.docker] + image = "alpine" + privileged = true + disable_cache = false + volumes = ["/cache"] +``` + +### `[runners.cache]` + +To speed up your jobs, GitLab Runner provides a cache mechanism where selected +directories and/or files are saved and shared between subsequent jobs. +While not required for this setup, it is recommended to use the shared cache +mechanism that GitLab Runner provides. Since new instances will be created on +demand, it is essential to have a common place where cache is stored. + +In the following example, we use Amazon S3: + +```toml + [runners.cache] + Type = "s3" + ServerAddress = "s3.amazonaws.com" + AccessKey = "" + SecretKey = "" + BucketName = "" + BucketLocation = "us-east-1" + Shared = true +``` + +Here's some more info to get you started: + +- [The `[runners.cache]` section](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) +- [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) +- [How cache works](../../ci/yaml/README.md#cache) + +### `[runners.machine]` + +This is the most important part of the configuration and it's the one that +tells GitLab Runner how and when to spawn new Docker Machine instances. + +```toml + [runners.machine] + IdleCount = 1 + IdleTime = 1800 + MaxBuilds = 100 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" + OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] + OffPeakIdleCount = 0 + OffPeakIdleTime = 1200 + MachineOptions = [ + "amazonec2-access-key=XXXX", + "amazonec2-secret-key=XXXX", + "amazonec2-region=us-east-1", + "amazonec2-vpc-id=vpc-xxxxx", + "amazonec2-subnet-id=subnet-xxxxx", + "amazonec2-use-private-address=true", + "amazonec2-tags=Name,gitlab-runner-autoscale", + "amazonec2-security-group=docker-machine-scaler", + "amazonec2-instance-type=m4.2xlarge", + "amazonec2-ssh-user=ubuntu", + "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", + "amazonec2-zone=a", + "amazonec2-root-size=32", + ] +``` + +TIP: **Tip:** +Under `MachineOptions` you can add anything that the [AWS Docker Machine driver +supports](https://docs.docker.com/machine/drivers/aws/#options). ## Cutting down costs with Amazon EC2 Spot instances -- cgit v1.2.1 From 675ea33350d8ee57953487b8f632e0fb4dc6727f Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Tue, 14 Nov 2017 13:18:32 +0100 Subject: Finish all sections except docker.machine --- doc/articles/runner_autoscale_aws/index.md | 58 ++++++++++++++++++++++-------- 1 file changed, 44 insertions(+), 14 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 40622aaac15..14fcb5b7072 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -61,20 +61,18 @@ Here's a full example of `/etc/gitlab-runner/config.toml`. Open it with your editor and edit as you see fit: ```toml -concurrent = 3 +concurrent = 10 check_interval = 0 [[runners]] name = "gitlab-aws-autoscaler" - url = "" - token = "" + url = "" + token = "" executor = "docker+machine" - limit = 4 [runners.docker] image = "alpine" privileged = true - disable_cache = false - volumes = ["/cache"] + disable_cache = true [runners.cache] Type = "s3" ServerAddress = "s3.amazonaws.com" @@ -114,36 +112,65 @@ Let's break it down to pieces. ### Global section ```toml -concurrent = 3 +concurrent = 10 check_interval = 0 ``` +In the global section, you can define the limit of the jobs that can be run +concurrently across all Runners (`concurrent`). This heavily depends on your +needs, like how many users your Runners will accommodate, how much time your +builds take, etc. You can start with something low, and increase its value going +forward. + +The `check_interval` setting defines in seconds how often the Runner should +check GitLab for new jobs. + +[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) +about all the options you can use. + ### `[[runners]]` ```toml [[runners]] name = "gitlab-aws-autoscaler" - url = "" - token = "" + url = "" + token = "" executor = "docker+machine" - limit = 4 ``` +From the `[[runners]]` section, the most important part is the `executor` which +must be set to `docker+machine`. All those settings are taken care of when you +register the Runner for the first time. + +[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) +about all the options you can use under `[[runners]]`. + ### `[runners.docker]` ```toml [runners.docker] image = "alpine" privileged = true - disable_cache = false - volumes = ["/cache"] + disable_cache = true ``` +In the `[runners.docker]` section you can define the default Docker image to +be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md). +Using `privileged = true`, all Runners will be able to run Docker in Docker +which is useful if you plan to build your own Docker images via GitLab CI/CD. + +Next, we use `disable_cache = true` to disable the Docker executor's inner +cache mechanism since we will use the distributed cache mode as described +below. + +[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section) +about all the options you can use under `[runners.docker]`. + ### `[runners.cache]` To speed up your jobs, GitLab Runner provides a cache mechanism where selected directories and/or files are saved and shared between subsequent jobs. -While not required for this setup, it is recommended to use the shared cache +While not required for this setup, it is recommended to use the distributed cache mechanism that GitLab Runner provides. Since new instances will be created on demand, it is essential to have a common place where cache is stored. @@ -162,7 +189,7 @@ In the following example, we use Amazon S3: Here's some more info to get you started: -- [The `[runners.cache]` section](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) +- [The `[runners.cache]` section reference](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) - [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) - [How cache works](../../ci/yaml/README.md#cache) @@ -202,6 +229,9 @@ TIP: **Tip:** Under `MachineOptions` you can add anything that the [AWS Docker Machine driver supports](https://docs.docker.com/machine/drivers/aws/#options). +[Read more](/runner/configuration/advanced-configuration.html#the-runners-machine-section) +about all the options you can use under `[runners.machine]`. + ## Cutting down costs with Amazon EC2 Spot instances As described by Amazon: -- cgit v1.2.1 From 48db4f2e9865fbc09ca8c96223d854b437374b49 Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 15 Nov 2017 09:54:55 +0100 Subject: Finish rest of the sections --- doc/articles/runner_autoscale_aws/index.md | 203 +++++++++++++++++++++-------- 1 file changed, 147 insertions(+), 56 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 14fcb5b7072..d7953e6be69 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -22,8 +22,37 @@ quite powerful autoscaling machines. ## Prerequisites +Your GitLab instance is going to need to talk to the Runners over the network, +so consider the sensitivity of your projects and communication between nodes +when moving forward. This has implications on the AWS security groups you'll +need, your DNS configuration, and other networking factors. + +For example, you can keep the EC2 resources segmented away from public traffic +in a different VPC. Your environment is likely different, so consider what works +best for your situation. + +### AWS security groups + +Docker Machine will attempt to use a +[default security group](https://docs.docker.com/machine/drivers/aws/#security-group) +with rules for port 2376, which is required for communication with the Docker +daemon. Instead of relying on Docker, you can create a security group with the +rules you need and provide that in the Runner options as we will see below. +This way, you can customize it to your liking ahead of time based on your +networking environment. + +### AWS credentials + +You'll need an AWS Access Key tied to a user with permission to scale (EC2) and +update the cache (via S3). Create a new user with policies for EC2 +(AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure, you can +disable console login for that user. Grab the security credentials as we'll use +them later during the Runner configuration. + +## Prepare the bastion instance + The first step is to install GitLab Runner in an EC2 instance that will serve -as the bastion to spawning new machines. This doesn't have to be a powerful +as the bastion that spawns new machines. This doesn't have to be a powerful machine since it will not run any jobs itself, a `t2.micro` instance will do. This machine will be a dedicated host since we need it always up and running, thus it will be the only standard cost. @@ -34,11 +63,19 @@ Runner support, for example either Ubuntu, Debian, CentOS or RHEL will work fine Install the prerequisites: -1. Log in your server +1. Log in to your server 1. [Install GitLab Runner from the official GitLab repository](https://docs.gitlab.com/runner/install/linux-repository.html) 1. [Install Docker](https://docs.docker.com/engine/installation/#server) 1. [Install Docker Machine](https://docs.docker.com/machine/install-machine/) +Now that the Runner is installed, it's time to register it. + +## Registering GitLab Runner + +TIP: **Tip:** +If you want every user in your instance to be able to use the autoscaled Runners, +register the Runner as a shared one. + Before configuring the GitLab Runner, you need to first register it, so that it connects with your GitLab instance: @@ -46,10 +83,6 @@ it connects with your GitLab instance: 1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux) 1. When asked the executor type, enter `docker+machine` -TIP: **Tip:** -If you want every user in your instance to be able to use the autoscaled Runners, -register the Runner as a shared one. - You can now move on to the most important part, configuring GitLab Runner. ## Configuring GitLab Runner to use the AWS machine driver @@ -69,6 +102,7 @@ check_interval = 0 url = "" token = "" executor = "docker+machine" + limit = 20 [runners.docker] image = "alpine" privileged = true @@ -85,11 +119,14 @@ check_interval = 0 IdleCount = 1 IdleTime = 1800 MaxBuilds = 100 - MachineDriver = "amazonec2" - MachineName = "gitlab-docker-machine-%s" - OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] + OffPeakPeriods = [ + "* * 0-9,18-23 * * mon-fri *", + "* * * * * sat,sun *" + ] OffPeakIdleCount = 0 OffPeakIdleTime = 1200 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" MachineOptions = [ "amazonec2-access-key=XXXX", "amazonec2-secret-key=XXXX", @@ -100,10 +137,6 @@ check_interval = 0 "amazonec2-tags=Name,gitlab-runner-autoscale", "amazonec2-security-group=docker-machine-scaler", "amazonec2-instance-type=m4.2xlarge", - "amazonec2-ssh-user=ubuntu", - "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", - "amazonec2-zone=a", - "amazonec2-root-size=32", ] ``` @@ -111,11 +144,6 @@ Let's break it down to pieces. ### Global section -```toml -concurrent = 10 -check_interval = 0 -``` - In the global section, you can define the limit of the jobs that can be run concurrently across all Runners (`concurrent`). This heavily depends on your needs, like how many users your Runners will accommodate, how much time your @@ -125,10 +153,25 @@ forward. The `check_interval` setting defines in seconds how often the Runner should check GitLab for new jobs. +Example: + +```toml +concurrent = 10 +check_interval = 0 +``` + [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) about all the options you can use. -### `[[runners]]` +### `[[runners]]` section + +From the `[[runners]]` section, the most important part is the `executor` which +must be set to `docker+machine`. Most of those settings are taken care of when +you register the Runner for the first time. + +`limit` defines how many jobs can be handled concurrently by this token. + +Example: ```toml [[runners]] @@ -136,23 +179,13 @@ about all the options you can use. url = "" token = "" executor = "docker+machine" + limit = 20 ``` -From the `[[runners]]` section, the most important part is the `executor` which -must be set to `docker+machine`. All those settings are taken care of when you -register the Runner for the first time. - [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) about all the options you can use under `[[runners]]`. -### `[runners.docker]` - -```toml - [runners.docker] - image = "alpine" - privileged = true - disable_cache = true -``` +### `[runners.docker]` section In the `[runners.docker]` section you can define the default Docker image to be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md). @@ -163,10 +196,19 @@ Next, we use `disable_cache = true` to disable the Docker executor's inner cache mechanism since we will use the distributed cache mode as described below. +Example: + +```toml + [runners.docker] + image = "alpine" + privileged = true + disable_cache = true +``` + [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section) about all the options you can use under `[runners.docker]`. -### `[runners.cache]` +### `[runners.cache]` section To speed up your jobs, GitLab Runner provides a cache mechanism where selected directories and/or files are saved and shared between subsequent jobs. @@ -193,21 +235,33 @@ Here's some more info to get you started: - [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) - [How cache works](../../ci/yaml/README.md#cache) -### `[runners.machine]` +### `[runners.machine]` section This is the most important part of the configuration and it's the one that -tells GitLab Runner how and when to spawn new Docker Machine instances. +tells GitLab Runner how and when to spawn new or remove old Docker Machine +instances. + +We will focus on the AWS machine options, for the rest of the settings read +about the: + +- [autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization +- [off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends + +Example: ```toml [runners.machine] IdleCount = 1 IdleTime = 1800 - MaxBuilds = 100 - MachineDriver = "amazonec2" - MachineName = "gitlab-docker-machine-%s" - OffPeakPeriods = ["* * 0-7,19-23 * * mon-fri *", "* * * * * sat,sun *"] + MaxBuilds = 10 + OffPeakPeriods = [ + "* * 0-9,18-23 * * mon-fri *", + "* * * * * sat,sun *" + ] OffPeakIdleCount = 0 OffPeakIdleTime = 1200 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" MachineOptions = [ "amazonec2-access-key=XXXX", "amazonec2-secret-key=XXXX", @@ -218,18 +272,44 @@ tells GitLab Runner how and when to spawn new Docker Machine instances. "amazonec2-tags=Name,gitlab-runner-autoscale", "amazonec2-security-group=docker-machine-scaler", "amazonec2-instance-type=m4.2xlarge", - "amazonec2-ssh-user=ubuntu", - "amazonec2-ssh-keypath=/etc/gitlab-runner/certs/gitlab-aws-autoscaler", - "amazonec2-zone=a", - "amazonec2-root-size=32", ] ``` +The Docker Machine driver is set to `amazonec2` and the machine name has a +standard prefix followed by `%s` (required) that is replaced by the ID of the +child Runner: `gitlab-docker-machine-%s`. + +Now, depending on your AWS infrastructure, there are many options you can set up +under `MachineOptions`. Let's see the most common ones: + +- `amazonec2-access-key=XXXX` - The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). +- `amazonec2-secret-key=XXXX` - The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). +- `amazonec2-region=eu-central-1` - The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. +- `amazonec2-vpc-id=vpc-xxxxx` - Your VPC ID to launch the instance in, read more in [Docker docs about the VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id). +- `amazonec2-subnet-id=subnet-xxxx` - AWS VPC subnet ID. +- `amazonec2-use-private-address=true` - Use the private IP address for docker-machine, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs. +- `amazonec2-tags=Name,gitlab-runner-autoscale` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. +- `amazonec2-security-group=docker-machine-scaler` - AWS VPC security group name, see [AWS security groups](#aws-security-groups). +- `amazonec2-instance-type=m4.2xlarge` - The instance type that the child Runners will run on. + TIP: **Tip:** Under `MachineOptions` you can add anything that the [AWS Docker Machine driver -supports](https://docs.docker.com/machine/drivers/aws/#options). +supports](https://docs.docker.com/machine/drivers/aws/#options). You are highly +encouraged to read Docker's docs as your infrastructure setup may warrant +different options to be applied. + +NOTE: **Note:** +The child instances will use by default Ubuntu 16.04 unless you choose a +different AMI ID by setting `amazonec2-ami`. + +NOTE: **Note:** +If you specify `amazonec2-private-address-only=true` as one of the machine +options, your EC2 instance won't get assigned a public IP. This is fine if your +VPC is configured correctly with an Internet Gateway (IGW) and routing is fine, +but it’s something to consider if you've got a more exotic configuration. Read +more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity). -[Read more](/runner/configuration/advanced-configuration.html#the-runners-machine-section) +[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section) about all the options you can use under `[runners.machine]`. ## Cutting down costs with Amazon EC2 Spot instances @@ -243,7 +323,9 @@ pricing, you can significantly reduce the cost of running your applications, grow your application’s compute capacity and throughput for the same budget, and enable new types of cloud computing applications. -In `/etc/gitlab-runner/config.toml` under the `MachineOptions` section: +In addition to the [`[runners.machine]`](#runners-machine-section) options +you picked above, in `/etc/gitlab-runner/config.toml` under the `MachineOptions` +section, add the following: ```toml MachineOptions = [ @@ -255,7 +337,8 @@ In `/etc/gitlab-runner/config.toml` under the `MachineOptions` section: With this configuration, Docker Machines are created on Spot instances with a maximum bid price of $0.03 per hour and the duration of the Spot instance is -capped at 60 minutes. +capped at 60 minutes. Be sure to check on the current pricing based on the +region you picked. To learn more about Amazon EC2 Spot instances, visit the following links: @@ -265,29 +348,37 @@ To learn more about Amazon EC2 Spot instances, visit the following links: ### Caveats of Spot instances -If the Spot price raises, the auto-scale Runner would fail to create new machines. +While Spot instances is a great way to use unused resources and minimize the +costs of your infrastructure, you must be aware of the implications. + +Running CI jobs on Spot instances may increase the failure rates because of the +Spot instances pricing model. If the price exceeds your bid, the existing Spot +instances will be immediately terminated and all your jobs on that host will fail. -This eventually eats 60 requests and then AWS won't accept any more. Then once -the spot price is acceptable, you are locked out for a bit because the call amount -limit is exceeded. +As a consequence, the auto-scale Runner would fail to create new machines while +it will continue to request new instances. This eventually will make 60 requests +and then AWS won't accept any more. Then once the Spot price is acceptable, you +are locked out for a bit because the call amount limit is exceeded. -You can use the following command in the bastion machine to see the Docker Machines -state: +If you encounter that case, you can use the following command in the bastion +machine to see the Docker Machines state: ```sh docker-machine ls -q --filter state=Error --format "{{.NAME}}" ``` NOTE: **Note:** -Follow [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) -for more information. +In [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) +there is a discussion to make GitLab Runner gracefully handle Spot price changes. ## Conclusion Using the autoscale feature of GitLab Runner can save you both time and money. -Using the spot instances that AWS provides can save you even more. +Using the Spot instances that AWS provides can save you even more, but you must +be aware of the implications. As long as your bid is high enough, there won't +be an issue. -You can read the following user cases from which this tutorial was influenced: +You can read the following use cases from which this tutorial was influenced: - [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html) - [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/) -- cgit v1.2.1 From 1982125081b2b029daabd5c965acb5e6e6851cdb Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Wed, 15 Nov 2017 11:16:38 +0100 Subject: Clarify usage of Runner limit and AWS tags --- doc/articles/runner_autoscale_aws/index.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index d7953e6be69..2f658635b1e 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -134,7 +134,7 @@ check_interval = 0 "amazonec2-vpc-id=vpc-xxxxx", "amazonec2-subnet-id=subnet-xxxxx", "amazonec2-use-private-address=true", - "amazonec2-tags=Name,gitlab-runner-autoscale", + "amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true", "amazonec2-security-group=docker-machine-scaler", "amazonec2-instance-type=m4.2xlarge", ] @@ -169,7 +169,9 @@ From the `[[runners]]` section, the most important part is the `executor` which must be set to `docker+machine`. Most of those settings are taken care of when you register the Runner for the first time. -`limit` defines how many jobs can be handled concurrently by this token. +`limit` sets the maximum number of machines (running and idle) that this Runner +will start. For more info check the [relationship between `limit`, `concurrent` +and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines). Example: @@ -269,7 +271,7 @@ Example: "amazonec2-vpc-id=vpc-xxxxx", "amazonec2-subnet-id=subnet-xxxxx", "amazonec2-use-private-address=true", - "amazonec2-tags=Name,gitlab-runner-autoscale", + "amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true", "amazonec2-security-group=docker-machine-scaler", "amazonec2-instance-type=m4.2xlarge", ] @@ -288,7 +290,7 @@ under `MachineOptions`. Let's see the most common ones: - `amazonec2-vpc-id=vpc-xxxxx` - Your VPC ID to launch the instance in, read more in [Docker docs about the VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id). - `amazonec2-subnet-id=subnet-xxxx` - AWS VPC subnet ID. - `amazonec2-use-private-address=true` - Use the private IP address for docker-machine, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs. -- `amazonec2-tags=Name,gitlab-runner-autoscale` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. +- `amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). - `amazonec2-security-group=docker-machine-scaler` - AWS VPC security group name, see [AWS security groups](#aws-security-groups). - `amazonec2-instance-type=m4.2xlarge` - The instance type that the child Runners will run on. -- cgit v1.2.1 From dfbfd3c7d7d4677ac99a7f8147a673911e8d4e98 Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Thu, 23 Nov 2017 14:32:16 +0100 Subject: Allow request namespace by ID or path --- doc/api/namespaces.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index d06b0d31c67..25cae5ce1f9 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -100,7 +100,7 @@ GET /namespaces/:id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | ID of the namespace | +| `id` | integer/string | yes | ID or path of the namespace | Example request: @@ -121,3 +121,23 @@ Example response: "members_count_with_descendants": 2 } ``` + +Example request: + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/namespaces/group1 +``` + +Example response: + +```json +{ + "id": 2, + "name": "group1", + "path": "group1", + "kind": "group", + "full_path": "group1", + "parent_id": "null", + "members_count_with_descendants": 2 +} +``` -- cgit v1.2.1 From bca153e538f8007d1dc8e906cdf85e4a92829b3b Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 24 Nov 2017 10:40:49 +0100 Subject: Address feedback for AWS Runner autoscale article --- doc/articles/index.md | 2 +- doc/articles/runner_autoscale_aws/index.md | 258 ++++++++++++++++------------- 2 files changed, 142 insertions(+), 118 deletions(-) (limited to 'doc') diff --git a/doc/articles/index.md b/doc/articles/index.md index bb941fc0023..862fe0868a6 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -26,7 +26,7 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Tutorial | 2017-11-10 | +| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 | | [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | diff --git a/doc/articles/runner_autoscale_aws/index.md b/doc/articles/runner_autoscale_aws/index.md index 2f658635b1e..9d4c4a57ce5 100644 --- a/doc/articles/runner_autoscale_aws/index.md +++ b/doc/articles/runner_autoscale_aws/index.md @@ -1,7 +1,12 @@ --- -last_updated: 2017-11-10 +last_updated: 2017-11-24 --- +> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** Admin guide || +> **Level:** intermediary || +> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || +> **Publication date:** 2017/11/24 + # Autoscaling GitLab Runner on AWS One of the biggest advantages of GitLab Runner is its ability to automatically @@ -16,38 +21,43 @@ In this tutorial, we'll explore how to properly configure a GitLab Runner in AWS that will serve as the bastion where it will spawn new Docker machines on demand. -In addition, we'll make use of [Amazon's EC2 Spot instances](https://aws.amazon.com/ec2/spot/) -which will greatly reduce the costs of the Runner instances while still using -quite powerful autoscaling machines. +In addition, we'll make use of [Amazon's EC2 Spot instances][spot] which will +greatly reduce the costs of the Runner instances while still using quite +powerful autoscaling machines. ## Prerequisites +NOTE: **Note:** +A familiarity with Amazon Web Services (AWS) is required as this is where most +of the configuration will take place. + Your GitLab instance is going to need to talk to the Runners over the network, -so consider the sensitivity of your projects and communication between nodes -when moving forward. This has implications on the AWS security groups you'll -need, your DNS configuration, and other networking factors. +and that is something you need think about when configuring any AWS security +groups or when setting up your DNS configuration. For example, you can keep the EC2 resources segmented away from public traffic -in a different VPC. Your environment is likely different, so consider what works -best for your situation. +in a different VPC to better strengthen your network security. Your environment +is likely different, so consider what works best for your situation. ### AWS security groups Docker Machine will attempt to use a [default security group](https://docs.docker.com/machine/drivers/aws/#security-group) -with rules for port 2376, which is required for communication with the Docker +with rules for port `2376`, which is required for communication with the Docker daemon. Instead of relying on Docker, you can create a security group with the -rules you need and provide that in the Runner options as we will see below. -This way, you can customize it to your liking ahead of time based on your -networking environment. +rules you need and provide that in the Runner options as we will +[see below](#the-runners-machine-section). This way, you can customize it to your +liking ahead of time based on your networking environment. ### AWS credentials -You'll need an AWS Access Key tied to a user with permission to scale (EC2) and -update the cache (via S3). Create a new user with policies for EC2 -(AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure, you can -disable console login for that user. Grab the security credentials as we'll use -them later during the Runner configuration. +You'll need an [AWS Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html) +tied to a user with permission to scale (EC2) and update the cache (via S3). +Create a new user with [policies](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-for-amazon-ec2.html) +for EC2 (AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure, +you can disable console login for that user. Keep the tab open or copy paste the +security credentials in an editor as we'll use them later during the +[Runner configuration](#the-runners-machine-section). ## Prepare the bastion instance @@ -70,11 +80,7 @@ Install the prerequisites: Now that the Runner is installed, it's time to register it. -## Registering GitLab Runner - -TIP: **Tip:** -If you want every user in your instance to be able to use the autoscaled Runners, -register the Runner as a shared one. +## Registering the GitLab Runner Before configuring the GitLab Runner, you need to first register it, so that it connects with your GitLab instance: @@ -83,75 +89,29 @@ it connects with your GitLab instance: 1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux) 1. When asked the executor type, enter `docker+machine` -You can now move on to the most important part, configuring GitLab Runner. +You can now move on to the most important part, configuring the GitLab Runner. -## Configuring GitLab Runner to use the AWS machine driver +TIP: **Tip:** +If you want every user in your instance to be able to use the autoscaled Runners, +register the Runner as a shared one. + +## Configuring the GitLab Runner Now that the Runner is registered, you need to edit its configuration file and add the required options for the AWS machine driver. -Here's a full example of `/etc/gitlab-runner/config.toml`. Open it with your -editor and edit as you see fit: +Let's first break it down to pieces. -```toml -concurrent = 10 -check_interval = 0 - -[[runners]] - name = "gitlab-aws-autoscaler" - url = "" - token = "" - executor = "docker+machine" - limit = 20 - [runners.docker] - image = "alpine" - privileged = true - disable_cache = true - [runners.cache] - Type = "s3" - ServerAddress = "s3.amazonaws.com" - AccessKey = "" - SecretKey = "" - BucketName = "" - BucketLocation = "us-east-1" - Shared = true - [runners.machine] - IdleCount = 1 - IdleTime = 1800 - MaxBuilds = 100 - OffPeakPeriods = [ - "* * 0-9,18-23 * * mon-fri *", - "* * * * * sat,sun *" - ] - OffPeakIdleCount = 0 - OffPeakIdleTime = 1200 - MachineDriver = "amazonec2" - MachineName = "gitlab-docker-machine-%s" - MachineOptions = [ - "amazonec2-access-key=XXXX", - "amazonec2-secret-key=XXXX", - "amazonec2-region=us-east-1", - "amazonec2-vpc-id=vpc-xxxxx", - "amazonec2-subnet-id=subnet-xxxxx", - "amazonec2-use-private-address=true", - "amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true", - "amazonec2-security-group=docker-machine-scaler", - "amazonec2-instance-type=m4.2xlarge", - ] -``` - -Let's break it down to pieces. - -### Global section +### The global section In the global section, you can define the limit of the jobs that can be run concurrently across all Runners (`concurrent`). This heavily depends on your needs, like how many users your Runners will accommodate, how much time your -builds take, etc. You can start with something low, and increase its value going -forward. +builds take, etc. You can start with something low like `10`, and increase or +decrease its value going forward. -The `check_interval` setting defines in seconds how often the Runner should -check GitLab for new jobs. +The `check_interval` option defines how often the Runner should check GitLab +for new jobs, in seconds. Example: @@ -163,14 +123,14 @@ check_interval = 0 [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) about all the options you can use. -### `[[runners]]` section +### The `runners` section From the `[[runners]]` section, the most important part is the `executor` which must be set to `docker+machine`. Most of those settings are taken care of when you register the Runner for the first time. `limit` sets the maximum number of machines (running and idle) that this Runner -will start. For more info check the [relationship between `limit`, `concurrent` +will spawn. For more info check the [relationship between `limit`, `concurrent` and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines). Example: @@ -187,16 +147,17 @@ Example: [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) about all the options you can use under `[[runners]]`. -### `[runners.docker]` section +### The `runners.docker` section In the `[runners.docker]` section you can define the default Docker image to be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md). -Using `privileged = true`, all Runners will be able to run Docker in Docker +By using `privileged = true`, all Runners will be able to run +[Docker in Docker](../../ci/docker/using_docker_build.md#use-docker-in-docker-executor) which is useful if you plan to build your own Docker images via GitLab CI/CD. Next, we use `disable_cache = true` to disable the Docker executor's inner cache mechanism since we will use the distributed cache mode as described -below. +in the following section. Example: @@ -210,13 +171,13 @@ Example: [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section) about all the options you can use under `[runners.docker]`. -### `[runners.cache]` section +### The `runners.cache` section To speed up your jobs, GitLab Runner provides a cache mechanism where selected directories and/or files are saved and shared between subsequent jobs. While not required for this setup, it is recommended to use the distributed cache mechanism that GitLab Runner provides. Since new instances will be created on -demand, it is essential to have a common place where cache is stored. +demand, it is essential to have a common place where the cache is stored. In the following example, we use Amazon S3: @@ -231,13 +192,13 @@ In the following example, we use Amazon S3: Shared = true ``` -Here's some more info to get you started: +Here's some more info to further explore the cache mechanism: -- [The `[runners.cache]` section reference](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) +- [Reference for `runners.cache`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) - [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) - [How cache works](../../ci/yaml/README.md#cache) -### `[runners.machine]` section +### The `runners.machine` section This is the most important part of the configuration and it's the one that tells GitLab Runner how and when to spawn new or remove old Docker Machine @@ -246,10 +207,10 @@ instances. We will focus on the AWS machine options, for the rest of the settings read about the: -- [autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization -- [off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends +- [Autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization +- [Off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends -Example: +Here's an example of the `runners.machine` section: ```toml [runners.machine] @@ -267,11 +228,11 @@ Example: MachineOptions = [ "amazonec2-access-key=XXXX", "amazonec2-secret-key=XXXX", - "amazonec2-region=us-east-1", + "amazonec2-region=us-central-1", "amazonec2-vpc-id=vpc-xxxxx", "amazonec2-subnet-id=subnet-xxxxx", "amazonec2-use-private-address=true", - "amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true", + "amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true", "amazonec2-security-group=docker-machine-scaler", "amazonec2-instance-type=m4.2xlarge", ] @@ -282,17 +243,19 @@ standard prefix followed by `%s` (required) that is replaced by the ID of the child Runner: `gitlab-docker-machine-%s`. Now, depending on your AWS infrastructure, there are many options you can set up -under `MachineOptions`. Let's see the most common ones: - -- `amazonec2-access-key=XXXX` - The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). -- `amazonec2-secret-key=XXXX` - The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). -- `amazonec2-region=eu-central-1` - The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. -- `amazonec2-vpc-id=vpc-xxxxx` - Your VPC ID to launch the instance in, read more in [Docker docs about the VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id). -- `amazonec2-subnet-id=subnet-xxxx` - AWS VPC subnet ID. -- `amazonec2-use-private-address=true` - Use the private IP address for docker-machine, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs. -- `amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). -- `amazonec2-security-group=docker-machine-scaler` - AWS VPC security group name, see [AWS security groups](#aws-security-groups). -- `amazonec2-instance-type=m4.2xlarge` - The instance type that the child Runners will run on. +under `MachineOptions`. Below you can see the most common ones. + +| Machine option | Description | +| -------------- | ----------- | +| `amazonec2-access-key=XXXX` | The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). | +| `amazonec2-secret-key=XXXX` | The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). | +| `amazonec2-region=eu-central-1` | The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. | +| `amazonec2-vpc-id=vpc-xxxxx` | Your [VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id) to launch the instance in. | +| `amazonec2-subnet-id=subnet-xxxx` | The AWS VPC subnet ID. | +| `amazonec2-use-private-address=true` | Use the private IP address of Docker Machines, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs.| +| `amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true` | AWS extra tag key-value pairs, useful to identify the instances on the AWS console. The "Name" tag is set to the machine name by default. We set the "runner-manager-name" to match the Runner name set in `[[runners]]`, so that we can filter all the EC2 instances created by a specific manager setup. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). | +| `amazonec2-security-group=docker-machine-scaler` | AWS VPC security group name, see [AWS security groups](#aws-security-groups). | +| `amazonec2-instance-type=m4.2xlarge` | The instance type that the child Runners will run on. | TIP: **Tip:** Under `MachineOptions` you can add anything that the [AWS Docker Machine driver @@ -306,17 +269,68 @@ different AMI ID by setting `amazonec2-ami`. NOTE: **Note:** If you specify `amazonec2-private-address-only=true` as one of the machine -options, your EC2 instance won't get assigned a public IP. This is fine if your +options, your EC2 instance won't get assigned a public IP. This is ok if your VPC is configured correctly with an Internet Gateway (IGW) and routing is fine, -but it’s something to consider if you've got a more exotic configuration. Read +but it’s something to consider if you've got a more complex configuration. Read more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity). [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section) about all the options you can use under `[runners.machine]`. +### Getting it all together + +Here's the full example of `/etc/gitlab-runner/config.toml`: + +```toml +concurrent = 10 +check_interval = 0 + +[[runners]] + name = "gitlab-aws-autoscaler" + url = "" + token = "" + executor = "docker+machine" + limit = 20 + [runners.docker] + image = "alpine" + privileged = true + disable_cache = true + [runners.cache] + Type = "s3" + ServerAddress = "s3.amazonaws.com" + AccessKey = "" + SecretKey = "" + BucketName = "" + BucketLocation = "us-east-1" + Shared = true + [runners.machine] + IdleCount = 1 + IdleTime = 1800 + MaxBuilds = 100 + OffPeakPeriods = [ + "* * 0-9,18-23 * * mon-fri *", + "* * * * * sat,sun *" + ] + OffPeakIdleCount = 0 + OffPeakIdleTime = 1200 + MachineDriver = "amazonec2" + MachineName = "gitlab-docker-machine-%s" + MachineOptions = [ + "amazonec2-access-key=XXXX", + "amazonec2-secret-key=XXXX", + "amazonec2-region=us-central-1", + "amazonec2-vpc-id=vpc-xxxxx", + "amazonec2-subnet-id=subnet-xxxxx", + "amazonec2-use-private-address=true", + "amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true", + "amazonec2-security-group=docker-machine-scaler", + "amazonec2-instance-type=m4.2xlarge", + ] +``` + ## Cutting down costs with Amazon EC2 Spot instances -As described by Amazon: +As [described by][spot] Amazon: > Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity. @@ -325,7 +339,7 @@ pricing, you can significantly reduce the cost of running your applications, grow your application’s compute capacity and throughput for the same budget, and enable new types of cloud computing applications. -In addition to the [`[runners.machine]`](#runners-machine-section) options +In addition to the [`runners.machine`](#the-runners-machine-section) options you picked above, in `/etc/gitlab-runner/config.toml` under the `MachineOptions` section, add the following: @@ -339,8 +353,8 @@ section, add the following: With this configuration, Docker Machines are created on Spot instances with a maximum bid price of $0.03 per hour and the duration of the Spot instance is -capped at 60 minutes. Be sure to check on the current pricing based on the -region you picked. +capped at 60 minutes. The `0.03` number mentioned above is just an example, so +be sure to check on the current pricing based on the region you picked. To learn more about Amazon EC2 Spot instances, visit the following links: @@ -370,17 +384,27 @@ docker-machine ls -q --filter state=Error --format "{{.NAME}}" ``` NOTE: **Note:** -In [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) -there is a discussion to make GitLab Runner gracefully handle Spot price changes. +There are some issues regarding making GitLab Runner gracefully handle Spot +price changes, and there are reports of `docker-machine` attempting to +continually remove a Docker Machine. GitLab has provided patches for both cases +in the upstream project. For more information, see issues +[#2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) and +[#2772](https://gitlab.com/gitlab-org/gitlab-runner/issues/2772). ## Conclusion +In this guide we learned how to install and configure a GitLab Runner in +autoscale mode on AWS. + Using the autoscale feature of GitLab Runner can save you both time and money. Using the Spot instances that AWS provides can save you even more, but you must -be aware of the implications. As long as your bid is high enough, there won't +be aware of the implications. As long as your bid is high enough, there shouldn't be an issue. -You can read the following use cases from which this tutorial was influenced: +You can read the following use cases from which this tutorial was (heavily) +influenced: - [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html) - [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/) + +[spot]: https://aws.amazon.com/ec2/spot/ -- cgit v1.2.1 From d6dd9d712ac24a095d0b0506731f9415b7c3b5f5 Mon Sep 17 00:00:00 2001 From: James Edwards-Jones Date: Fri, 24 Nov 2017 12:41:36 +0000 Subject: Fix ProtectedBranch access level validations Before an access_level was required in EE even when an it had been set for a user/group. --- doc/api/protected_branches.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 10faa95d7e8..81fe854060a 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -4,7 +4,7 @@ **Valid access levels** -The access levels are defined in the `ProtectedBranchAccess::ALLOWED_ACCESS_LEVELS` constant. Currently, these levels are recognized: +The access levels are defined in the `ProtectedRefAccess::ALLOWED_ACCESS_LEVELS` constant. Currently, these levels are recognized: ``` 0 => No access 30 => Developer access -- cgit v1.2.1 From 308f2ecd1d9355959298828c83fdabf5208c59a1 Mon Sep 17 00:00:00 2001 From: Christiaan Van den Poel Date: Sat, 25 Nov 2017 00:03:48 +0100 Subject: removed 'only once' from the docs --- doc/user/project/pipelines/schedules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index eac706be3a7..2101e3b1d58 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -5,7 +5,7 @@ - In 9.2, the feature was [renamed to Pipeline Schedule][ce-10853]. - Cron notation is parsed by [Rufus-Scheduler](https://github.com/jmettraux/rufus-scheduler). -Pipeline schedules can be used to run pipelines only once, or for example every +Pipeline schedules can be used to run a pipeline at specific intervals, for example every month on the 22nd for a certain branch. ## Using Pipeline schedules -- cgit v1.2.1 From 73397a3e6c87065de2e0031c6ecc997b31b9a804 Mon Sep 17 00:00:00 2001 From: David Muckle Date: Sat, 25 Nov 2017 02:18:27 +0000 Subject: Add in the Oxford comma --- doc/user/project/clusters/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index cf0c7c109a8..e2924c66e70 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -82,7 +82,7 @@ added directly to your configured cluster. Those applications are needed for | Application | GitLab version | Description | | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | ## Enabling or disabling the Cluster integration -- cgit v1.2.1 From 623eb68195d51ea50e09970771442d992ff19a4a Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Thu, 16 Nov 2017 17:12:33 +0100 Subject: Add new API endpoint - list jobs of a specified runner --- doc/api/runners.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) (limited to 'doc') diff --git a/doc/api/runners.md b/doc/api/runners.md index 6304a496f94..171857ef49e 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -215,6 +215,57 @@ DELETE /runners/:id curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/6" ``` +## List runner's running jobs + +List running jobs assigned to the specified Runner. + +``` +GET /runners/:id/jobs +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a runner | + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/1/jobs" +``` + +Example response: + +```json +[ + { + "id": 2, + "status": "running", + "stage": "test", + "name": "test", + "ref": "master", + "tag": false, + "coverage": null, + "created_at": "2017-11-16T08:50:29.000Z", + "started_at": "2017-11-16T08:51:29.000Z", + "finished_at": "2017-11-16T08:53:29.000Z", + "duration": 120, + "user": null, + "commit": null, + "runner": { + "id": 1, + "description": "My runner1", + "active": true, + "is_shared": true, + "name": null + }, + "pipeline": { + "id": 2, + "sha": "97de212e80737a608d939f648d959671fb0a0142", + "ref": "master", + "status": "pending" + } + } +] +``` + ## List project's runners List all runners (specific and shared) available in the project. Shared runners -- cgit v1.2.1 From 8d3e80692cbeea06dd28a052554f0c262004e18d Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Thu, 16 Nov 2017 19:44:14 +0100 Subject: Add information about project --- doc/api/runners.md | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'doc') diff --git a/doc/api/runners.md b/doc/api/runners.md index 171857ef49e..f9a9e2bee5a 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -261,6 +261,15 @@ Example response: "sha": "97de212e80737a608d939f648d959671fb0a0142", "ref": "master", "status": "pending" + }, + "project": { + "id": 1, + "description": null, + "name": "project1", + "name_with_namespace": "John Doe2 / project1", + "path": "project1", + "path_with_namespace": "namespace1/project1", + "created_at": "2017-11-16T18:38:46.620Z" } } ] -- cgit v1.2.1 From 9d27ce1630004e37cd0615830857a3135dc8d5fe Mon Sep 17 00:00:00 2001 From: Tomasz Maczukin Date: Tue, 21 Nov 2017 20:16:14 +0100 Subject: Update documentation --- doc/api/runners.md | 47 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) (limited to 'doc') diff --git a/doc/api/runners.md b/doc/api/runners.md index f9a9e2bee5a..015b09a745e 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -215,9 +215,9 @@ DELETE /runners/:id curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/6" ``` -## List runner's running jobs +## List runner's jobs -List running jobs assigned to the specified Runner. +List jobs that are being processed or were processed by specified Runner. ``` GET /runners/:id/jobs @@ -226,9 +226,10 @@ GET /runners/:id/jobs | Attribute | Type | Required | Description | |-----------|---------|----------|---------------------| | `id` | integer | yes | The ID of a runner | +| `status` | string | no | Status of the job; one of: `running`, `success`, `failed`, `canceled` | ``` -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/1/jobs" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/1/jobs?status=running" ``` Example response: @@ -247,20 +248,44 @@ Example response: "started_at": "2017-11-16T08:51:29.000Z", "finished_at": "2017-11-16T08:53:29.000Z", "duration": 120, - "user": null, - "commit": null, - "runner": { + "user": { "id": 1, - "description": "My runner1", - "active": true, - "is_shared": true, - "name": null + "name": "John Doe2", + "username": "user2", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", + "web_url": "http://localhost/user2", + "created_at": "2017-11-16T18:38:46.000Z", + "bio": null, + "location": null, + "skype": "", + "linkedin": "", + "twitter": "", + "website_url": "", + "organization": null + }, + "commit": { + "id": "97de212e80737a608d939f648d959671fb0a0142", + "short_id": "97de212e", + "title": "Update configuration\r", + "created_at": "2017-11-16T08:50:28.000Z", + "parent_ids": [ + "1b12f15a11fc6e62177bef08f47bc7b5ce50b141", + "498214de67004b1da3d820901307bed2a68a8ef6" + ], + "message": "See merge request !123", + "author_name": "John Doe2", + "author_email": "user2@example.org", + "authored_date": "2017-11-16T08:50:27.000Z", + "committer_name": "John Doe2", + "committer_email": "user2@example.org", + "committed_date": "2017-11-16T08:50:27.000Z" }, "pipeline": { "id": 2, "sha": "97de212e80737a608d939f648d959671fb0a0142", "ref": "master", - "status": "pending" + "status": "running" }, "project": { "id": 1, -- cgit v1.2.1 From 9cb38f0433930f85964ab3c3f07d677676fa265b Mon Sep 17 00:00:00 2001 From: Sean McGivern Date: Fri, 24 Nov 2017 09:55:23 +0000 Subject: Fix instructions for creating project templates Sidekiq has to be running too. --- doc/development/rake_tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 4773b6773e8..ceff57276d2 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -163,7 +163,7 @@ Starting a project from a template needs this project to be exported. On a up to date master branch with run: ``` -gdk run db +gdk run # In a new terminal window bundle exec rake gitlab:update_project_templates git checkout -b update-project-templates -- cgit v1.2.1 From 4ebbfe5d3e0dbe06346ee0c64a8f62ec11f9b6e8 Mon Sep 17 00:00:00 2001 From: Sean McGivern Date: Tue, 21 Nov 2017 16:58:08 +0000 Subject: Remove serialised diff and commit columns The st_commits and st_diffs columns on merge_request_diffs historically held the YAML-serialised data for a merge request diff, in a variety of formats. Since 9.5, these have been migrated in the background to two new tables: merge_request_diff_commits and merge_request_diff_files. That has the advantage that we can actually query the data (for instance, to find out how many commits we've stored), and that it can't be in a variety of formats, but must match the new schema. This is the final step of that journey, where we drop those columns and remove all references to them. This is a breaking change to the importer, because we can no longer import diffs created in the old format, and we cannot guarantee the export will be in the new format unless it was generated after this commit. --- doc/user/project/integrations/webhooks.md | 2 -- doc/user/project/settings/import_export.md | 3 ++- 2 files changed, 2 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 5896f8f72a0..eafdd28071d 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -731,8 +731,6 @@ X-Gitlab-Event: Merge Request Hook "title": "MS-Viewport", "created_at": "2013-12-03T17:23:34Z", "updated_at": "2013-12-03T17:23:34Z", - "st_commits": null, - "st_diffs": null, "milestone_id": null, "state": "opened", "merge_status": "unchecked", diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 23b1c61cd16..1b8a84c9599 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -30,7 +30,8 @@ with all their related data and be moved into a new GitLab instance. | GitLab version | Import/Export version | | ---------------- | --------------------- | -| 10.0 to current | 0.2.0 | +| 10.3 to current | 0.2.1 | +| 10.0 | 0.2.0 | | 9.4.0 | 0.1.8 | | 9.2.0 | 0.1.7 | | 8.17.0 | 0.1.6 | -- cgit v1.2.1 From 769505eb767eb4ae1ad76fc51d6abb1062e2a9b1 Mon Sep 17 00:00:00 2001 From: Abubakar Ango Date: Tue, 28 Nov 2017 17:16:04 +0000 Subject: Update google.md to reflect changes in the new Google APIs Library --- doc/integration/google.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/integration/google.md b/doc/integration/google.md index 727ca13ebcf..07a700f7b64 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -35,7 +35,7 @@ In Google's side: 1. You should now be able to see a Client ID and Client secret. Note them down or keep this page open as you will need them later. -1. From the **Dashboard** select **ENABLE APIS AND SERVICES > Google Cloud APIs > Container Engine API > Enable** +1. From the **Dashboard** select **ENABLE APIS AND SERVICES > Compute > Google Container Engine API > Enable** On your GitLab server: -- cgit v1.2.1 From a35d6d3c479fa2259e6d32d207be99d980a711c6 Mon Sep 17 00:00:00 2001 From: Greg Stark Date: Tue, 28 Nov 2017 18:14:05 +0000 Subject: I am certainly weary of debugging sidekiq but I don't think that's what was meant --- doc/administration/troubleshooting/sidekiq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/administration/troubleshooting/sidekiq.md b/doc/administration/troubleshooting/sidekiq.md index b71f8fabbc8..9d157720ad2 100644 --- a/doc/administration/troubleshooting/sidekiq.md +++ b/doc/administration/troubleshooting/sidekiq.md @@ -11,7 +11,7 @@ troubleshooting steps that will help you diagnose the bottleneck. debug steps with GitLab Support so the backtraces can be analyzed by our team. It may reveal a bug or necessary improvement in GitLab. -> **Note:** In any of the backtraces, be weary of suspecting cases where every +> **Note:** In any of the backtraces, be wary of suspecting cases where every thread appears to be waiting in the database, Redis, or waiting to acquire a mutex. This **may** mean there's contention in the database, for example, but look for one thread that is different than the rest. This other thread -- cgit v1.2.1 From ac87bd0c113e5265d7913e13c005850a4f6bf4c3 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Mon, 20 Nov 2017 10:40:01 -0600 Subject: Add checkboxes to automatically run AutoDevops pipeline Fix https://gitlab.com/gitlab-org/gitlab-ce/issues/38962 --- doc/topics/autodevops/img/auto_devops_settings.png | Bin 0 -> 67845 bytes doc/topics/autodevops/index.md | 6 +++++- 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 doc/topics/autodevops/img/auto_devops_settings.png (limited to 'doc') diff --git a/doc/topics/autodevops/img/auto_devops_settings.png b/doc/topics/autodevops/img/auto_devops_settings.png new file mode 100644 index 00000000000..b572cc5b855 Binary files /dev/null and b/doc/topics/autodevops/img/auto_devops_settings.png differ diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 28308fc905c..914217772b8 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -121,7 +121,7 @@ Google Cloud. ## Enabling Auto DevOps -NOTE: **Note:** +**Note:** If you haven't done already, read the [prerequisites](#prerequisites) to make full use of Auto DevOps. If this is your fist time, we recommend you follow the [quick start guide](#quick-start). @@ -129,10 +129,14 @@ full use of Auto DevOps. If this is your fist time, we recommend you follow the 1. Go to your project's **Settings > CI/CD > General pipelines settings** and find the Auto DevOps section 1. Select "Enable Auto DevOps" +1. After selecting an option to enable Auto DevOps, a checkbox will appear below + so you can immediately run a pipeline on the default branch 1. Optionally, but recommended, add in the [base domain](#auto-devops-base-domain) that will be used by Kubernetes to deploy your application 1. Hit **Save changes** for the changes to take effect +![Project AutoDevops settings section](img/auto_devops_settings.png) + Now that it's enabled, there are a few more steps depending on whether your project has a `.gitlab-ci.yml` or not: -- cgit v1.2.1 From 9295c827b85ef76f40dd23b2e00909df94eea093 Mon Sep 17 00:00:00 2001 From: digitalMoksha Date: Wed, 29 Nov 2017 10:44:33 +0100 Subject: fix link that was linking to `html` instead of `md` (to be consistent) --- doc/development/what_requires_downtime.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index c4830322fa8..05e0a64af18 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -37,7 +37,7 @@ when using the migration helper method `Gitlab::Database::MigrationHelpers#add_column_with_default`. This method works similar to `add_column` except it updates existing rows in batches without blocking access to the table being modified. See ["Adding Columns With Default -Values"](migration_style_guide.html#adding-columns-with-default-values) for more +Values"](migration_style_guide.md#adding-columns-with-default-values) for more information on how to use this method. ## Dropping Columns -- cgit v1.2.1 From 57d9121127eb9745ea196bbd8596ffa03afdee68 Mon Sep 17 00:00:00 2001 From: haseeb Date: Wed, 29 Nov 2017 16:22:22 +0000 Subject: support ordering of project notes in notes api --- doc/api/notes.md | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) (limited to 'doc') diff --git a/doc/api/notes.md b/doc/api/notes.md index e627369e17b..d02ef84d0bd 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -10,12 +10,15 @@ Gets a list of all notes for a single issue. ``` GET /projects/:id/issues/:issue_iid/notes +GET /projects/:id/issues/:issue_iid/notes?sort=asc&order_by=updated_at ``` -Parameters: - -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `issue_iid` (required) - The IID of an issue +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `issue_iid` | integer | yes | The IID of an issue +| `sort` | string | no | Return issue notes sorted in `asc` or `desc` order. Default is `desc` +| `order_by` | string | no | Return issue notes ordered by `created_at` or `updated_at` fields. Default is `created_at` ```json [ @@ -133,12 +136,15 @@ Gets a list of all notes for a single snippet. Snippet notes are comments users ``` GET /projects/:id/snippets/:snippet_id/notes +GET /projects/:id/snippets/:snippet_id/notes?sort=asc&order_by=updated_at ``` -Parameters: - -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `snippet_id` (required) - The ID of a project snippet +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `snippet_id` | integer | yes | The ID of a project snippet +| `sort` | string | no | Return snippet notes sorted in `asc` or `desc` order. Default is `desc` +| `order_by` | string | no | Return snippet notes ordered by `created_at` or `updated_at` fields. Default is `created_at` ### Get single snippet note @@ -231,12 +237,15 @@ Gets a list of all notes for a single merge request. ``` GET /projects/:id/merge_requests/:merge_request_iid/notes +GET /projects/:id/merge_requests/:merge_request_iid/notes?sort=asc&order_by=updated_at ``` -Parameters: - -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `merge_request_iid` (required) - The IID of a project merge request +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `merge_request_iid` | integer | yes | The IID of a project merge request +| `sort` | string | no | Return merge request notes sorted in `asc` or `desc` order. Default is `desc` +| `order_by` | string | no | Return merge request notes ordered by `created_at` or `updated_at` fields. Default is `created_at` ### Get single merge request note -- cgit v1.2.1 From aaae74c6c3099a7d590f8bd968ba06d371df533c Mon Sep 17 00:00:00 2001 From: Lyle Kozloff Date: Fri, 1 Dec 2017 15:27:56 +0000 Subject: Updating trouble shooting links --- doc/administration/raketasks/maintenance.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index 5b6ee354887..136192191f9 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -58,7 +58,9 @@ Runs the following rake tasks: It will check that each component was setup according to the installation guide and suggest fixes for issues found. -You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). +You may also have a look at our Trouble Shooting Guides: +- [Trouble Shooting Guide (GitLab)](http://docs.gitlab.com/ee/README.html#troubleshooting) +- [Trouble Shooting Guide (Omnibus Gitlab)](http://docs.gitlab.com/omnibus/README.html#troubleshooting) **Omnibus Installation** -- cgit v1.2.1 From 82da87f5fbc6524eca731d8de467eb93c6d86cce Mon Sep 17 00:00:00 2001 From: Achilleas Pipinellis Date: Fri, 1 Dec 2017 18:20:44 +0100 Subject: Add version mermaid support was introduced --- doc/user/markdown.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) (limited to 'doc') diff --git a/doc/user/markdown.md b/doc/user/markdown.md index fb61e360996..a671c92640a 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -41,7 +41,7 @@ Line-breaks, or softreturns, are rendered if you end a line with two or more spa Sugar is sweet -Roses are red +Roses are red Violets are blue Sugar is sweet @@ -370,14 +370,17 @@ This also works for the asciidoctor `:stem: latexmath`. For details see the [asc ### Mermaid +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15107) in +GitLab 10.3. + > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#mermaid -It is possible to generate diagrams and flowcharts from text using [Mermaid][mermaid]. +It is possible to generate diagrams and flowcharts from text using [Mermaid][mermaid]. -In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. +In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. -Example: +Example: ```mermaid graph TD; @@ -385,7 +388,7 @@ Example: A-->C; B-->D; C-->D; - ``` + ``` Becomes: @@ -395,7 +398,7 @@ graph TD; A-->C; B-->D; C-->D; -``` +``` For details see the [Mermaid official page][mermaid]. @@ -697,7 +700,7 @@ This line is separated from the one above by two newlines, so it will be a *sepa This line is also a separate paragraph, but... This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. -This line is also a separate paragraph, and... +This line is also a separate paragraph, and... This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) spaces. @@ -710,7 +713,7 @@ This line is separated from the one above by two newlines, so it will be a *sepa This line is also a separate paragraph, but... This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. -This line is also a separate paragraph, and... +This line is also a separate paragraph, and... This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) spaces. -- cgit v1.2.1 From 60a36b7dd89235a0f8347ea1359c82b60d5e1cef Mon Sep 17 00:00:00 2001 From: Koichiro Mikami Date: Fri, 1 Dec 2017 17:57:10 +0000 Subject: Documentation bug fixes: Added procedure to disable built-in Issues. --- .../integrations/img/issue_configuration.png | Bin 0 -> 20288 bytes doc/user/project/integrations/redmine.md | 29 ++++++++++++--------- 2 files changed, 16 insertions(+), 13 deletions(-) create mode 100644 doc/user/project/integrations/img/issue_configuration.png (limited to 'doc') diff --git a/doc/user/project/integrations/img/issue_configuration.png b/doc/user/project/integrations/img/issue_configuration.png new file mode 100644 index 00000000000..2049d60fdd2 Binary files /dev/null and b/doc/user/project/integrations/img/issue_configuration.png differ diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index cf92465da53..f530b6cb649 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -1,26 +1,29 @@ # Redmine Service -To enable the Redmine integration in a project, navigate to the +1. To enable the Redmine integration in a project, navigate to the [Integrations page](project_services.md#accessing-the-project-services), click the **Redmine** service, and fill in the required details on the page as described in the table below. -| Field | Description | -| ----- | ----------- | -| `description` | A name for the issue tracker (to differentiate between instances, for example) | -| `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | -| `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | -| `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project | + | Field | Description | + | ----- | ----------- | + | `description` | A name for the issue tracker (to differentiate between instances, for example) | + | `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | + | `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | + | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project | -Once you have configured and enabled Redmine: + Once you have configured and enabled Redmine: + - the **Issues** link on the GitLab project pages takes you to the appropriate + Redmine issue index + - clicking **New issue** on the project dashboard creates a new Redmine issue -- the **Issues** link on the GitLab project pages takes you to the appropriate - Redmine issue index -- clicking **New issue** on the project dashboard creates a new Redmine issue + As an example, below is a configuration for a project named gitlab-ci. -As an example, below is a configuration for a project named gitlab-ci. + ![Redmine configuration](img/redmine_configuration.png) -![Redmine configuration](img/redmine_configuration.png) +2. To disable the internal issue tracking system in a project, navigate to the General page, expand [Permissions](../settings/index.md#sharing-and-permissions), and slide the Issues switch invalid. + + ![Issue configuration](img/issue_configuration.png) ## Referencing issues in Redmine -- cgit v1.2.1 From 0f99e9a50b3faf1da47d37a0642d22e636498a80 Mon Sep 17 00:00:00 2001 From: Takuya Noguchi Date: Sun, 3 Dec 2017 13:03:45 +0900 Subject: Fix typo in docs about Elasticsearch --- doc/development/changelog.md | 2 +- doc/university/glossary/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/development/changelog.md b/doc/development/changelog.md index f869938fe11..48cffc0dd18 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -80,7 +80,7 @@ changes. The first example focuses on _how_ we fixed something, not on _what_ it fixes. The rewritten version clearly describes the _end benefit_ to the user (fewer 500 -errors), and _when_ (searching commits with ElasticSearch). +errors), and _when_ (searching commits with Elasticsearch). Use your best judgement and try to put yourself in the mindset of someone reading the compiled changelog. Does this entry add value? Does it offer context diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 076fbf6f710..fbe7353c6ca 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -175,7 +175,7 @@ A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of ### EC2 Instance -### ElasticSearch +### Elasticsearch Elasticsearch is a flexible, scalable and powerful search service. When [enabled](https://gitlab.com/help/integration/elasticsearch.md), it helps keep GitLab's search fast when dealing with a huge amount of data. -- cgit v1.2.1 From 6cf76d652d0ae0b77b1250a81aa24e3ce164ccbc Mon Sep 17 00:00:00 2001 From: Annabel Dunstone Gray Date: Mon, 4 Dec 2017 11:00:52 +0000 Subject: Add underline hover state to all links --- doc/development/doc_styleguide.md | 6 ------ 1 file changed, 6 deletions(-) (limited to 'doc') diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index aaa7032cadb..db13e0e6249 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -170,12 +170,6 @@ You can combine one or more of the following: = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' ``` -1. **Underlining a link.** - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), class: 'underlined-link' - ``` - 1. **Using links inline of some text.** ```haml -- cgit v1.2.1 From b8a393192529015bc2fa9d04c2782cf96e7ec896 Mon Sep 17 00:00:00 2001 From: Markus Koller Date: Wed, 22 Nov 2017 14:48:38 +0100 Subject: Add custom brand text on new project pages --- doc/README.md | 1 + doc/customization/new_project_page.md | 20 ++++++++++++++++++++ .../new_project_page/appearance_settings.png | Bin 0 -> 71178 bytes .../new_project_page/custom_new_project_page.png | Bin 0 -> 164962 bytes .../new_project_page/default_new_project_page.png | Bin 0 -> 146906 bytes 5 files changed, 21 insertions(+) create mode 100644 doc/customization/new_project_page.md create mode 100644 doc/customization/new_project_page/appearance_settings.png create mode 100644 doc/customization/new_project_page/custom_new_project_page.png create mode 100644 doc/customization/new_project_page/default_new_project_page.png (limited to 'doc') diff --git a/doc/README.md b/doc/README.md index d4119d35162..95cb9683a15 100644 --- a/doc/README.md +++ b/doc/README.md @@ -189,6 +189,7 @@ have access to GitLab administration tools and settings. - [Issue closing pattern](administration/issue_closing_pattern.md): Customize how to close an issue from commit messages. - [Libravatar](customization/libravatar.md): Use Libravatar instead of Gravatar for user avatars. - [Welcome message](customization/welcome_message.md): Add a custom welcome message to the sign-in page. +- [New project page](customization/new_project_page.md): Customize the new project page. ### Admin tools diff --git a/doc/customization/new_project_page.md b/doc/customization/new_project_page.md new file mode 100644 index 00000000000..148bf9512c6 --- /dev/null +++ b/doc/customization/new_project_page.md @@ -0,0 +1,20 @@ +# Customizing the new project page + +It is possible to add a markdown-formatted message to your GitLab +new project page. + +By default, the new project page shows a sidebar with general information: + +![](new_project_page/default_new_project_page.png) + +## Changing the appearance of the new project page + +Navigate to the **Admin** area and go to the **Appearance** page. + +Fill in your project guidelines: + +![](new_project_page/appearance_settings.png) + +After saving the page, your new project page will show the guidelines in the sidebar, below the general information: + +![](new_project_page/custom_new_project_page.png) diff --git a/doc/customization/new_project_page/appearance_settings.png b/doc/customization/new_project_page/appearance_settings.png new file mode 100644 index 00000000000..08eea684e14 Binary files /dev/null and b/doc/customization/new_project_page/appearance_settings.png differ diff --git a/doc/customization/new_project_page/custom_new_project_page.png b/doc/customization/new_project_page/custom_new_project_page.png new file mode 100644 index 00000000000..662c715f193 Binary files /dev/null and b/doc/customization/new_project_page/custom_new_project_page.png differ diff --git a/doc/customization/new_project_page/default_new_project_page.png b/doc/customization/new_project_page/default_new_project_page.png new file mode 100644 index 00000000000..4a0bcf09903 Binary files /dev/null and b/doc/customization/new_project_page/default_new_project_page.png differ -- cgit v1.2.1 From 29e058ba953dab54c32b4662c13da0b88c351ffa Mon Sep 17 00:00:00 2001 From: Victor Wu Date: Mon, 4 Dec 2017 19:55:16 +0000 Subject: Update slash_commands.md --- doc/integration/slash_commands.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md index aa52b5415cf..36a8844e953 100644 --- a/doc/integration/slash_commands.md +++ b/doc/integration/slash_commands.md @@ -17,6 +17,9 @@ Taking the trigger term as `project-name`, the commands are: | `/project-name issue search ` | Shows up to 5 issues matching `` | | `/project-name deploy to ` | Deploy from the `` environment to the `` environment | +Note that if you are using the [GitLab Slack application](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html) for +your GitLab.com projects, you need to [add the `gitlab` keyword at the beginning of the command](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html#usage). + ## Issue commands It is possible to create new issue, display issue details and search up to 5 issues. -- cgit v1.2.1 From 8cce70730c2fb9c705e1f1177f6d1effc665b3c7 Mon Sep 17 00:00:00 2001 From: Jan Provaznik Date: Thu, 24 Aug 2017 08:20:36 +0200 Subject: Create merge request from email * new merge request can be created by sending an email to the specific email address (similar to creating issues by email) * for the first iteration, source branch must be specified in the mail subject, other merge request parameters can not be set yet * user should enable "Receive notifications about your own activity" in user settings to receive a notification about created merge request Part of #32878 --- doc/user/project/merge_requests/index.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 4b2e042251b..d76ea259301 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -27,7 +27,7 @@ With GitLab merge requests, you can: - [Resolve merge conflicts from the UI](#resolve-conflicts) - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch - +- [Create new merge requests by email](#create_by_email) With **[GitLab Enterprise Edition][ee]**, you can also: @@ -132,6 +132,14 @@ those conflicts in the GitLab UI. [Learn more about resolving merge conflicts in the UI.](resolve_conflicts.md) +## Create new merge requests by email + +You can create a new merge request by sending an email to a user-specific email +address. The address can be obtained on the merge requests page by clicking on +a **Email a new merge request to this project** button. The subject will be +used as the source branch name for the new merge request and the target branch +will be the default branch for the project. + ## Revert changes GitLab implements Git's powerful feature to revert any commit with introducing -- cgit v1.2.1 From 0b15570e497d3c5c515be59a43b686087b985f5c Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 28 Nov 2017 17:08:30 +0100 Subject: Add ApplicationWorker and make every worker include it --- doc/development/sidekiq_style_guide.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index 1e9fdbc65e2..a120f3e4771 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -18,8 +18,7 @@ include the `DedicatedSidekiqQueue` concern as follows: ```ruby class ProcessSomethingWorker - include Sidekiq::Worker - include DedicatedSidekiqQueue + include ApplicationWorker end ``` -- cgit v1.2.1 From a5c3f1c8ff7da20183b172b2b0693a6010c9e86d Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Thu, 30 Nov 2017 16:28:09 +0100 Subject: Update docs --- doc/development/sidekiq_style_guide.md | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) (limited to 'doc') diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index a120f3e4771..085fb8f902c 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -3,6 +3,12 @@ This document outlines various guidelines that should be followed when adding or modifying Sidekiq workers. +## ApplicationWorker + +All workers should include `ApplicationWorker` instead of `Sidekiq::Worker`, +which adds some convenience methods and automatically sets the queue based on +the worker's name. + ## Default Queue Use of the "default" queue is not allowed. Every worker should use a queue that @@ -13,18 +19,10 @@ A list of all available queues can be found in `config/sidekiq_queues.yml`. ## Dedicated Queues -Most workers should use their own queue. To ease this process a worker can -include the `DedicatedSidekiqQueue` concern as follows: - -```ruby -class ProcessSomethingWorker - include ApplicationWorker -end -``` - -This will set the queue name based on the class' name, minus the `Worker` -suffix. In the above example this would lead to the queue being -`process_something`. +Most workers should use their own queue, which is automatically set based on the +worker class name. For a worker named `ProcessSomethingWorker`, the queue name +would be `process_something`. If you're not sure what a worker's queue name is, +you can find it using `SomeWorker.queue`. In some cases multiple workers do use the same queue. For example, the various workers for updating CI pipelines all use the `pipeline` queue. Adding workers @@ -38,7 +36,7 @@ tests should be placed in `spec/workers`. ## Removing or renaming queues -Try to avoid renaming or removing queues in minor and patch releases. -During online update instance can have pending jobs and removing the queue can -lead to those jobs being stuck forever. If you can't write migration for those -Sidekiq jobs, please consider doing rename or remove queue in major release only. \ No newline at end of file +Try to avoid renaming or removing queues in minor and patch releases. +During online update instance can have pending jobs and removing the queue can +lead to those jobs being stuck forever. If you can't write migration for those +Sidekiq jobs, please consider doing rename or remove queue in major release only. -- cgit v1.2.1 From 1e6ca3c41ead23c5e433460c8c807ea73d9ec0ef Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Wed, 29 Nov 2017 16:30:17 +0100 Subject: Consistently schedule Sidekiq jobs --- doc/development/background_migrations.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 5452b0e7a2f..fd2b9d0e908 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -68,10 +68,10 @@ BackgroundMigrationWorker.perform_async('BackgroundMigrationClassName', [arg1, a ``` Usually it's better to enqueue jobs in bulk, for this you can use -`BackgroundMigrationWorker.perform_bulk`: +`BackgroundMigrationWorker.bulk_perform_async`: ```ruby -BackgroundMigrationWorker.perform_bulk( +BackgroundMigrationWorker.bulk_perform_async( [['BackgroundMigrationClassName', [1]], ['BackgroundMigrationClassName', [2]]] ) @@ -85,13 +85,13 @@ updates. Removals in turn can be handled by simply defining foreign keys with cascading deletes. If you would like to schedule jobs in bulk with a delay, you can use -`BackgroundMigrationWorker.perform_bulk_in`: +`BackgroundMigrationWorker.bulk_perform_in`: ```ruby jobs = [['BackgroundMigrationClassName', [1]], ['BackgroundMigrationClassName', [2]]] -BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs) +BackgroundMigrationWorker.bulk_perform_in(5.minutes, jobs) ``` ## Cleaning Up @@ -201,7 +201,7 @@ class ScheduleExtractServicesUrl < ActiveRecord::Migration ['ExtractServicesUrl', [id]] end - BackgroundMigrationWorker.perform_bulk(jobs) + BackgroundMigrationWorker.bulk_perform_async(jobs) end end -- cgit v1.2.1