diff options
Diffstat (limited to 'doc')
32 files changed, 1168 insertions, 478 deletions
diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 42666357faf..b85a166089d 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -1,6 +1,6 @@ # Configuring GitLab for HA -Assuming you have already configured a database, Redis, and NFS, you can +Assuming you have already configured a [database](database.md), [Redis](redis.md), and [NFS](nfs.md), you can configure the GitLab application server(s) now. Complete the steps below for each GitLab application server in your environment. @@ -48,34 +48,33 @@ for each GitLab application server in your environment. data locations. See [NFS documentation](nfs.md) for `/etc/gitlab/gitlab.rb` configuration values for various scenarios. The example below assumes you've added NFS mounts in the default data locations. - + ```ruby external_url 'https://gitlab.example.com' # Prevent GitLab from starting if NFS data mounts are not available high_availability['mountpoint'] = '/var/opt/gitlab/git-data' - + # Disable components that will not be on the GitLab application server - postgresql['enable'] = false - redis['enable'] = false - + roles ['application_role'] + # PostgreSQL connection details gitlab_rails['db_adapter'] = 'postgresql' gitlab_rails['db_encoding'] = 'unicode' gitlab_rails['db_host'] = '10.1.0.5' # IP/hostname of database server gitlab_rails['db_password'] = 'DB password' - + # Redis connection details gitlab_rails['redis_port'] = '6379' gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server gitlab_rails['redis_password'] = 'Redis Password' ``` - - > **Note:** To maintain uniformity of links across HA clusters, the `external_url` - on the first application server as well as the additional application - servers should point to the external url that users will use to access GitLab. + + > **Note:** To maintain uniformity of links across HA clusters, the `external_url` + on the first application server as well as the additional application + servers should point to the external url that users will use to access GitLab. In a typical HA setup, this will be the url of the load balancer which will - route traffic to all GitLab application servers in the HA cluster. + route traffic to all GitLab application servers in the HA cluster. 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. diff --git a/doc/api/commits.md b/doc/api/commits.md index 5a4a8d888b3..c9b72d4a1dd 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -84,6 +84,7 @@ POST /projects/:id/repository/commits | `previous_path` | string | no | Original full path to the file being moved. Ex. `lib/class1.rb` | | `content` | string | no | File content, required for all except `delete`. Optional for `move` | | `encoding` | string | no | `text` or `base64`. `text` is default. | +| `last_commit_id` | string | no | Last known file commit id. Will be only considered in update, move and delete actions. | ```bash PAYLOAD=$(cat << 'JSON' diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 880b0ed2c65..4d3592e8f71 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -47,6 +47,7 @@ Parameters: | `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | | `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `search` | string | no | Search merge requests against their `title` and `description` | ```json [ @@ -161,6 +162,7 @@ Parameters: | `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | | `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `search` | string | no | Search merge requests against their `title` and `description` | ```json [ diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index c517a38a8ba..a1a0b1b756c 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -151,3 +151,4 @@ Parameters: - `author_email` (optional) - Specify the commit author's email address - `author_name` (optional) - Specify the commit author's name - `commit_message` (required) - Commit message +- `last_commit_id` (optional) - Last known file commit id diff --git a/doc/articles/artifactory_and_gitlab/index.md b/doc/articles/artifactory_and_gitlab/index.md index c64851bad2b..6a590b53727 100644 --- a/doc/articles/artifactory_and_gitlab/index.md +++ b/doc/articles/artifactory_and_gitlab/index.md @@ -1,278 +1 @@ -# How to deploy Maven projects to Artifactory with GitLab CI/CD - -> **Article [Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Fabio Busatto](https://gitlab.com/bikebilly) || -> **Publication date:** 2017-08-15 - -## Introduction - -In this article, we will show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) -to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://www.jfrog.com/artifactory/), and then use it from another Maven application as a dependency. - -You'll create two different projects: - -- `simple-maven-dep`: the app built and deployed to Artifactory (available at https://gitlab.com/gitlab-examples/maven/simple-maven-dep) -- `simple-maven-app`: the app using the previous one as a dependency (available at https://gitlab.com/gitlab-examples/maven/simple-maven-app) - -We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/). -We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it. - -## Create the simple Maven dependency - -First of all, you need an application to work with: in this specific case we will -use a simple one, but it could be any Maven application. This will be the -dependency you want to package and deploy to Artifactory, in order to be -available to other projects. - -### Prepare the dependency application - -For this article you'll use a Maven app that can be cloned from our example -project: - -1. Log in to your GitLab account -1. Create a new project by selecting **Import project from ➔ Repo by URL** -1. Add the following URL: - - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git - ``` -1. Click **Create project** - -This application is nothing more than a basic class with a stub for a JUnit based test suite. -It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen. - -The project structure is really simple, and you should consider these two resources: - -- `pom.xml`: project object model (POM) configuration file -- `src/main/java/com/example/dep/Dep.java`: source of our application - -### Configure the Artifactory deployment - -The application is ready to use, but you need some additional steps to deploy it to Artifactory: - -1. Log in to Artifactory with your user's credentials. -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. -1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value in order to have it configurable via secret variables. -1. Copy the snippet in the `pom.xml` file for your project, just after the - `dependencies` section. The snippet should look like this: - - ```xml - <distributionManagement> - <repository> - <id>central</id> - <name>83d43b5afeb5-releases</name> - <url>${env.MAVEN_REPO_URL}/libs-release-local</url> - </repository> - </distributionManagement> - ``` - -Another step you need to do before you can deploy the dependency to Artifactory -is to configure the authentication data. It is a simple task, but Maven requires -it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory -in the user's homedir. - -Since you want to use GitLab Runner to automatically deploy the application, you -should create the file in the project's home directory and set a command line -parameter in `.gitlab-ci.yml` to use the custom location instead of the default one: - -1. Create a folder called `.m2` in the root of your repository -1. Create a file called `settings.xml` in the `.m2` folder -1. Copy the following content into a `settings.xml` file: - - ```xml - <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" - xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> - <servers> - <server> - <id>central</id> - <username>${env.MAVEN_REPO_USER}</username> - <password>${env.MAVEN_REPO_PASS}</password> - </server> - </servers> - </settings> - ``` - - Username and password will be replaced by the correct values using secret variables. - -### Configure GitLab CI/CD for `simple-maven-dep` - -Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and deploy the dependency! - -GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs -that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). - -First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page -and add the following secret variables (replace them with your current values, of course): - -- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) -- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) -- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password) - -Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repo: - -```yaml -image: maven:latest - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -deploy: - stage: deploy - script: - - mvn $MAVEN_CLI_OPTS deploy - only: - - master -``` - -GitLab Runner will use the latest [Maven Docker image](https://hub.docker.com/_/maven/), which already contains all the tools and the dependencies you need to manage the project, -in order to run the jobs. - -Environment variables are set to instruct Maven to use the `homedir` of the repo instead of the user's home when searching for configuration and dependencies. - -Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process -by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary. - -Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. - -Deploy to Artifactory is done as defined by the secret variables we have just set up. -The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. - -Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. -If the deployment has been successful, the deploy job log will output: - -``` -[INFO] ------------------------------------------------------------------------ -[INFO] BUILD SUCCESS -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 1.983 s -``` - ->**Note**: -the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it. - -Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repo. - -## Create the main Maven application - -Now that you have the dependency available on Artifactory, it's time to use it! -Let's see how we can have it as a dependency to our main application. - -### Prepare the main application - -We'll use again a Maven app that can be cloned from our example project: - -1. Create a new project by selecting **Import project from ➔ Repo by URL** -1. Add the following URL: - - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-app.git - ``` -1. Click **Create project** - -This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` -file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter. - -Since Maven doesn't know how to resolve the dependency, you need to modify the configuration: - -1. Go back to Artifactory -1. Browse the `libs-release-local` repository -1. Select the `simple-maven-dep-1.0.jar` file -1. Find the configuration snippet from the **Dependency Declaration** section of the main panel -1. Copy the snippet in the `dependencies` section of the `pom.xml` file. - The snippet should look like this: - - ```xml - <dependency> - <groupId>com.example.dep</groupId> - <artifactId>simple-maven-dep</artifactId> - <version>1.0</version> - </dependency> - ``` - -### Configure the Artifactory repository location - -At this point you defined the dependency for the application, but you still miss where you can find the required files. -You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables. - -Here is how you can get the content of the file directly from Artifactory: - -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel -1. Click on **Generate Maven Settings** -1. Click on **Generate Settings** -1. Copy to clipboard the configuration file -1. Save the file as `.m2/settings.xml` in your repo - -Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application! - -### Configure GitLab CI/CD for `simple-maven-app` - -You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`. - -You want to leverage [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and run your awesome application, -and see if you can get the greeting as expected! - -All you need to do is to add the following `.gitlab-ci.yml` to the repo: - -```yaml -image: maven:latest - -stages: - - build - - test - - run - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -run: - stage: run - script: - - mvn $MAVEN_CLI_OPTS package - - mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App" -``` - -It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job. -Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically. - -And that's it! In the `run` job output log you will find a friendly hello to GitLab! - -## Conclusion - -In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts. - -A similar approach could be used to interact with any other Maven compatible Binary Repository Manager. -Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow. +This document was moved to [another location](../../ci/examples/artifactory_and_gitlab/index.md) diff --git a/doc/articles/index.md b/doc/articles/index.md index 862fe0868a6..7b60ce1d8b9 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -28,7 +28,6 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | :------------ | :------: | --------------: | | [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 | | [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | -| [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | | [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) | Concepts | 2017-07-11 | | [Continuous Integration: From Jenkins to GitLab Using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/) | Concepts | 2017-07-27 | diff --git a/doc/articles/laravel_with_gitlab_and_envoy/index.md b/doc/articles/laravel_with_gitlab_and_envoy/index.md index e0d8fb8d081..b20bd8c247a 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/index.md +++ b/doc/articles/laravel_with_gitlab_and_envoy/index.md @@ -502,8 +502,8 @@ stages: unit_test: stage: test script: - - composer install - cp .env.example .env + - composer install - php artisan key:generate - php artisan migrate - vendor/bin/phpunit diff --git a/doc/ci/README.md b/doc/ci/README.md index 12404eddbe2..05d792dea0f 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -121,7 +121,7 @@ Here is an collection of tutorials and guides on setting up your CI pipeline. - [Analyze code quality with the Code Climate CLI](examples/code_climate.md) - **Articles** - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](../articles/laravel_with_gitlab_and_envoy/index.md) - - [How to deploy Maven projects to Artifactory with GitLab CI/CD](../articles/artifactory_and_gitlab/index.md) + - [How to deploy Maven projects to Artifactory with GitLab CI/CD](examples/artifactory_and_gitlab/index.md) - [Automated Debian packaging](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) - [Spring boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) - [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 0a2419b7ed2..22afcb9199d 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -164,6 +164,21 @@ not without its own challenges: - By default, `docker:dind` uses `--storage-driver vfs` which is the slowest form offered. To use a different driver, see [Using the overlayfs driver](#using-the-overlayfs-driver). +- Since the `docker:dind` container and the runner container don't share their + root filesystem, the job's working directory can be used as a mount point for + children containers. For example, if you have files you want to share with a + child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` + and use it as your mount point (for a more thorough explanation, check [issue + #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): + + ```yaml + variables: + MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt + + script: + - mkdir -p "$MOUNT_POINT" + - docker run -v "$MOUNT_POINT:/mnt" my-docker-image + ``` An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index d05b4db953a..25a0c5dcff5 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -27,8 +27,7 @@ Apart from those, here is an collection of tutorials and guides on setting up yo ### Java -- **Articles:** - - [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) +- [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) ### Scala @@ -41,18 +40,15 @@ Apart from those, here is an collection of tutorials and guides on setting up yo ### Elixir - [Test a Phoenix application](test-phoenix-application.md) -- **Articles:** - - [Building an Elixir Release into a Docker image using GitLab CI](https://about.gitlab.com/2016/08/11/building-an-elixir-release-into-docker-image-using-gitlab-ci-part-1/) +- [Building an Elixir Release into a Docker image using GitLab CI](https://about.gitlab.com/2016/08/11/building-an-elixir-release-into-docker-image-using-gitlab-ci-part-1/) ### iOS -- **Articles:** - - [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) +- [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) ### Android -- **Articles:** - - [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) +- [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) ### Code quality analysis @@ -63,20 +59,19 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - [Using `dpl` as deployment tool](deployment/README.md) - [Repositories with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -- **Articles:** - - [Continuous Deployment with GitLab: how to build and deploy a Debian Package with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) +- [Continuous Deployment with GitLab: how to build and deploy a Debian Package with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) +- [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) -## GitLab CI for GitLab Pages +## GitLab CI/CD for GitLab Pages - [Example projects](https://gitlab.com/pages) -- **Articles:** - - [Creating and Tweaking `.gitlab-ci.yml` for GitLab Pages](../../user/project/pages/getting_started_part_four.md) - - [SSGs Part 3: Build any SSG site with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/): - examples for Ruby-, NodeJS-, Python-, and GoLang-based SSGs - - [Building a new GitLab docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) - - [Publish code coverage reports with GitLab Pages](https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) - -See the topic [GitLab Pages](../../user/project/pages/index.md) for a complete overview. +- [Creating and Tweaking `.gitlab-ci.yml` for GitLab Pages](../../user/project/pages/getting_started_part_four.md) +- [SSGs Part 3: Build any SSG site with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/): +examples for Ruby-, NodeJS-, Python-, and GoLang-based SSGs +- [Building a new GitLab docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) +- [Publish code coverage reports with GitLab Pages](https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) + +See the documentation on [GitLab Pages](../../user/project/pages/index.md) for a complete overview. ## More diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md new file mode 100644 index 00000000000..8e91cd05d8a --- /dev/null +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -0,0 +1,282 @@ +--- +redirect_from: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' +--- + +# How to deploy Maven projects to Artifactory with GitLab CI/CD + +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Fabio Busatto](https://gitlab.com/bikebilly) || +> **Publication date:** 2017-08-15 + +## Introduction + +In this article, we will show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) +to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://www.jfrog.com/artifactory/), and then use it from another Maven application as a dependency. + +You'll create two different projects: + +- `simple-maven-dep`: the app built and deployed to Artifactory (available at https://gitlab.com/gitlab-examples/maven/simple-maven-dep) +- `simple-maven-app`: the app using the previous one as a dependency (available at https://gitlab.com/gitlab-examples/maven/simple-maven-app) + +We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/). +We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it. + +## Create the simple Maven dependency + +First of all, you need an application to work with: in this specific case we will +use a simple one, but it could be any Maven application. This will be the +dependency you want to package and deploy to Artifactory, in order to be +available to other projects. + +### Prepare the dependency application + +For this article you'll use a Maven app that can be cloned from our example +project: + +1. Log in to your GitLab account +1. Create a new project by selecting **Import project from ➔ Repo by URL** +1. Add the following URL: + + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git + ``` +1. Click **Create project** + +This application is nothing more than a basic class with a stub for a JUnit based test suite. +It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen. + +The project structure is really simple, and you should consider these two resources: + +- `pom.xml`: project object model (POM) configuration file +- `src/main/java/com/example/dep/Dep.java`: source of our application + +### Configure the Artifactory deployment + +The application is ready to use, but you need some additional steps to deploy it to Artifactory: + +1. Log in to Artifactory with your user's credentials. +1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. +1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. +1. Change the `url` value in order to have it configurable via secret variables. +1. Copy the snippet in the `pom.xml` file for your project, just after the + `dependencies` section. The snippet should look like this: + + ```xml + <distributionManagement> + <repository> + <id>central</id> + <name>83d43b5afeb5-releases</name> + <url>${env.MAVEN_REPO_URL}/libs-release-local</url> + </repository> + </distributionManagement> + ``` + +Another step you need to do before you can deploy the dependency to Artifactory +is to configure the authentication data. It is a simple task, but Maven requires +it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory +in the user's homedir. + +Since you want to use GitLab Runner to automatically deploy the application, you +should create the file in the project's home directory and set a command line +parameter in `.gitlab-ci.yml` to use the custom location instead of the default one: + +1. Create a folder called `.m2` in the root of your repository +1. Create a file called `settings.xml` in the `.m2` folder +1. Copy the following content into a `settings.xml` file: + + ```xml + <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" + xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> + <servers> + <server> + <id>central</id> + <username>${env.MAVEN_REPO_USER}</username> + <password>${env.MAVEN_REPO_PASS}</password> + </server> + </servers> + </settings> + ``` + + Username and password will be replaced by the correct values using secret variables. + +### Configure GitLab CI/CD for `simple-maven-dep` + +Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and deploy the dependency! + +GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs +that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). + +First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page +and add the following secret variables (replace them with your current values, of course): + +- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) +- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) +- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password) + +Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repo: + +```yaml +image: maven:latest + +variables: + MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + +cache: + paths: + - .m2/repository/ + - target/ + +build: + stage: build + script: + - mvn $MAVEN_CLI_OPTS compile + +test: + stage: test + script: + - mvn $MAVEN_CLI_OPTS test + +deploy: + stage: deploy + script: + - mvn $MAVEN_CLI_OPTS deploy + only: + - master +``` + +GitLab Runner will use the latest [Maven Docker image](https://hub.docker.com/_/maven/), which already contains all the tools and the dependencies you need to manage the project, +in order to run the jobs. + +Environment variables are set to instruct Maven to use the `homedir` of the repo instead of the user's home when searching for configuration and dependencies. + +Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process +by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary. + +Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. + +Deploy to Artifactory is done as defined by the secret variables we have just set up. +The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. + +Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. +If the deployment has been successful, the deploy job log will output: + +``` +[INFO] ------------------------------------------------------------------------ +[INFO] BUILD SUCCESS +[INFO] ------------------------------------------------------------------------ +[INFO] Total time: 1.983 s +``` + +>**Note**: +the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it. + +Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repo. + +## Create the main Maven application + +Now that you have the dependency available on Artifactory, it's time to use it! +Let's see how we can have it as a dependency to our main application. + +### Prepare the main application + +We'll use again a Maven app that can be cloned from our example project: + +1. Create a new project by selecting **Import project from ➔ Repo by URL** +1. Add the following URL: + + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-app.git + ``` +1. Click **Create project** + +This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` +file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter. + +Since Maven doesn't know how to resolve the dependency, you need to modify the configuration: + +1. Go back to Artifactory +1. Browse the `libs-release-local` repository +1. Select the `simple-maven-dep-1.0.jar` file +1. Find the configuration snippet from the **Dependency Declaration** section of the main panel +1. Copy the snippet in the `dependencies` section of the `pom.xml` file. + The snippet should look like this: + + ```xml + <dependency> + <groupId>com.example.dep</groupId> + <artifactId>simple-maven-dep</artifactId> + <version>1.0</version> + </dependency> + ``` + +### Configure the Artifactory repository location + +At this point you defined the dependency for the application, but you still miss where you can find the required files. +You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables. + +Here is how you can get the content of the file directly from Artifactory: + +1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel +1. Click on **Generate Maven Settings** +1. Click on **Generate Settings** +1. Copy to clipboard the configuration file +1. Save the file as `.m2/settings.xml` in your repo + +Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application! + +### Configure GitLab CI/CD for `simple-maven-app` + +You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`. + +You want to leverage [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and run your awesome application, +and see if you can get the greeting as expected! + +All you need to do is to add the following `.gitlab-ci.yml` to the repo: + +```yaml +image: maven:latest + +stages: + - build + - test + - run + +variables: + MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + +cache: + paths: + - .m2/repository/ + - target/ + +build: + stage: build + script: + - mvn $MAVEN_CLI_OPTS compile + +test: + stage: test + script: + - mvn $MAVEN_CLI_OPTS test + +run: + stage: run + script: + - mvn $MAVEN_CLI_OPTS package + - mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App" +``` + +It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job. +Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically. + +And that's it! In the `run` job output log you will find a friendly hello to GitLab! + +## Conclusion + +In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts. + +A similar approach could be used to interact with any other Maven compatible Binary Repository Manager. +Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow. diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md new file mode 100644 index 00000000000..a7945d05cd0 --- /dev/null +++ b/doc/ci/examples/browser_performance.md @@ -0,0 +1,50 @@ +# Browser Performance Testing with the Sitespeed.io container + +This example shows how to run the [Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) on your code by using +GitLab CI/CD and [Sitespeed.io](https://www.sitespeed.io) using Docker-in-Docker. + +First, you need a GitLab Runner with the [docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `performance`: + +```yaml + stage: performance + image: docker:git + services: + - docker:dind + script: + - mkdir sitespeed-results + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io --outputFolder sitespeed-results https://my.website.com + artifacts: + paths: + - sitespeed-results/ +``` + +This will create a `performance` job in your CI/CD pipeline and will run Sitespeed.io against the webpage you define. The full HTML Sitespeed.io report will be saved as an artifact, and if you have Pages enabled it can be viewed directly in your browser. For further customization options of Sitespeed.io, including the ability to provide a list of URLs to test, please consult their [documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/). + +For GitLab [Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) users, a performance score can be automatically +extracted and shown right in the merge request widget. Learn more about [Browser Performance Testing](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). + +## Performance testing on Review Apps + +The above CI YML is great for testing against static environments, and it can be extended for dynamic environments. There are a few extra steps to take to set this up: +1. The `performance` job should run after the environment has started. +1. In the `deploy` job, persist the hostname so it is available to the `performance` job. The same can be done for static environments like staging and production to unify the code path. Saving it as an artifact is as simple as `echo $CI_ENVIRONMENT_URL > environment_url.txt`. +1. In the `performance` job read the artifact into an environment variable, like `$CI_ENVIRONMENT_URL`, and use it to parameterize the test URL's. +1. Now you can run the Sitespeed.io container against the desired hostname and paths. + +A simple `performance` job would look like: + +```yaml + stage: performance + image: docker:git + services: + - docker:dind + script: + - export CI_ENVIRONMENT_URL=$(cat environment_url.txt) + - mkdir sitespeed-results + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io --outputFolder sitespeed-results $CI_ENVIRONMENT_URL + artifacts: + paths: + - sitespeed-results/ +``` diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index db13e0e6249..9cb1f708a6a 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -18,6 +18,10 @@ like `docs.gitlab.com/user/project/merge_requests.html`. With this pattern, you can immediately tell that you are navigating a user related documentation and is about the project and its merge requests. +Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), +rather organise content by its subject (e.g. everything related to CI goes together) +and cross-link between any related content. + The table below shows what kind of documentation goes where. | Directory | What belongs here | diff --git a/doc/development/fe_guide/axios.md b/doc/development/fe_guide/axios.md index 962fe3dcec9..1daa6758171 100644 --- a/doc/development/fe_guide/axios.md +++ b/doc/development/fe_guide/axios.md @@ -11,7 +11,7 @@ This exported module should be used instead of directly using `axios` to ensure ## Usage ```javascript - import axios from '~/lib/utils/axios_utils'; + import axios from './lib/utils/axios_utils'; axios.get(url) .then((response) => { diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 43b996d9395..f493ad4ae66 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -262,6 +262,21 @@ Sometimes you need to add some context to the text that you want to translate s__('OpenedNDaysAgo|Opened') ``` +### Dates / times + +- In JavaScript: + +```js +import { createDateTimeFormat } from '.../locale'; + +const dateFormat = createDateTimeFormat({ year: 'numeric', month: 'long', day: 'numeric' }); +console.log(dateFormat.format(new Date('2063-04-05'))) // April 5, 2063 +``` + +This makes use of [`Intl.DateTimeFormat`]. + +[`Intl.DateTimeFormat`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DateTimeFormat + ## Adding a new language Let's suppose you want to add translations for a new language, let's say French. diff --git a/doc/development/i18n/index.md b/doc/development/i18n/index.md index 4cb2624c098..8aa0462d213 100644 --- a/doc/development/i18n/index.md +++ b/doc/development/i18n/index.md @@ -59,6 +59,7 @@ Requests to become a proof reader will be considered on the merits of previous t - French - German - Italian + - [Paolo Falomo](https://crowdin.com/profile/paolo.falomo) - Japanese - Korean - [Huang Tao](https://crowdin.com/profile/htve) diff --git a/doc/development/ux_guide/img/james-mackey.png b/doc/development/ux_guide/img/james-mackey.png Binary files differindex 6db257c5b39..c8f9097f69f 100644 --- a/doc/development/ux_guide/img/james-mackey.png +++ b/doc/development/ux_guide/img/james-mackey.png diff --git a/doc/development/ux_guide/img/karolina-plaskaty.png b/doc/development/ux_guide/img/karolina-plaskaty.png Binary files differindex 2e356c99762..ae2e98b7bad 100644 --- a/doc/development/ux_guide/img/karolina-plaskaty.png +++ b/doc/development/ux_guide/img/karolina-plaskaty.png diff --git a/doc/development/ux_guide/img/matthieu-poirier.png b/doc/development/ux_guide/img/matthieu-poirier.png Binary files differnew file mode 100644 index 00000000000..dd21948ebe2 --- /dev/null +++ b/doc/development/ux_guide/img/matthieu-poirier.png diff --git a/doc/development/ux_guide/img/nazim-ramesh.png b/doc/development/ux_guide/img/nazim-ramesh.png Binary files differindex 01ba0391630..cc3e197679d 100644 --- a/doc/development/ux_guide/img/nazim-ramesh.png +++ b/doc/development/ux_guide/img/nazim-ramesh.png diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md index fce882a45f1..6afb33cfc36 100644 --- a/doc/development/ux_guide/users.md +++ b/doc/development/ux_guide/users.md @@ -1,127 +1,252 @@ # UX Personas * [Nazim Ramesh](#nazim-ramesh) - - Small to medium size organisations using GitLab CE + - Small to medium size organizations using GitLab CE +* [Matthieu Poirier](#matthieu-poirier) + - Responsible for managing and maintaining GitLab installation + - Any size organization + - Using CE or EE * [James Mackey](#james-mackey) - - Medium to large size organisations using CE or EE - - Small organisations using EE + - Medium to large size organizations using CE or EE + - Small organizations using EE * [Karolina Plaskaty](#karolina-plaskaty) - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work - - Working for a medium to large size organisation + - Working within a medium to large size organization --- ## Nazim Ramesh -- Small to medium size organisations using GitLab CE +- Small to medium size organizations using GitLab CE -<img src="img/nazim-ramesh.png" width="300px"> + ### Demographics -- **Age**<br>32 years old -- **Location**<br>Germany -- **Education**<br>Bachelor of Science in Computer Science -- **Occupation**<br>Full-stack web developer -- **Programming experience**<br>Over 10 years -- **Frequently used programming languages**<br>JavaScript, SQL, PHP -- **Hobbies / interests**<br>Functional programming, open source, gaming, web development and web security. +**Age** + +32 years old + +**Location** + +Germany + +**Education** + +Bachelor of Science in Computer Science + +**Occupation** + +Full-stack web developer + +**Programming experience** + +Over 10 years + +**Frequently used programming languages** + +JavaScript, SQL, PHP + +**Hobbies / interests** + +Functional programming, open source, gaming, web development, and web security. ### Motivations -Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wishlist of features, Nazim began comparing source control tools. A search for “self-hosted Git server repository management” returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab: +Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wish list of features, Nazim began comparing source control tools. A search for "self-hosted Git server repository management" returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab: -> -“I wanted them to switch away from SVN. I needed a server application to manage repositories. The common tools that were around just didn’t meet the requirements. Most of them were too simple or plain...GitLab provided all the required features. Also costs had to be low, since we don’t have a big budget for those things...the Community Edition was perfect in this regard.” -> +>"I wanted them to switch away from SVN. I needed a server application to manage repositories. The common tools that were around just didn't meet the requirements. Most of them were too simple or plain...GitLab provided all the required features. Also, costs had to be low since we don't have a big budget for those things...the Community Edition was perfect in this regard." In his role as a full-stack web developer, Nazim could recommend products that he would like the engineering team to use, but final approval lay with his line manager, Mike, VP of Engineering. Nazim recalls that he was met with reluctance from his colleagues when he raised moving to Git and using GitLab. -> -“The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use.” -> +>"The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use." Undeterred, Nazim decided to migrate a couple of projects across to GitLab. -> -“Old SVN users couldn’t see the benefits of Git at first. It took a month or two to convince them.” -> +>"Old SVN users couldn't see the benefits of Git at first. It took a month or two to convince them." -Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab. +Slowly, by showing his colleagues how easy it was to use Git, the majority of the team's projects were migrated to GitLab. -The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab. +The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company's decision to move to GitLab. ### Frustrations #### Adoption to GitLab has been slow -Not only has the engineering team had to get to grips with Git, they’ve also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilised. Nazim sold GitLab to his manager as an ‘all in one’ tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn’t had the time to integrate the legacy tools to GitLab and he’s struggling to convince his peers to change their habits. +Not only has the engineering team had to get to grips with Git, they've also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilized. Nazim sold GitLab to his manager as an ‘all in one' tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn't had the time to integrate the legacy tools to GitLab and he's struggling to convince his peers to change their habits. #### Missing Features -Nazim’s company want GitLab to be able to do everything. There isn’t a large budget for software, so they’re selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim’s company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn’t able to do what he or his colleagues need it to do. +Nazim's company want GitLab to be able to do everything. There isn't a large budget for software, so they're selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim's company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn't able to do what he or his colleagues need it to do. #### Regressions and bugs -Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it “every new version adds something awesome, but breaks something”. He feels that “old issues for "minor" annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs.” Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it’s also counterproductive to fix workflows every month. +Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it "every new version adds something awesome, but breaks something". He feels that "old issues for minor annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs." Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it's also counterproductive to fix workflows every month. #### Uses too much RAM and CPU -> -“Memory usages mean that if we host it from a cloud based host like AWS, we spend almost as much on the instance as what we would pay GitHub” -> +>"Memory usages mean that if we host it from a cloud-based host like AWS, we spend almost as much on the instance as what we would pay GitHub" + #### UI/UX -GitLab’s interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab’s interface daily has left him frustrated at the lack of personalisation / control over his user experience. He’s also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. “Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release.” +GitLab's interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab's interface daily has left him frustrated at the lack of personalization/control over his user experience. He's also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. "Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release." ### Goals * To convince his colleagues to fully adopt GitLab CE, thus improving workflow and collaboration. -* To use a feature rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools. +* To use a feature-rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools. * To use an intuitive and stable product, so he can spend more time on his core job responsibilities and less time bug-fixing, guiding colleagues, etc. --- +## Matthieu Poirier +- Responsible for managing and maintaining GitLab installation +- Any size organization +- Using CE or EE + + + +### Demographics + +**Age** + +42 years old + +**Location** + +France + +**Education** + +Masters Degree in Computer Science + +**Occupation** + +DevOps Engineer + +**Programming experience** + +Over 10 years + +**Frequently used programming languages** + +JavaScript, SQL, PHP and Node.js + +**Hobbies / interests** + +Functional programming, data analysis, building apps, and tools. + +### Motivations +Matthieu works in DevOps for a web services company which currently hires 90 staff. When Matthieu first joined the company, he was responsible for managing a custom built in-house bug tracking tool and release management system. Over time, as the company grew, his colleagues requested more features and tools to help them in their day-to-day work. To meet their needs, Matthieu was forced to "hack together" a solution. In his own words, Matthieu explains that it became: + +>"...a huge pain managing access to all the individual pieces. In addition, they didn't have any integration with each other, nobody ended up using them and we couldn't do any workflows with merge requests and the like. I was sick of managing all those separate parts and wanted to move to a single platform that would handle it all." + + +He further explains that he wanted to introduce "better, easier, more formal code reviews" and to start using continuous integration and deployment. + +Matthieu tried to find a platform which would consolidate the company's existing toolset, and centralize code, documentation, and issues. He discovered GitHub, but the price was an issue: + +>"We needed to host our code on-site and wanted GitHub Enterprise functionality without the GitHub Enterprise costs." + + +Not only was GitLab cheaper than GitHub, it was also more cost-effective than maintaining multiple tools. Subsequently, Matthieu found it easy to sell the merits of GitLab to his manager. + +Matthieu describes GitLab as: + +>"the only tool that offers the real feeling of having everything you need in one place." + + +He credits himself as being entirely responsible for moving his company to GitLab. + +### Frustrations +#### Updating to the latest release +Matthieu introduced his company to GitLab. He is responsible for maintaining and managing the company's installation in addition to his day job. He feels updates are too frequent and he doesn't always have sufficient time to update GitLab. As a result, he's not up to date with releases. + +Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his set-up. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates." + +Matthieu is looking for confirmation that his update procedure is "secure and efficient" so more tutorials related to this topic would be useful to him. + +#### Configuration +Matthieu dislikes using the combination of gitlab.rb and the UI for changing settings. He explains that it "would be nice to be able to configure more from the Admin UI rather than just the config files." + +#### Creating a backup +Matthieu explains that the "backup solution is not well integrated into the UI", for example, he "cannot see if backups succeeded" or whether they have been rolled back to via the UI. + +#### Onboarding +It's Matthieu's responsibility to get teams across his organization up and running with GitLab. He explains that whilst many teams might be leveraging GitLab, they are: + +>"..not aware of GitLab's powerful CI or our omnibus install of Mattermost...It would be nice to have a tutorial type walkthrough available when a new user logs in on how to get started with all these features. AutoDevOps may solve some of this, but GitLab has many powerful features wrapped up into it and some [teams] may just think that it is only a Git repo similar to GitHub." + + +He states that there has been: "a sluggishness of others to adapt" and it's "a low-effort adaptation at that." + +### Goals +* To save time. One of the reasons Matthieu moved his company to GitLab was to reduce the effort it took him to manage and configure multiple tools, thus saving him time. He has to balance his day job in addition to managing the company's GitLab installation and onboarding new teams to GitLab. +* To use a platform which is easy to manage. Matthieu isn't a Systems Administrator, and when updating GitLab, creating backups, etc. He would prefer to work within GitLab's UI. Explanations / guided instructions when configuring settings in GitLab's interface would really help Matthieu. He needs reassurance that what he is about to change is + +1. the right setting +2. will provide him with the desired result he wants. + +* Matthieu needs to educate his colleagues about GitLab. Matthieu's colleagues won't adopt GitLab as they're unaware of its capabilities and the positive impact it could have on their work. Matthieu needs support in getting this message across to them. + +--- ## James Mackey -- Medium to large size organisations using CE or EE -- Small organisations using EE +- Medium to large size organizations using CE or EE +- Small organizations using EE -<img src="img/james-mackey.png" width="300px"> + ### Demographics -- **Age**<br>36 years old -- **Location**<br>US -- **Education**<br>Masters degree in Computer Science -- **Occupation**<br>Full-stack web developer -- **Programming experience**<br>Over 10 years -- **Frequently used programming languages**<br>JavaScript, SQL, Node.js, Java, PHP, Python -- **Hobbies / interests**<br>DevOps, open source, web development, science, automation and electronics. +**Age** + +36 years old + +**Location** + +US + +**Education** + +Masters degree in Computer Science + +**Occupation** + +Full-stack web developer + +**Programming experience** + +Over 10 years + +**Frequently used programming languages** + +JavaScript, SQL, Node.js, Java, PHP, Python + +**Hobbies / interests** + +DevOps, open source, web development, science, automation, and electronics. ### Motivations -James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved “the simplicity of installation, administration and use”. After using GitLab for over a year, he began to wonder about using it at work. James explains: +James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved "the simplicity of installation, administration and use". After using GitLab for over a year, he began to wonder about using it at work. James explains: -> -“We first installed the CE edition...on a staging server for a PoC and asked a beta team to use it, specifically for the Merge Request features. Soon other teams began asking us to be beta users too, because the team that was already using GitLab was really enjoying it.” -> +>"We first installed the CE edition...on a staging server for a PoC and asked a beta team to use it, specifically for the Merge Request features. Soon other teams began asking us to be beta users too because the team that was already using GitLab was really enjoying it." -James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it “less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub”. + +James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it "less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub". The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE. -James feels partially responsible for his organisation’s decision to start using GitLab. +James feels partially responsible for his organization's decision to start using GitLab. + +>"It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching." -> -“It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching.” -> ### Frustrations #### Third Party Integration -Some of GitLab EE’s features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn’t essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn’t sure what is/isn’t possible with plugins and what level of custom development he and his team will need to do. +Some of GitLab EE's features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn't essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn't sure what is/isn't possible with plugins and what level of custom development he and his team will need to do. #### UX/UI -James and his team use CI quite heavily for several projects. Whilst they’ve welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage’s log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab’s navigation needs to reviewed and optimised. +James and his team use CI quite heavily for several projects. Whilst they've welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage's log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab's navigation needs to reviewed and optimized. #### Permissions -> -“There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines.” -> +>"There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines." + ### Goals -* To be able to integrate third party tools easily with GitLab EE and to create custom integrations and patches where needed. +* To be able to integrate third-party tools easily with GitLab EE and to create custom integrations and patches where needed. * To use GitLab EE primarily for code hosting, merge requests, continuous integration and issue management. James and his team want to be able to understand and use these particular features easily. * To able to share one instance of GitLab EE with multiple teams across the business. Advanced user management, the ability to separate permissions on different parts of the source code, etc are important to James. @@ -130,36 +255,56 @@ James and his team use CI quite heavily for several projects. Whilst they’ve w ## Karolina Plaskaty - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work -- Working for a medium to large size organisation +- Working within a medium to large size organization -<img src="img/karolina-plaskaty.png" width="300px"> + ### Demographics -- **Age**<br>26 years old -- **Location**<br>UK -- **Education**<br>Self taught -- **Occupation**<br>Junior web-developer -- **Programming experience**<br>6 years -- **Frequently used programming languages**<br>JavaScript and SQL -- **Hobbies / interests**<br>Web development, mobile development, UX, open source, gaming and travel. +**Age** + +26 years old + +**Location** + +UK + +**Education** + +Self taught + +**Occupation** + +Junior web-developer + +**Programming experience** + +6 years + +**Frequently used programming languages** + +JavaScript and SQL + +**Hobbies / interests** + +Web development, mobile development, UX, open source, gaming, and travel. ### Motivations -Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab’s values and the fact that it isn’t a “behemoth of a company”. She explains that “displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work.” She’s also an avid reader of GitLab’s blog. +Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab's values and the fact that it isn't a "behemoth of a company". She explains that "displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work." She's also an avid reader of GitLab's blog. -Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as “old fashioned” and explains that it’s “less of a technical issue and more of a cultural issue” to convince upper management to move to GitLab. Karolina is also relatively new to the company so she’s apprehensive about pushing too hard to change version control platforms. +Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as "old fashioned" and explains that it's "less of a technical issue and more of a cultural issue" to convince upper management to move to GitLab. Karolina is also relatively new to the company so she's apprehensive about pushing too hard to change version control platforms. ### Frustrations #### Unable to use GitLab at work -Karolina wants to use GitLab at work but isn’t sure how to approach the subject with management. In her current role, she doesn’t feel that she has the authority to request GitLab. +Karolina wants to use GitLab at work but isn't sure how to approach the subject with management. In her current role, she doesn't feel that she has the authority to request GitLab. #### Performance -GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a “memory hog” which has deterred her from running GitLab on her own machine for just hobby / personal projects. +GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a "memory hog" which has deterred her from running GitLab on her own machine for just hobby / personal projects. #### UX/UI -Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, “it has too many links/buttons” and the navigation “feels a bit weird sometimes. I get lost if I don’t pay attention.” As Karolina also enjoys contributing to open-source projects, it’s important to her that GitLab is well designed for public repositories, she doesn’t feel that GitLab currently achieves this. +Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, "it has too many links/buttons" and the navigation "feels a bit weird sometimes. I get lost if I don't pay attention." As Karolina also enjoys contributing to open-source projects, it's important to her that GitLab is well designed for public repositories, she doesn't feel that GitLab currently achieves this. ### Goals * To develop her programming experience and to learn from other developers. * To contribute to both her own and other open source projects. -* To use a fast and intuitive version control platform. +* To use a fast and intuitive version control platform.
\ No newline at end of file diff --git a/doc/install/installation.md b/doc/install/installation.md index 56888b05609..67b89d608cc 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -133,9 +133,9 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz - echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz - cd ruby-2.3.5 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz + echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz + cd ruby-2.3.6 ./configure --disable-install-rdoc make sudo make install @@ -367,7 +367,7 @@ sudo usermod -aG redis git # Enable packfile bitmaps sudo -u git -H git config --global repack.writeBitmaps true - + # Enable push options sudo -u git -H git config --global receive.advertisePushOptions true diff --git a/doc/update/10.1-to-10.2.md b/doc/update/10.1-to-10.2.md index 9e0d8f79522..632e8befa74 100644 --- a/doc/update/10.1-to-10.2.md +++ b/doc/update/10.1-to-10.2.md @@ -339,11 +339,11 @@ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production If all items are green, then congratulations, the upgrade is complete! -## Things went south? Revert to previous version (10.0) +## Things went south? Revert to previous version (10.1) ### 1. Revert the code to the previous version -Follow the [upgrade guide from 9.5 to 10.0](9.5-to-10.0.md), except for the +Follow the [upgrade guide from 10.0 to 10.1](10.0-to-10.1.md), except for the database migration (the backup is already migrated to the previous version). ### 2. Restore from the backup diff --git a/doc/update/10.2-to-10.3.md b/doc/update/10.2-to-10.3.md index 07f9ee965f0..d6e2db8a353 100644 --- a/doc/update/10.2-to-10.3.md +++ b/doc/update/10.2-to-10.3.md @@ -339,11 +339,11 @@ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production If all items are green, then congratulations, the upgrade is complete! -## Things went south? Revert to previous version (10.0) +## Things went south? Revert to previous version (10.2) ### 1. Revert the code to the previous version -Follow the [upgrade guide from 9.5 to 10.0](9.5-to-10.0.md), except for the +Follow the [upgrade guide from 10.1 to 10.2](10.1-to-10.2.md), except for the database migration (the backup is already migrated to the previous version). ### 2. Restore from the backup diff --git a/doc/update/10.3-to-10.4.md b/doc/update/10.3-to-10.4.md new file mode 100644 index 00000000000..850cb3103f4 --- /dev/null +++ b/doc/update/10.3-to-10.4.md @@ -0,0 +1,360 @@ +--- +comments: false +--- + +# From 10.3 to 10.4 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz +echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz +cd ruby-2.3.6 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-4-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-4-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-3-stable:config/gitlab.yml.example origin/10-4-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-3-stable:lib/support/nginx/gitlab-ssl origin/10-4-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-3-stable:lib/support/nginx/gitlab origin/10-4-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-4-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-4-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-3-stable:lib/support/init.d/gitlab.default.example origin/10-4-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.3) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.2 to 10.3](10.2-to-10.3.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-4-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-4-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/user/group/img/transfer_project_to_other_group.png b/doc/user/group/img/transfer_project_to_other_group.png Binary files differdeleted file mode 100644 index 042c002f83f..00000000000 --- a/doc/user/group/img/transfer_project_to_other_group.png +++ /dev/null diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 1733017cbc0..7f77a33aadc 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -65,7 +65,7 @@ together in a single list view. > **Notes:** - For a list of words that are not allowed to be used as group names see the - [reserved names][reserved]. + [reserved names](../reserved_names.md). You can create a group in GitLab from: @@ -97,7 +97,7 @@ Add members to a group by navigating to the group's dashboard, and clicking **Me  -Select the [permission level][permissions] and add the new member. You can also set the expiring +Select the [permission level](../permissions.md#permissions) and add the new member. You can also set the expiring date for that user, from which they will no longer have access to your group. One of the benefits of putting multiple projects in one group is that you can @@ -152,16 +152,9 @@ There are two different ways to add a new project to a group:  -## Transfer an existing project into a group +## Transfer projects into groups -You can transfer an existing project into a group as long as you have at least **Master** [permissions][permissions] to that group -and if you are an **Owner** of the project. - - - -Find this option under your project's settings. - -GitLab administrators can use the admin interface to move any project to any namespace if needed. +Learn how to [transfer a project into a group](../project/index.md#transfer-an-existing-project-into-a-group). ## Sharing a project with a group @@ -190,24 +183,29 @@ access further configurations for your group. #### Changing a group's path -> **Note:** If you want to retain ownership over the original namespace and -protect the URL redirects, then instead of changing a group's path or renaming a -username, you can create a new group and transfer projects to it. +Changing a group's path can have unintended side effects. Read +[how redirects will behave](../project/index.md#redirects-when-changing-repository-paths) +before proceeding. -Changing a group's path can have unintended side effects. +If you are vacating the path so it can be claimed by another group or user, +you may need to rename the group name as well since both names and paths must +be unique. -* Existing web URLs for the group and anything under it (i.e. projects) will -redirect to the new URLs. -* Existing Git remote URLs for projects under the group will redirect to the new remote URL, and they -will show a warning with the new remote URL. -* The redirect to the new URL is permanent, that implies the original namespace -can't be claimed again by any group or user. -* If you are vacating the path so it can be claimed by another group or user, -you may need to rename the group name as well since both names and paths must be -unique +To change your group path: -> It is currently not possible to rename a namespace if it contains a -project with container registry tags, because the project cannot be moved. +1. Navigate to your group's **Settings > General**. +1. Enter a new name under "Group path". +1. Hit **Save group**. + +CAUTION: **Caution:** +It is currently not possible to rename a namespace if it contains a +project with [Container Registry](../project/container_registry.md) tags, +because the project cannot be moved. + +TIP: **TIP:** +If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing a group's path or renaming a +username, you can create a new group and transfer projects to it. #### Enforce 2FA to group members @@ -246,11 +244,7 @@ Learn more about [Member Lock](https://docs.gitlab.com/ee/user/group/index.html# - **Projects**: view all projects within that group, add members to each project, access each project's settings, and remove any project from the same screen. - **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) -and [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html#push-rules) to your group (Push Rules is available in [GitLab Enteprise Edition Starter][ee].) +and [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html#push-rules) to your group (Push Rules is available in [GitLab Enteprise Edition Starter](https://about.gitlab.com/products/).) - **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html#audit-events) for the group (GitLab admins only, available in [GitLab Enterprise Edition Starter][ee]). - **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group - -[permissions]: ../permissions.md#permissions -[ee]: https://about.gitlab.com/products/ -[reserved]: ../reserved_names.md diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index dae4cbe170b..ab16f8d14c1 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -8,51 +8,6 @@ experience according to the best approach to their cases. There are several ways to sign into your GitLab account. See the [authentication topic](../../topics/authentication/index.md) for more details. -### Why do I keep getting signed out? - -When signing in to the main GitLab application, a `_gitlab_session` cookie is -set. `_gitlab_session` is cleared client-side when you close your browser -and expires after "Application settings -> Session duration (minutes)"/`session_expire_delay` -(defaults to `10080` minutes = 7 days). - -When signing in to the main GitLab application, you can also check the -"Remember me" option which sets the `remember_user_token` -cookie (via [`devise`](https://github.com/plataformatec/devise)). -`remember_user_token` expires after -`config/initializers/devise.rb` -> `config.remember_for` (defaults to 2 weeks). - -When the `_gitlab_session` expires or isn't available, GitLab uses the `remember_user_token` -to get you a new `_gitlab_session` and keep you signed in through browser restarts. - -After your `remember_user_token` expires and your `_gitlab_session` is cleared/expired, -you will be asked to sign in again to verify your identity (which is for security reasons). - -## Username - -Your `username` is a unique [`namespace`](../group/index.md#namespaces) -related to your user ID. - -### Changing your username - -You can change your `username` from your -[profile settings](#profile-settings). - -> **Note:** If you want to retain ownership over the original namespace and -protect the URL redirects, then instead of changing your username, you can -create a new group and transfer projects to it. -Alternatively, you can follow [this detailed procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). - -Changing your username can have unintended side effects. - -* Existing web URLs for the user and anything under it (i.e. projects) will -redirect to the new URLs. -* Existing Git remote URLs for projects under the user will redirect to the new remote URL. Git responses -will show a warning with the new remote URL. -* The redirect to the new URL is permanent, that implies the original namespace can't be claimed again by any group or user. - -> It is currently not possible to rename a namespace if it contains a -project with container registry tags, because the project cannot be moved. - ## User profile Your profile is available from the up-right corner menu bar (user's avatar) > **Profile**, @@ -84,4 +39,51 @@ From there, you can: - Manage [SSH keys](../../ssh/README.md#ssh) to access your account via SSH - Manage your [preferences](preferences.md#syntax-highlighting-theme) to customize your own GitLab experience -- Acess your audit log, a security log of important events involving your account +- Access your audit log, a security log of important events involving your account + +## Changing your username + +Your `username` is a unique [`namespace`](../group/index.md#namespaces) +related to your user ID. Changing it can have unintended side effects, read +[how redirects will behave](../project/index.md#redirects-when-changing-repository-paths) +before proceeding. + +To change your `username`: + +1. Navigate to your [profile's](#profile-settings) **Settings > Account**. +1. Enter a new username under "Change username". +1. Hit **Update username**. + +CAUTION: **Caution:** +It is currently not possible to change your username if it contains a +project with [Container Registry](../project/container_registry.md) tags, +because the project cannot be moved. + +TIP: **Tip:** +If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing a group's path or renaming a +username, you can create a new group and transfer projects to it. +Alternatively, you can follow [this detailed procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom) +which also covers the case where you have projects hosted with +[GitLab Pages](../project/pages/index.md). + +## Troubleshooting + +### Why do I keep getting signed out? + +When signing in to the main GitLab application, a `_gitlab_session` cookie is +set. `_gitlab_session` is cleared client-side when you close your browser +and expires after "Application settings -> Session duration (minutes)"/`session_expire_delay` +(defaults to `10080` minutes = 7 days). + +When signing in to the main GitLab application, you can also check the +"Remember me" option which sets the `remember_user_token` +cookie (via [`devise`](https://github.com/plataformatec/devise)). +`remember_user_token` expires after +`config/initializers/devise.rb` -> `config.remember_for` (defaults to 2 weeks). + +When the `_gitlab_session` expires or isn't available, GitLab uses the `remember_user_token` +to get you a new `_gitlab_session` and keep you signed in through browser restarts. + +After your `remember_user_token` expires and your `_gitlab_session` is cleared/expired, +you will be asked to sign in again to verify your identity (which is for security reasons). diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 5d91aef5735..4c772c62f8d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -93,6 +93,9 @@ from your fork to the upstream project ## Project settings +Set the project's visibility level and the access levels to its various pages +and perform actions like archiving, renaming or transferring a project. + Read through the documentation on [project settings](settings/index.md). ## Import or export a project @@ -116,3 +119,32 @@ when a project is part of a group (under a [group namespace](../group/index.md#namespaces)). If you choose to leave a project you will no longer be a project member, therefore, unable to contribute. + +## Redirects when changing repository paths + +When a repository path changes, it is essential to smoothly transition from the +old location to the new one. GitLab provides two kinds of redirects: the web UI +and Git push/pull redirects. + +Depending on the situation, different things apply. + +When [transferring a project](settings/index.md#transferring-an-existing-project-into-another-namespace), +or [renaming a user](../profile/index.md#changing-your-username) or +[changing a group path](../group/index.md#changing-a-group-s-path): + +- **The redirect to the new URL is permanent**, which means that the original + namespace can't be claimed again by any group or user. +- Existing web URLs for the namespace and anything under it (e.g., projects) will + redirect to the new URLs. +- Starting with GitLab 10.3, existing Git remote URLs for projects under the + namespace will redirect to the new remote URL. Every time you push/pull to a + repository that has changed its location, a warning message to update + your remote will be displayed instead of rejecting your action. + This means that any automation scripts, or Git clients will continue to + work after a rename, making any transition a lot smoother. + To avoid pulling from or pushing to an entirely incorrect repository, the old + path will be reserved. + +When [renaming-a-repository](settings/index.md#renaming-a-repository), the same +things apply, except for the Git push/pull actions which will be rejected with a +warning message to change to the new remote URL. diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 93aec56f8dc..7dc234a9759 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -98,6 +98,9 @@ password as they will be needed when configuring GitLab in the next section. - GitLab 8.14 introduced a new way to integrate with JIRA which greatly simplified the configuration options you have to enter. If you are using an older version, [follow this documentation][jira-repo-old-docs]. +- In order to support Oracle's Access Manager, GitLab will send additional cookies + to enable Basic Auth. The cookie being added to each request is `OBBasicAuth` with + a value of `fromDialog`. To enable JIRA integration in a project, navigate to the [Integrations page](project_services.md#accessing-the-project-services), click diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 2b6fde1e2a5..f01fa5b1860 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -1,11 +1,20 @@ # Project settings +NOTE: **Note:** +Only project Masters and Admin users have the [permissions] to access a project +settings. + You can adjust your [project](../index.md) settings by navigating to your project's homepage and clicking **Settings**. ## General settings -Adjust your project's path and name, description, avatar, [default branch](../repository/branches/index.md#default-branch), and tags: +Under a project's general settings you can find everything concerning the +functionality of a project. + +### General project settings + +Adjust your project's name, description, avatar, [default branch](../repository/branches/index.md#default-branch), and tags:  @@ -45,14 +54,64 @@ Here you can run housekeeping, archive, rename, transfer, or remove a project. #### Archiving a project ->**Note:** Only Project Owners and Admin users have the permission to archive a project +NOTE: **Note:** +Only project Owners and Admin users have the [permissions] to archive a project. + +An archived project will be hidden by default in the project listings. + +1. Navigate to your project's **Settings > General > Advanced settings**. +1. Under "Archive project", hit the **Archive project** button. +1. Confirm the action when asked to. + +An archived project can be fully restored and will therefore retain its +repository and all associated resources whilst in an archived state. + +#### Renaming a repository + +NOTE: **Note:** +Only project Masters and Admin users have the [permissions] to rename a +repository. Not to be confused with a project's name where it can also be +changed from the [general project settings](#general-project-settings). + +A project's repository name defines its URL (the one you use to access the +project via a browser) and its place on the file disk where GitLab is installed. + +To rename a repository: + +1. Navigate to your project's **Settings > General > Advanced settings**. +1. Under "Rename repository", change the "Path" to your liking. +1. Hit **Rename project**. + +Remember that this can have unintended side effects since everyone with the +old URL will not be able to push or pull. Read more about what happens with the +[redirects when renaming repositories](../index.md#redirects-when-changing-repository-paths). + +#### Transferring an existing project into another namespace + +NOTE: **Note:** +Only project Owners and Admin users have the [permissions] to transfer a project. + +You can transfer an existing project into a [group](../../group/index.md) if: + +1. you have at least **Master** [permissions] to that group +1. you are an **Owner** of the project. + +Similarly, if you are an owner of a group, you can transfer any of its projects +under your own user. -It's possible to mark a project as archived via the Project Settings. An archived project will be hidden by default in the project listings. +To transfer a project: -An archived project can be fully restored and will therefore retain it's repository and all associated resources whilst in an archived state. +1. Navigate to your project's **Settings > General > Advanced settings**. +1. Under "Transfer project", choose the namespace you want to transfer the + project to. +1. Confirm the transfer by typing the project's path as instructed. -#### Renaming a project +Once done, you will be taken to the new project's namespace. At this point, +read what happens with the +[redirects from the old project to the new one](../index.md#redirects-when-changing-repository-paths). ->**Note:** Only Project Owners and Admin users have the permission to rename a project +NOTE: **Note:** +GitLab administrators can use the admin interface to move any project to any +namespace if needed. -It's possible to rename a project from "Rename repository" or "Transfer project" sections. When doing so, you will need to update your local repositories to point to the new location, otherwise Git operations will be rejected. +[permissions]: ../../permissions.md##project-members-permissions diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 6adde447975..195285f9157 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -163,3 +163,11 @@ For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Wi More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). + +### LFS objects are missing on push + +GitLab checks files to detect LFS pointers on push. If LFS pointers are detected, GitLab tries to verify that those files already exist in LFS on GitLab. + +Verify that LFS in installed locally and consider a manual push with `git lfs push --all`. + +If you are storing LFS files outside of GitLab you can disable LFS on the project by settting `lfs_enabled: false` with the [projets api](../../api/projects.md#edit-project). |
