diff options
Diffstat (limited to 'doc')
22 files changed, 252 insertions, 1081 deletions
diff --git a/doc/api/graphql/reference/gitlab_schema.graphql b/doc/api/graphql/reference/gitlab_schema.graphql index 64dd074a912..ac24ab0bc49 100644 --- a/doc/api/graphql/reference/gitlab_schema.graphql +++ b/doc/api/graphql/reference/gitlab_schema.graphql @@ -19316,7 +19316,17 @@ type Project { tagList: String """ - Terraform states associated with the project + Find a single Terraform state by name. + """ + terraformState( + """ + Name of the Terraform state. + """ + name: String! + ): TerraformState + + """ + Terraform states associated with the project. """ terraformStates( """ diff --git a/doc/api/graphql/reference/gitlab_schema.json b/doc/api/graphql/reference/gitlab_schema.json index 790c641a6c0..3c751925558 100644 --- a/doc/api/graphql/reference/gitlab_schema.json +++ b/doc/api/graphql/reference/gitlab_schema.json @@ -56123,8 +56123,35 @@ "deprecationReason": null }, { + "name": "terraformState", + "description": "Find a single Terraform state by name.", + "args": [ + { + "name": "name", + "description": "Name of the Terraform state.", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "TerraformState", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "terraformStates", - "description": "Terraform states associated with the project", + "description": "Terraform states associated with the project.", "args": [ { "name": "after", diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 31e9e46b6be..1497e8c2aca 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -2787,7 +2787,8 @@ Autogenerated return type of PipelineRetry. | `statistics` | ProjectStatistics | Statistics of the project | | `suggestionCommitMessage` | String | The commit message used to apply merge request suggestions | | `tagList` | String | List of project topics (not Git tags) | -| `terraformStates` | TerraformStateConnection | Terraform states associated with the project | +| `terraformState` | TerraformState | Find a single Terraform state by name. | +| `terraformStates` | TerraformStateConnection | Terraform states associated with the project. | | `userPermissions` | ProjectPermissions! | Permissions for the current user on the resource | | `visibility` | String | Visibility of the project | | `vulnerabilities` | VulnerabilityConnection | Vulnerabilities reported on the project | diff --git a/doc/api/groups.md b/doc/api/groups.md index eb255f8de00..958e876ba01 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -941,6 +941,19 @@ The `shared_runners_setting` attribute determines whether shared runners are ena | `disabled_with_override` | Disables shared runners for all projects and subgroups in this group, but allows subgroups to override this setting. | | `disabled_and_unoverridable` | Disables shared runners for all projects and subgroups in this group, and prevents subgroups from overriding this setting. | +### Upload a group avatar + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36681) in GitLab 12.9. + +To upload an avatar file from your file system, use the `--form` argument. This causes +curl to post data using the header `Content-Type: multipart/form-data`. The +`file=` parameter must point to a file on your file system and be preceded by +`@`. For example: + +```shell +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/22" --form "avatar=@/tmp/example.png" +``` + ## Remove group Only available to group owners and administrators. diff --git a/doc/ci/README.md b/doc/ci/README.md index 740be7d1dbd..391d32edfa7 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -160,8 +160,6 @@ Its feature set is listed on the table below according to DevOps stages. Find example project code and tutorials for using GitLab CI/CD with a variety of app frameworks, languages, and platforms on the [CI Examples](examples/README.md) page. -GitLab also provides [example projects](https://gitlab.com/gitlab-examples) pre-configured to use GitLab CI/CD. - ## Administration **(CORE ONLY)** As a GitLab administrator, you can change the default behavior diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 9fa0bb080ac..3384f76b686 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -23,31 +23,35 @@ Examples are available in several forms. As a collection of: The following table lists examples with step-by-step tutorials that are contained in this section: | Use case | Resource | -|:------------------------------|:---------| +|-------------------------------|----------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | | Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | | End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | | Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | | Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | | NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | | PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | | PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | | PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | | Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | | Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -### How to contributing examples +### Contributed examples + +You can help people that use your favorite programming language by submitting a link +to a guide for that language. These contributed guides are hosted externally or in +separate example projects: -Contributions are welcome! You can help your favorite programming -language users and GitLab by sending a merge request with a guide for that language. +| Use case | Resource | +|-------------------------------|----------| +| Game development | [DevOps and Game Dev with GitLab CI/CD](https://gitlab.com/gitlab-examples/gitlab-game-demo/). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](https://gitlab.com/gitlab-examples/maven/simple-maven-example). | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). | +| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](https://gitlab.com/gitlab-examples/scala-sbt). | ## CI/CD templates diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index c1df21297e3..a1a7de26cf2 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -1,289 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' -author: Fabio Busatto -author_gitlab: bikebilly -type: tutorial -date: 2017-08-15 +redirect_to: '../README.md#contributed-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# How to deploy Maven projects to Artifactory with GitLab CI/CD - -## Introduction - -In this article, we show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) -to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://jfrog.com/artifactory/), and then use it from another Maven application as a dependency. - -You'll create two different projects: - -- `simple-maven-dep`: the app built and deployed to Artifactory (see the [simple-maven-dep](https://gitlab.com/gitlab-examples/maven/simple-maven-dep) example project) -- `simple-maven-app`: the app using the previous one as a dependency (see the [simple-maven-app](https://gitlab.com/gitlab-examples/maven/simple-maven-app) example project) - -We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/). -We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it. - -## Create the simple Maven dependency - -First, you need an application to work with: in this specific case we'll use a -simple one, but it could be any Maven application. This will be the dependency -you want to package and deploy to Artifactory, to be available to other -projects. - -### Prepare the dependency application - -For this article you'll use a Maven app that can be cloned from our example -project: - -1. Log in to your GitLab account -1. Create a new project by selecting **Import project from > Repo by URL** -1. Add the following URL: - - ```plaintext - https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git - ``` - -1. Click **Create project** - -This application is nothing more than a basic class with a stub for a JUnit based test suite. -It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen. - -The project structure is really simple, and you should consider these two resources: - -- `pom.xml`: project object model (POM) configuration file -- `src/main/java/com/example/dep/Dep.java`: source of our application - -### Configure the Artifactory deployment - -The application is ready to use, but you need some additional steps to deploy it to Artifactory: - -1. Log in to Artifactory with your user's credentials. -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. -1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value to have it configurable by using variables. -1. Copy the snippet in the `pom.xml` file for your project, just after the - `dependencies` section. The snippet should look like this: - - ```xml - <distributionManagement> - <repository> - <id>central</id> - <name>83d43b5afeb5-releases</name> - <url>${env.MAVEN_REPO_URL}/libs-release-local</url> - </repository> - </distributionManagement> - ``` - -Another step you need to do before you can deploy the dependency to Artifactory -is to configure the authentication data. It is a simple task, but Maven requires -it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory -in the user's homedir. - -Since you want to use a runner to automatically deploy the application, you -should create the file in the project's home directory and set a command line -parameter in `.gitlab-ci.yml` to use the custom location instead of the default one: - -1. Create a folder called `.m2` in the root of your repository -1. Create a file called `settings.xml` in the `.m2` folder -1. Copy the following content into a `settings.xml` file: - - ```xml - <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" - xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> - <servers> - <server> - <id>central</id> - <username>${env.MAVEN_REPO_USER}</username> - <password>${env.MAVEN_REPO_PASS}</password> - </server> - </servers> - </settings> - ``` - - Username and password will be replaced by the correct values using variables. - -### Configure GitLab CI/CD for `simple-maven-dep` - -Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and deploy the dependency! - -GitLab CI/CD uses a file in the root of the repository, named `.gitlab-ci.yml`, to read the definitions for jobs -that will be executed by the configured runners. You can read more about this file in the [GitLab Documentation](../../yaml/README.md). - -First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Environment variables** page -and add the following ones (replace them with your current values, of course): - -- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) -- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) -- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password) - -Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repository: - -```yaml -image: maven:latest - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -deploy: - stage: deploy - script: - - mvn $MAVEN_CLI_OPTS deploy - only: - - master -``` - -The runner uses the latest [Maven Docker image](https://hub.docker.com/_/maven/), -which contains all of the tools and dependencies needed to manage the project -and to run the jobs. - -Environment variables are set to instruct Maven to use the `homedir` of the repository instead of the user's home when searching for configuration and dependencies. - -Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process -by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary. - -Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. - -Deploy to Artifactory is done as defined by the variables we have just set up. -The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. - -Done! Now you have all the changes in the GitLab repository, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. -If the deployment has been successful, the deploy job log will output: - -```plaintext -[INFO] ------------------------------------------------------------------------ -[INFO] BUILD SUCCESS -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 1.983 s -``` - ->**Note**: -the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it. - -Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repository. - -## Create the main Maven application - -Now that you have the dependency available on Artifactory, it's time to use it! -Let's see how we can have it as a dependency to our main application. - -### Prepare the main application - -We'll use again a Maven app that can be cloned from our example project: - -1. Create a new project by selecting **Import project from ➔ Repo by URL** -1. Add the following URL: - - ```plaintext - https://gitlab.com/gitlab-examples/maven/simple-maven-app.git - ``` - -1. Click **Create project** - -This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` -file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter. - -Since Maven doesn't know how to resolve the dependency, you need to modify the configuration: - -1. Go back to Artifactory -1. Browse the `libs-release-local` repository -1. Select the `simple-maven-dep-1.0.jar` file -1. Find the configuration snippet from the **Dependency Declaration** section of the main panel -1. Copy the snippet in the `dependencies` section of the `pom.xml` file. - The snippet should look like this: - - ```xml - <dependency> - <groupId>com.example.dep</groupId> - <artifactId>simple-maven-dep</artifactId> - <version>1.0</version> - </dependency> - ``` - -### Configure the Artifactory repository location - -At this point you defined the dependency for the application, but you still miss where you can find the required files. -You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables. - -Here is how you can get the content of the file directly from Artifactory: - -1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel -1. Click on **Generate Maven Settings** -1. Click on **Generate Settings** -1. Copy to clipboard the configuration file -1. Save the file as `.m2/settings.xml` in your repository - -Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application! - -### Configure GitLab CI/CD for `simple-maven-app` - -You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`. - -You want to leverage [GitLab CI/CD](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/) to automatically build, test and run your awesome application, -and see if you can get the greeting as expected! - -All you need to do is to add the following `.gitlab-ci.yml` to the repository: - -```yaml -image: maven:latest - -stages: - - build - - test - - run - -variables: - MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" - MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" - -cache: - paths: - - .m2/repository/ - - target/ - -build: - stage: build - script: - - mvn $MAVEN_CLI_OPTS compile - -test: - stage: test - script: - - mvn $MAVEN_CLI_OPTS test - -run: - stage: run - script: - - mvn $MAVEN_CLI_OPTS package - - mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App" -``` - -It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job. -Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically. - -And that's it! In the `run` job output log you will find a friendly hello to GitLab! - -## Conclusion - -In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts. - -A similar approach could be used to interact with any other Maven compatible Binary Repository Manager. -Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png Binary files differdeleted file mode 100644 index e76767741ce..00000000000 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_variables.png +++ /dev/null diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png Binary files differdeleted file mode 100644 index f3761632556..00000000000 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png +++ /dev/null diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index 9c145677f6e..a1a7de26cf2 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -1,145 +1,8 @@ --- -stage: Release -group: Release -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -author: Dylan Griffith -author_gitlab: DylanGriffith -type: tutorial -date: 2018-06-07 -description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD" +redirect_to: '../README.md#contributed-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD - -## Introduction - -This article demonstrates how to use the [Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) -method to deploy a [Spring Boot](https://projects.spring.io/spring-boot/) application to -[Cloud Foundry (CF)](https://www.cloudfoundry.org/) -with GitLab CI/CD. - -All the code for this project can be found in this [GitLab -repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). - -In case you're interested in deploying Spring Boot applications to Kubernetes -using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/blog/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/). - -## Requirements - -This tutorial assumes you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD. - -To follow along, you need: - -- An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any - other Cloud Foundry (CF) instance. -- An account on GitLab. - -NOTE: -If you're not deploying to PWS, you must replace the `api.run.pivotal.io` URL in all the below -commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) -of your CF instance. - -## Create your project - -To create your Spring Boot application you can use the Spring template in -GitLab when creating a new project: - -![New Project From Template](img/create_from_template.png) - -## Configure the deployment to Cloud Foundry - -To deploy to Cloud Foundry you must add a `manifest.yml` file. This -is the configuration for the CF CLI you must use to deploy the application. -Create this in the root directory of your project with the following -content: - -```yaml ---- -applications: - - name: gitlab-hello-world - random-route: true - memory: 1G - path: target/demo-0.0.1-SNAPSHOT.jar -``` - -## Configure GitLab CI/CD to deploy your application - -Now you must add the GitLab CI/CD configuration file -([`.gitlab-ci.yml`](../../yaml/README.md)) -to your project's root. This is how GitLab figures out what commands must run whenever -code is pushed to your repository. Add the following `.gitlab-ci.yml` -file to the root directory of the repository. GitLab detects it -automatically and runs the defined steps once you push your code: - -```yaml -image: java:8 - -stages: - - build - - deploy - -before_script: - - chmod +x mvnw - -build: - stage: build - script: ./mvnw package - artifacts: - paths: - - target/demo-0.0.1-SNAPSHOT.jar - -production: - stage: deploy - script: - - curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx - - ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io - - ./cf push - only: - - master -``` - -This uses the `java:8` [Docker image](../../docker/using_docker_images.md) -to build your application, as it provides the up-to-date Java 8 JDK on [Docker Hub](https://hub.docker.com/). -You also added the [`only` clause](../../yaml/README.md#onlyexcept-basic) -to ensure your deployments only happen when you push to the master branch. - -Because the steps defined in `.gitlab-ci.yml` require credentials to sign in to -CF, you must add your CF credentials as -[environment variables](../../variables/README.md#predefined-environment-variables) -in GitLab CI/CD. To set the environment variables, navigate to your project's -**Settings > CI/CD**, and then expand **Variables**. Name the variables -`CF_USERNAME` and `CF_PASSWORD` and set them to the correct values. - -![Variable Settings in GitLab](img/cloud_foundry_variables.png) - -After set up, GitLab CI/CD deploys your app to CF at every push to your -repository's default branch. To review the build logs or watch your builds -running live, navigate to **CI/CD > Pipelines**. - -WARNING: -It's considered best practice for security to create a separate deploy user for -your application and add its credentials to GitLab instead of using a -developer's credentials. - -To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click -**Run Pipeline**. After the app is finished deploying, it displays the -URL of your application in the logs for the `production` job: - -```shell -requested state: started -instances: 1/1 -usage: 1G x 1 instances -urls: gitlab-hello-world-undissembling-hotchpot.cfapps.io -last uploaded: Mon Nov 6 10:02:25 UTC 2017 -stack: cflinuxfs2 -buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1... - - state since cpu memory disk details -#0 running 2017-11-06 09:03:22 PM 120.4% 291.9M of 1G 137.6M of 1G -``` - -You can then visit your deployed application (for this example, -`https://gitlab-hello-world-undissembling-hotchpot.cfapps.io/`) and you should -see the "Spring is here!" message. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png Binary files differdeleted file mode 100644 index 09eef98202f..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png Binary files differdeleted file mode 100644 index 71ffcdea289..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png Binary files differdeleted file mode 100644 index a9452577a42..00000000000 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png +++ /dev/null diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 298ffff568a..0dc412bf002 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -1,534 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -author: Ryan Hall -author_gitlab: blitzgren -type: tutorial -date: 2018-03-07 +redirect_to: '../README.md#ci-cd-examples' --- -<!-- vale off --> +This document was moved to [another location](../README.md#contributed-examples). -# DevOps and Game Dev with GitLab CI/CD - -With advances in WebGL and WebSockets, browsers are extremely viable as game development -platforms without the use of plugins like Adobe Flash. Furthermore, by using GitLab and [AWS](https://aws.amazon.com/), -single game developers, as well as game dev teams, can easily host browser-based games online. - -In this tutorial, we'll focus on DevOps, as well as testing and hosting games with Continuous -Integration/Deployment methods using [GitLab CI/CD](../../README.md). We assume you are familiar with GitLab, JavaScript, -and the basics of game development. - -## The game - -Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction. - -Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](https://www.darknova.io), -was essential for the fast pace the team worked at. This tutorial will build upon my -[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps: - -1. Using code from the previous article to start with a bare-bones [Phaser](https://phaser.io) game built by a gulp file -1. Adding and running unit tests -1. Creating a `Weapon` class that can be triggered to spawn a `Bullet` in a given direction -1. Adding a `Player` class that uses this weapon and moves around the screen -1. Adding the sprites we will use for the `Player` and `Weapon` -1. Testing and deploying with Continuous Integration and Continuous Deployment methods - -By the end, we'll have the core of a [playable game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) -that's tested and deployed on every push to the `master` branch of the [codebase](https://gitlab.com/blitzgren/gitlab-game-demo). -This will also provide -boilerplate code for starting a browser-based game with the following components: - -- Written in [TypeScript](https://www.typescriptlang.org/) and [PhaserJs](https://phaser.io) -- Building, running, and testing with [Gulp](https://gulpjs.com) -- Unit tests with [Chai](https://www.chaijs.com) and [Mocha](https://mochajs.org/) -- CI/CD with GitLab -- Hosting the codebase on GitLab.com -- Hosting the game on AWS -- Deploying to AWS - -## Requirements and setup - -Please refer to my previous article [DevOps and Game Dev](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) to learn the foundational -development tools, running a Hello World-like game, and building this game using GitLab -CI/CD from every new push to master. The `master` branch for this game's [repository](https://gitlab.com/blitzgren/gitlab-game-demo) -contains a completed version with all configurations. If you would like to follow along -with this article, you can clone and work from the `devops-article` branch: - -```shell -git clone git@gitlab.com:blitzgren/gitlab-game-demo.git -git checkout devops-article -``` - -Next, we'll create a small subset of tests that exemplify most of the states I expect -this `Weapon` class to go through. To get started, create a folder called `lib/tests` -and add the following code to a new file `weaponTests.ts`: - -```typescript -import { expect } from 'chai'; -import { Weapon, BulletFactory } from '../lib/weapon'; - -describe('Weapon', () => { - var subject: Weapon; - var shotsFired: number = 0; - // Mocked bullet factory - var bulletFactory: BulletFactory = <BulletFactory>{ - generate: function(px, py, vx, vy, rot) { - shotsFired++; - } - }; - var parent: any = { x: 0, y: 0 }; - - beforeEach(() => { - shotsFired = 0; - subject = new Weapon(bulletFactory, parent, 0.25, 1); - }); - - it('should shoot if not in cooldown', () => { - subject.trigger(true); - subject.update(0.1); - expect(shotsFired).to.equal(1); - }); - - it('should not shoot during cooldown', () => { - subject.trigger(true); - subject.update(0.1); - subject.update(0.1); - expect(shotsFired).to.equal(1); - }); - - it('should shoot after cooldown ends', () => { - subject.trigger(true); - subject.update(0.1); - subject.update(0.3); // longer than timeout - expect(shotsFired).to.equal(2); - }); - - it('should not shoot if not triggered', () => { - subject.update(0.1); - subject.update(0.1); - expect(shotsFired).to.equal(0); - }); -}); -``` - -To build and run these tests using gulp, let's also add the following gulp functions -to the existing `gulpfile.js` file: - -```typescript -gulp.task('build-test', function () { - return gulp.src('src/tests/**/*.ts', { read: false }) - .pipe(tap(function (file) { - // replace file contents with browserify's bundle stream - file.contents = browserify(file.path, { debug: true }) - .plugin(tsify, { project: "./tsconfig.test.json" }) - .bundle(); - })) - .pipe(buffer()) - .pipe(sourcemaps.init({loadMaps: true}) ) - .pipe(gulp.dest('built/tests')); -}); - -gulp.task('run-test', function() { - gulp.src(['./built/tests/**/*.ts']).pipe(mocha()); -}); -``` - -We will start implementing the first part of our game and get these `Weapon` tests to pass. -The `Weapon` class will expose a method to trigger the generation of a bullet at a given -direction and speed. Later we will implement a `Player` class that ties together the user input -to trigger the weapon. In the `src/lib` folder create a `weapon.ts` file. We'll add two classes -to it: `Weapon` and `BulletFactory` which will encapsulate Phaser's **sprite** and -**group** objects, and the logic specific to our game. - -```typescript -export class Weapon { - private isTriggered: boolean = false; - private currentTimer: number = 0; - - constructor(private bulletFactory: BulletFactory, private parent: Phaser.Sprite, private cooldown: number, private bulletSpeed: number) { - } - - public trigger(on: boolean): void { - this.isTriggered = on; - } - - public update(delta: number): void { - this.currentTimer -= delta; - - if (this.isTriggered && this.currentTimer <= 0) { - this.shoot(); - } - } - - private shoot(): void { - // Reset timer - this.currentTimer = this.cooldown; - - // Get velocity direction from player rotation - var parentRotation = this.parent.rotation + Math.PI / 2; - var velx = Math.cos(parentRotation); - var vely = Math.sin(parentRotation); - - // Apply a small forward offset so bullet shoots from head of ship instead of the middle - var posx = this.parent.x - velx * 10 - var posy = this.parent.y - vely * 10; - - this.bulletFactory.generate(posx, posy, -velx * this.bulletSpeed, -vely * this.bulletSpeed, this.parent.rotation); - } -} - -export class BulletFactory { - - constructor(private bullets: Phaser.Group, private poolSize: number) { - // Set all the defaults for this BulletFactory's bullet object - this.bullets.enableBody = true; - this.bullets.physicsBodyType = Phaser.Physics.ARCADE; - this.bullets.createMultiple(30, 'bullet'); - this.bullets.setAll('anchor.x', 0.5); - this.bullets.setAll('anchor.y', 0.5); - this.bullets.setAll('outOfBoundsKill', true); - this.bullets.setAll('checkWorldBounds', true); - } - - public generate(posx: number, posy: number, velx: number, vely: number, rot: number): Phaser.Sprite { - // Pull a bullet from Phaser's Group pool - var bullet = this.bullets.getFirstExists(false); - - // Set the few unique properties about this bullet: rotation, position, and velocity - if (bullet) { - bullet.reset(posx, posy); - bullet.rotation = rot; - bullet.body.velocity.x = velx; - bullet.body.velocity.y = vely; - } - - return bullet; - } -} -``` - -Lastly, we'll redo our entry point, `game.ts`, to tie together both `Player` and `Weapon` objects -as well as add them to the update loop. Here is what the updated `game.ts` file looks like: - -```typescript -import { Player } from "./player"; -import { Weapon, BulletFactory } from "./weapon"; - -window.onload = function() { - var game = new Phaser.Game(800, 600, Phaser.AUTO, 'gameCanvas', { preload: preload, create: create, update: update }); - var player: Player; - var weapon: Weapon; - - // Import all assets prior to loading the game - function preload () { - game.load.image('player', 'assets/player.png'); - game.load.image('bullet', 'assets/bullet.png'); - } - - // Create all entities in the game, after Phaser loads - function create () { - // Create and position the player - var playerSprite = game.add.sprite(400, 550, 'player'); - playerSprite.anchor.setTo(0.5); - player = new Player(game.input, playerSprite, 150); - - var bulletFactory = new BulletFactory(game.add.group(), 30); - weapon = new Weapon(bulletFactory, player.sprite, 0.25, 1000); - - player.loadWeapon(weapon); - } - - // This function is called once every tick, default is 60fps - function update() { - var deltaSeconds = game.time.elapsedMS / 1000; // convert to seconds - player.update(deltaSeconds); - weapon.update(deltaSeconds); - } -} -``` - -Run `gulp serve` and you can run around and shoot. Wonderful! Let's update our CI -pipeline to include running the tests along with the existing build job. - -## Continuous Integration - -To ensure our changes don't break the build and all tests still pass, we use -Continuous Integration (CI) to run these checks automatically for every push. -Read through this article to understand [Continuous Integration, Continuous Delivery, and Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/), -and how these methods are leveraged by GitLab. -From the [last tutorial](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) we already have a `.gitlab-ci.yml` file set up for building our app from -every push. We need to set up a new CI job for testing, which GitLab CI/CD will run after the build job using our generated artifacts from gulp. - -Please read through the [documentation on CI/CD configuration](../../../ci/yaml/README.md) file to explore its contents and adjust it to your needs. - -### Build your game with GitLab CI/CD - -We need to update our build job to ensure tests get run as well. Add `gulp build-test` -to the end of the `script` array for the existing `build` job. After these commands run, -we know we will need to access everything in the `built` folder, given by GitLab CI/CD's `artifacts`. -We'll also cache `node_modules` to avoid having to do a full re-pull of those dependencies: -just pack them up in the cache. Here is the full `build` job: - -```yaml -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules - artifacts: - paths: - - built -``` - -### Test your game with GitLab CI/CD - -For testing locally, we simply run `gulp run-tests`, which requires gulp to be installed -globally like in the `build` job. We pull `node_modules` from the cache, so the `npm i` -command won't have to do much. In preparation for deployment, we know we will still need -the `built` folder in the artifacts, which will be brought over as default behavior from -the previous job. Lastly, by convention, we let GitLab CI/CD know this needs to be run after -the `build` job by giving it a `test` [stage](../../../ci/yaml/README.md#stages). -Following the YAML structure, the `test` job should look like this: - -```yaml -test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ -``` - -We have added unit tests for a `Weapon` class that shoots on a specified interval. -The `Player` class implements `Weapon` along with the ability to move around and shoot. Also, -we've added test artifacts and a test stage to our GitLab CI/CD pipeline using `.gitlab-ci.yml`, -allowing us to run our tests by every push. -Our entire `.gitlab-ci.yml` file should now look like this: - -```yaml -image: node:10 - -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ - -test: - stage: test - script: - - npm i gulp -g - - npm i - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ -``` - -### Run your CI/CD pipeline - -That's it! Add all your new files, commit, and push. For a reference of what our repository should -look like at this point, please refer to the [final commit related to this article on my sample repository](https://gitlab.com/blitzgren/gitlab-game-demo/commit/8b36ef0ecebcf569aeb251be4ee13743337fcfe2). -By applying both build and test stages, GitLab will run them sequentially at every push to -our repository. If all goes well you'll end up with a green check mark on each job for the pipeline: - -![Passing Pipeline](img/test_pipeline_pass.png) - -You can confirm that the tests passed by clicking on the `test` job to enter the full build logs. -Scroll to the bottom and observe, in all its passing glory: - -```shell -$ gulp run-test -[18:37:24] Using gulpfile /builds/blitzgren/gitlab-game-demo/gulpfile.js -[18:37:24] Starting 'run-test'... -[18:37:24] Finished 'run-test' after 21 ms - - - Weapon - ✓ should shoot if not in cooldown - ✓ should not shoot during cooldown - ✓ should shoot after cooldown ends - ✓ should not shoot if not triggered - - - 4 passing (18ms) - -Uploading artifacts... -built/: found 17 matching files -Uploading artifacts to coordinator... ok id=17095874 responseStatus=201 Created token=aaaaaaaa Job succeeded -``` - -## Continuous Deployment - -We have our codebase built and tested on every push. To complete the full pipeline with Continuous Deployment, -let's set up [free web hosting with AWS S3](https://aws.amazon.com/free/) and a job through which our build artifacts get -deployed. GitLab also has a free static site hosting service we can use, [GitLab Pages](https://about.gitlab.com/stages-devops-lifecycle/pages/), -however Dark Nova specifically uses other AWS tools that necessitates using `AWS S3`. -Read through this article that describes [deploying to both S3 and GitLab Pages](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) -and further delves into the principles of GitLab CI/CD than discussed in this article. - -### Set up S3 Bucket - -1. Log into your AWS account and go to [S3](https://console.aws.amazon.com/s3/home) -1. Click the **Create Bucket** link at the top -1. Enter a name of your choosing and click next -1. Keep the default **Properties** and click next -1. Click the **Manage group permissions** and allow **Read** for the **Everyone** group, click next -1. Create the bucket, and select it in your S3 bucket list -1. On the right side, click **Properties** and enable the **Static website hosting** category -1. Update the radio button to the **Use this bucket to host a website** selection. Fill in `index.html` and `error.html` respectively - -### Set up AWS Secrets - -We need to be able to deploy to AWS with our AWS account credentials, but we certainly -don't want to put secrets into source code. Luckily GitLab provides a solution for this -with [Variables](../../../ci/variables/README.md). This can get complicated -due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't -use root security credentials. Proper IAM credential management is beyond the scope of this -article, but AWS will remind you that using root credentials is unadvised and against their -best practices, as they should. Feel free to follow best practices and use a custom IAM user's -credentials, which will be the same two credentials (Key ID and Secret). It's a good idea to -fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html). We need to add these credentials to GitLab: - -1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential) -1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the ID and secret around, you'll need them later - - ![AWS Access Key Configuration](img/aws_config_window.png) - -1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar -1. Expand the **Variables** section - - ![GitLab Secret Configuration](img/gitlab_config.png) - -1. Add a key named `AWS_KEY_ID` and copy the key ID from Step 2 into the **Value** field -1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** field - -### Deploy your game with GitLab CI/CD - -To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on -the shared runner. The shared runner also needs to be able to authenticate with your AWS -account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab CI/CD gives us a way to pass the variables we -set up in the prior section using the `variables` portion of the `deploy` job. At the end, -we add directives to ensure deployment `only` happens on pushes to `master`. This way, every -single branch still runs through CI, and only merging (or committing directly) to master will -trigger the `deploy` job of our pipeline. Put these together to get the following: - -```yaml -deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master -``` - -Be sure to update the region and S3 URL in that last script command to fit your setup. -Our final configuration file `.gitlab-ci.yml` looks like: - -```yaml -image: node:10 - -build: - stage: build - script: - - npm i gulp -g - - npm i - - gulp - - gulp build-test - cache: - policy: push - paths: - - node_modules/ - artifacts: - paths: - - built/ - -test: - stage: test - script: - - npm i gulp -g - - gulp run-test - cache: - policy: pull - paths: - - node_modules/ - artifacts: - paths: - - built/ - -deploy: - stage: deploy - variables: - AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" - AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" - script: - - apt-get update - - apt-get install -y python3-dev python3-pip - - easy_install3 -U pip - - pip3 install --upgrade awscli - - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete - only: - - master -``` - -## Conclusion - -Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get -[TypeScript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](https://gulpjs.com/) and [Phaser](https://phaser.io) all playing -together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](https://www.darknova.io). -Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation, -and unit tests, all running and deployed at every push to master - with shockingly little code. -Errors can be easily debugged through GitLab build logs, and within minutes of a successful commit, -you can see the changes live on your game. - -Setting up Continuous Integration and Continuous Deployment from the start with Dark Nova enables -rapid but stable development. We can easily test changes in a separate [environment](../../environments/index.md), -or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing -and tedious, but having faith in a stable deployment with GitLab CI/CD allows -a lot of breathing room in quickly getting changes to players. - -## Further settings - -Here are some ideas to further investigate that can speed up or improve your pipeline: - -- [Yarn](https://yarnpkg.com) instead of npm -- Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) image that can pre-load dependencies and tools (like AWS CLI) -- Forward a [custom domain](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website -- Combine jobs if you find it unnecessary for a small project -- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/blog/2016/03/01/gitlab-runner-with-docker/) +<!-- This redirect file can be deleted after 2021-04-19. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md index 52db5740c34..42b7681bd10 100644 --- a/doc/ci/examples/test-phoenix-application.md +++ b/doc/ci/examples/test-phoenix-application.md @@ -3,3 +3,6 @@ redirect_to: '../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md' --- The content of this page was incorporated in [this document](../../ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index 5c499d6a855..a1a7de26cf2 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -1,81 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: '../README.md#contributed-examples' --- -# Test and deploy a Scala application to Heroku +This document was moved to [another location](../README.md#contributed-examples). -This example demonstrates the integration of GitLab CI/CD with Scala -applications using SBT. You can view or fork the [example project](https://gitlab.com/gitlab-examples/scala-sbt) -and view the logs of its past [CI jobs](https://gitlab.com/gitlab-examples/scala-sbt/-/jobs?scope=finished). - -## Add `.gitlab-ci.yml` file to project - -The following `.gitlab-ci.yml` should be added in the root of your -repository to trigger CI: - -``` yaml -image: openjdk:8 - -stages: - - test - - deploy - -before_script: - - apt-get update -y - - apt-get install apt-transport-https -y - ## Install SBT - - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 - - apt-get update -y - - apt-get install sbt -y - - sbt sbtVersion - -test: - stage: test - script: - - sbt clean coverage test coverageReport - -deploy: - stage: deploy - script: - - apt-get update -yq - - apt-get install rubygems ruby-dev -y - - gem install dpl - - dpl --provider=heroku --app=gitlab-play-sample-app --api-key=$HEROKU_API_KEY -``` - -In the above configuration: - -- The `before_script` installs [SBT](https://www.scala-sbt.org/) and - displays the version that is being used. -- The `test` stage executes SBT to compile and test the project. - - [sbt-scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT - plugin to measure test coverage. -- The `deploy` stage automatically deploys the project to Heroku using dpl. - -You can use other versions of Scala and SBT by defining them in -`build.sbt`. - -## Display test coverage in job - -Add the `Coverage was \[\d+.\d+\%\]` regular expression in the -**Settings > Pipelines > Coverage report** project setting to -retrieve the [test coverage](../pipelines/settings.md#test-coverage-report-badge) -rate from the build trace and have it displayed with your jobs. - -**Pipelines** must be enabled for this option to appear. - -## Heroku application - -A Heroku application is required. You can create one through the -[Dashboard](https://dashboard.heroku.com/). Substitute `gitlab-play-sample-app` -in the `.gitlab-ci.yml` file with your application's name. - -## Heroku API key - -You can look up your Heroku API key in your -[account](https://dashboard.heroku.com/account). Add a [protected variable](../variables/README.md#protect-a-custom-variable) with -this value in **Project ➔ Variables** with key `HEROKU_API_KEY`. +<!-- This redirect file can be deleted after 2021-04-18. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 96bdad0cf82..56594103362 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -4450,22 +4450,31 @@ You can use [YAML anchors](#anchors) with [script](#script), [`before_script`](# and [`after_script`](#after_script) to use predefined commands in multiple jobs: ```yaml -.some-script: &some-script - - echo "Execute this script in `before_script` sections" - .some-script-before: &some-script-before - - echo "Execute this script in `script` sections" + - echo "Execute this script first" + +.some-script: &some-script + - echo "Execute this script second" + - echo "Execute this script too" .some-script-after: &some-script-after - - echo "Execute this script in `after_script` sections" + - echo "Execute this script last" -job_name: +job1: before_script: - *some-script-before script: - *some-script + - echo "Execute something, for this job only" after_script: - *some-script-after + +job2: + script: + - *some-script-before + - *some-script + - echo "Execute something else, for this job only" + - *some-script-after ``` #### YAML anchors for variables diff --git a/doc/development/README.md b/doc/development/README.md index 0d3c1b3cbe9..087b2b2e7eb 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -151,6 +151,7 @@ In these cases, use the following workflow: ## Backend guides +- [Directory structure](directory_structure.md) - [GitLab utilities](utilities.md) - [Issuable-like Rails models](issuable-like-models.md) - [Logging](logging.md) diff --git a/doc/development/directory_structure.md b/doc/development/directory_structure.md new file mode 100644 index 00000000000..c2329feb941 --- /dev/null +++ b/doc/development/directory_structure.md @@ -0,0 +1,36 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Backend directory structure + +## Use namespaces to define bounded contexts + +A healthy application is divided into macro and sub components that represent the contexts at play, +whether they are related to business domain or infrastructure code. + +As GitLab code has so many features and components it's hard to see what contexts are involved. +We should expect any class to be defined inside a module/namespace that represents the contexts where it operates. + +When we namespace classes inside their domain: + +- Similar terminology becomes unambiguous as the domain clarifies the meaning: + For example, `MergeRequests::Diff` and `Notes::Diff`. +- Top-level namespaces could be associated to one or more groups identified as domain experts. +- We can better identify the interactions and coupling between components. + For example, several classes inside `MergeRequests::` domain interact more with `Ci::` + domain and less with `ImportExport::`. + +```ruby +# bad +class MyClass +end + +# good +module MyDomain + class MyClass + end +end +``` diff --git a/doc/development/usage_ping.md b/doc/development/usage_ping.md index 10c3de2f0a1..f3203379994 100644 --- a/doc/development/usage_ping.md +++ b/doc/development/usage_ping.md @@ -122,6 +122,60 @@ sequenceDiagram the hostname is `version.gitlab.com`, the protocol is `TCP`, and the port number is `443`, the required URL is <https://version.gitlab.com/>. +## Usage Ping Metric Life cycle + +### 1. New metrics addition + +Please follow the [Implementing Usage Ping](#implementing-usage-ping) guide. + +### 2. Existing metric change + +Because we do not control when customers update their self-managed instances of GitLab, +we **STRONGLY DISCOURAGE** changes to the logic used to calculate any metric. +Any such changes lead to inconsistent reports from multiple GitLab instances. +If there is a problem with an existing metric, it's best to deprecate the existing metric, +and use it, side by side, with the desired new metric. + +Example: +Consider following change. Before GitLab 12.6, the `example_metric` was implemented as: + +```ruby +{ + ... + example_metric: distinct_count(Project, :creator_id) +} +``` + +For GitLab 12.6, the metric was changed to filter out archived projects: + +```ruby +{ + ... + example_metric: distinct_count(Project.non_archived, :creator_id) +} +``` + +In this scenario all instances running up to GitLab 12.5 continue to report `example_metric`, +including all archived projects, while all instances running GitLab 12.6 and higher filters +out such projects. As Usage Ping data is collected from all reporting instances, the +resulting dataset includes mixed data, which distorts any following business analysis. + +The correct approach is to add a new metric for GitLab 12.6 release with updated logic: + +```ruby +{ + ... + example_metric_without_archived: distinct_count(Project.non_archived, :creator_id) +} +``` + +and update existing business analysis artefacts to use `example_metric_without_archived` instead of `example_metric` + +### 3. Metrics deprecation and removal + +The process for deprecating and removing metrics is currently under development. For +more information, see the following [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/284637). + ## Implementing Usage Ping Usage Ping consists of two kinds of data, counters and observations. Counters track how often a certain event @@ -446,6 +500,8 @@ Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd) and [PF Aggregation on a `daily` basis does not pull more fine grained data. - `feature_flag`: optional. For details, see our [GitLab internal Feature flags](feature_flags/) documentation. +Use one of the following methods to track events: + 1. Track event in controller using `RedisTracking` module with `track_redis_hll_event(*controller_actions, name:, feature:, feature_default_enabled: false)`. Arguments: @@ -562,21 +618,6 @@ Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd) and [PF api.trackRedisHllUserEvent('my_already_defined_event_name'), ``` -1. Track event using base module `Gitlab::UsageDataCounters::HLLRedisCounter.track_event(event_name, values:)`. - - Arguments: - - - `event_name`: event name. - - `values`: One value or array of values we count. For example: user_id, visitor_id, user_ids. - -1. Track event on context level using base module `Gitlab::UsageDataCounters::HLLRedisCounter.track_event_in_context(event_name, values:, context:)`. - - Arguments: - - - `event_name`: event name. - - `values`: values we count. For example: user_id, visitor_id. - - `context`: context value. Allowed values are `default`, `free`, `bronze`, `silver`, `gold`, `starter`, `premium`, `ultimate` - 1. Get event data using `Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names:, start_date:, end_date:, context: '')`. Arguments: diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index e0f5a400977..58730cd6682 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -186,10 +186,11 @@ published to the GitLab Package Registry. ## Authenticate to the Package Registry with Maven -To authenticate to the Package Registry, you need either a personal access token or deploy token. +To authenticate to the Package Registry, you need one of the following: -- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), set the scope to `api`. -- If you use a [deploy token](../../project/deploy_tokens/index.md), set the scope to `read_package_registry`, `write_package_registry`, or both. +- A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. +- A [deploy token](../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. +- A [CI_JOB_TOKEN](#authenticate-with-a-ci-job-token-in-maven). ### Authenticate with a personal access token in Maven @@ -354,12 +355,13 @@ repositories { To use the GitLab endpoint for Maven packages, choose an option: -- **Project-level**: Use when you have few Maven packages and they are not in - the same GitLab group. -- **Group-level**: Use when you have many Maven packages in the same GitLab - group. -- **Instance-level**: Use when you have many Maven packages in different - GitLab groups or in their own namespace. +- **Project-level**: To publish Maven packages to a project, use a project-level endpoint. + To install Maven packages, use a project-level endpoint when you have few Maven packages + and they are not in the same GitLab group. +- **Group-level**: Use a group-level endpoint when you want to install packages from + many different projects in the same GitLab group. +- **Instance-level**: Use an instance-level endpoint when you want to install many + packages from different GitLab groups or in their own namespace. The option you choose determines the settings you add to your `pom.xml` file. @@ -533,7 +535,7 @@ repositories { After you have set up the [remote and authentication](#authenticate-to-the-package-registry-with-maven) and [configured your project](#use-the-gitlab-endpoint-for-maven-packages), -publish a Maven artifact from your project. +publish a Maven package to your project. ### Publish by using Maven @@ -604,11 +606,20 @@ To publish a package by using Gradle: Now navigate to your project's **Packages & Registries** page and view the published artifacts. +### Publishing a package with the same name or version + +When you publish a package with the same name or version as an existing package, +the existing package is overwritten. + ## Install a package To install a package from the GitLab Package Registry, you must configure the [remote and authenticate](#authenticate-to-the-package-registry-with-maven). -When this is completed, there are two ways to install a package. +When this is completed, you can install a package from a project, +group, or namespace. + +If multiple packages have the same name and version, when you install +a package, the most recently-published package is retrieved. ### Use Maven with `mvn install` diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 74311eefd83..d07905c0ead 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -119,7 +119,7 @@ and modify them if you have the necessary [permissions](../../permissions.md). > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 13.3. Assignees in the sidebar are updated in real time. This feature is **disabled by default**. -To enable, you need to enable [ActionCable in-app mode](https://docs.gitlab.com/omnibus/settings/actioncable.html). +To enable it, you need to enable [ActionCable in-app mode](https://docs.gitlab.com/omnibus/settings/actioncable.html). ### Issues List @@ -137,7 +137,22 @@ view, you can also make certain changes [in bulk](../bulk_editing.md) to the dis For more information, see the [Issue Data and Actions](issue_data_and_actions.md) page for a rundown of all the fields and information in an issue. -You can sort a list of issues in several ways, for example by issue creation date, milestone due date. For more information, see the [Sorting and Ordering Issue Lists](sorting_issue_lists.md) page. +You can sort a list of issues in several ways, for example by issue creation date, milestone due date. +For more information, see the [Sorting and ordering issue lists](sorting_issue_lists.md) page. + +#### Cached issue count + +> - [Introduced]([link-to-issue](https://gitlab.com/gitlab-org/gitlab/-/issues/243753)) in GitLab 13.9. +> - It's [deployed behind a feature flag](../../feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use this feature in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cached-issue-count) **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +In a group, the sidebar displays the total count of open issues and this value is cached if higher +than 1000. The cached value is rounded to thousands (or millions) and updated every 24 hours. ### Issue boards @@ -226,3 +241,22 @@ You can then see issue statuses in the [issue list](#issues-list) and the - [Issues API](../../../api/issues.md) - Configure an [external issue tracker](../../../integration/external-issue-tracker.md) such as Jira, Redmine, Bugzilla, or EWM. + +## Enable or disable cached issue count **(CORE ONLY)** + +Cached issue count in the left sidebar is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:cached_sidebar_open_issues_count) +``` + +To disable it: + +```ruby +Feature.disable(:cached_sidebar_open_issues_count) +``` |