summaryrefslogtreecommitdiff
path: root/doc/ci
diff options
context:
space:
mode:
Diffstat (limited to 'doc/ci')
-rw-r--r--doc/ci/autodeploy/index.md3
-rw-r--r--doc/ci/environments.md8
-rw-r--r--doc/ci/examples/README.md6
-rw-r--r--doc/ci/examples/deployment/composer-npm-deploy.md156
-rw-r--r--doc/ci/examples/php.md6
-rw-r--r--doc/ci/yaml/README.md40
6 files changed, 212 insertions, 7 deletions
diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md
index 630207ffa09..c4c4d95b68a 100644
--- a/doc/ci/autodeploy/index.md
+++ b/doc/ci/autodeploy/index.md
@@ -1,6 +1,6 @@
# Auto deploy
-> [Introduced][mr-8135] in GitLab 8.15.
+> [Introduced][mr-8135] in GitLab 8.15. Currently requires a [Public project][project-settings].
Auto deploy is an easy way to configure GitLab CI for the deployment of your
application. GitLab Community maintains a list of `.gitlab-ci.yml`
@@ -33,6 +33,7 @@ enable [Kubernetes service][kubernetes-service].
created automatically for you.
[mr-8135]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8135
+[project-settings]: https://docs.gitlab.com/ce/public_access/public_access.html
[project-services]: ../../project_services/project_services.md
[auto-deploy-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml/tree/master/autodeploy
[kubernetes-service]: ../../project_services/kubernetes.md
diff --git a/doc/ci/environments.md b/doc/ci/environments.md
index 98cd29c9567..ef04c537367 100644
--- a/doc/ci/environments.md
+++ b/doc/ci/environments.md
@@ -297,7 +297,7 @@ deploy_review:
- echo "Deploy a review app"
environment:
name: review/$CI_BUILD_REF_NAME
- url: https://$CI_BUILD_REF_SLUG.review.example.com
+ url: https://$CI_ENVIRONMENT_SLUG.example.com
only:
- branches
except:
@@ -318,15 +318,15 @@ also contain `/`, or other characters that would be invalid in a domain name or
URL, we use `$CI_ENVIRONMENT_SLUG` in the `environment:url` so that the
environment can get a specific and distinct URL for each branch. In this case,
given a `$CI_BUILD_REF_NAME` of `100-Do-The-Thing`, the URL will be something
-like `https://review-100-do-the-4f99a2.example.com`. Again, the way you set up
+like `https://100-do-the-4f99a2.example.com`. Again, the way you set up
the web server to serve these requests is based on your setup.
You could also use `$CI_BUILD_REF_SLUG` in `environment:url`, e.g.:
-`https://$CI_BUILD_REF_SLUG.review.example.com`. We use `$CI_ENVIRONMENT_SLUG`
+`https://$CI_BUILD_REF_SLUG.example.com`. We use `$CI_ENVIRONMENT_SLUG`
here because it is guaranteed to be unique, but if you're using a workflow like
[GitLab Flow][gitlab-flow], collisions are very unlikely, and you may prefer
environment names to be more closely based on the branch name - the example
-above would give you an URL like `https://100-do-the-thing.review.example.com`
+above would give you an URL like `https://100-do-the-thing.example.com`
Last but not least, we tell the job to run [`only`][only] on branches
[`except`][only] master.
diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md
index ffc310ec8c7..5377bf9ee80 100644
--- a/doc/ci/examples/README.md
+++ b/doc/ci/examples/README.md
@@ -14,6 +14,12 @@ Apart from those, here is an collection of tutorials and guides on setting up yo
- [Test a Phoenix application](test-phoenix-application.md)
- [Using `dpl` as deployment tool](deployment/README.md)
- [Example project that shows how to use Review Apps](https://gitlab.com/gitlab-examples/review-apps-nginx/)
+- [Run PHP Composer & NPM scripts then deploy them to a staging server](deployment/composer-npm-deploy.md)
+- Help your favorite programming language and GitLab by sending a merge request
+ with a guide for that language.
+
+## Outside the documentation
+
- [Blog post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/)
- [Repositories with examples for various languages](https://gitlab.com/groups/gitlab-examples)
- [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md
new file mode 100644
index 00000000000..5334a73e1f5
--- /dev/null
+++ b/doc/ci/examples/deployment/composer-npm-deploy.md
@@ -0,0 +1,156 @@
+## Running Composer and NPM scripts with deployment via SCP
+
+This guide covers the building dependencies of a PHP project while compiling assets via an NPM script.
+
+While is possible to create your own image with custom PHP and Node JS versions, for brevity, we will use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and NodeJS installed.
+
+
+```yaml
+image: tetraweb/php
+```
+
+The next step is to install zip/unzip packages and make composer available. We will place these in the `before_script` section:
+
+```yaml
+before_script:
+ - apt-get update
+ - apt-get install zip unzip
+ - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
+ - php composer-setup.php
+ - php -r "unlink('composer-setup.php');"
+```
+
+This will make sure we have all requirements ready. Next, we want to run `composer update` to fetch all PHP dependencies and `npm install` to load node packages, then run the `npm` script. We need to append them into `before_script` section:
+
+```yaml
+before_script:
+ # ...
+ - php composer.phar update
+ - npm install
+ - npm run deploy
+```
+
+In this particular case, the `npm deploy` script is a Gulp script that does the following:
+
+1. Compile CSS & JS
+2. Create sprites
+3. Copy various assets (images, fonts) around
+4. Replace some strings
+
+All these operations will put all files into a `build` folder, which is ready to be deployed to a live server.
+
+### How to transfer files to a live server?
+
+You have multiple options: rsync, scp, sftp and so on. For now, we will use scp.
+
+To make this work, you need to add a GitLab Secret Variable (accessible on _gitlab.example/your-project-name/variables_). That variable will be called `STAGING_PRIVATE_KEY` and it's the **private** ssh key of your server.
+
+#### Security tip
+
+Create a user that has access **only** to the folder that needs to be updated!
+
+After you create that variable, you need to make sure that key will be added to the docker container on run:
+
+```yaml
+before_script:
+ # - ....
+ - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
+ - mkdir -p ~/.ssh
+ - eval $(ssh-agent -s)
+ - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
+```
+
+In order, this means that:
+
+1. We check if the `ssh-agent` is available and we install it if it's not;
+2. We create the `~/.ssh` folder;
+3. We make sure we're running bash;
+4. We disable host checking (we don't ask for user accept when we first connect to a server; and since every build will equal a first connect, we kind of need this)
+
+And this is basically all you need in the `before_script` section.
+
+## How to deploy things?
+
+As we stated above, we need to deploy the `build` folder from the docker image to our server. To do so, we create a new job:
+
+```yaml
+stage_deploy:
+ artifacts:
+ paths:
+ - build/
+ only:
+ - dev
+ script:
+ - ssh-add <(echo "$STAGING_PRIVATE_KEY")
+ - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
+ - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
+ - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
+ - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"
+```
+
+### What's going on here?
+
+1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want)
+2. `ssh-add ...` we will add that private key you added on the web UI to the docker container
+3. We will connect via `ssh` and create a new `_tmp` folder
+4. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder
+5. We will connect again to `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`.
+6. We connect to ssh and remove the `_old` folder
+
+What's the deal with the artifacts? We just tell GitLab CI to keep the `build` directory (later on, you can download that as needed).
+
+#### Why we do it this way?
+
+If you're using this only for stage server, you could do this in two steps:
+
+```yaml
+- ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/live/*"
+- scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/live
+```
+
+The problem is that there will be a small period of time when you won't have the app on your server.
+
+So we use so many steps because we want to make sure that at any given time we have a functional app in place.
+
+## Where to go next?
+
+Since this was a WordPress project, I gave real life code snippets. Some ideas you can pursuit:
+
+- Having a slightly different script for `master` branch will allow you to deploy to a production server from that branch and to a stage server from any other branches;
+- Instead of pushing it live, you can push it to WordPress official repo (with creating a SVN commit & stuff);
+- You could generate i18n text domains on the fly.
+
+---
+
+Our final `.gitlab-ci.yml` will look like this:
+
+```yaml
+image: tetraweb/php
+
+before_script:
+ - apt-get update
+ - apt-get install zip unzip
+ - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
+ - php composer-setup.php
+ - php -r "unlink('composer-setup.php');"
+ - php composer.phar update
+ - npm install
+ - npm run deploy
+ - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
+ - mkdir -p ~/.ssh
+ - eval $(ssh-agent -s)
+ - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
+
+stage_deploy:
+ artifacts:
+ paths:
+ - build/
+ only:
+ - dev
+ script:
+ - ssh-add <(echo "$STAGING_PRIVATE_KEY")
+ - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
+ - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
+ - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
+ - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"
+``` \ No newline at end of file
diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md
index 82ffb841729..5eeec92d976 100644
--- a/doc/ci/examples/php.md
+++ b/doc/ci/examples/php.md
@@ -235,7 +235,11 @@ cache:
before_script:
# Install composer dependencies
-- curl --silent --show-error https://getcomposer.org/installer | php
+- wget https://composer.github.io/installer.sig -O - -q | tr -d '\n' > installer.sig
+- php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
+- php -r "if (hash_file('SHA384', 'composer-setup.php') === file_get_contents('installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
+- php composer-setup.php
+- php -r "unlink('composer-setup.php'); unlink('installer.sig');"
- php composer.phar install
...
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index 75a0897eb15..06810898cfe 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -76,6 +76,7 @@ There are a few reserved `keywords` that **cannot** be used as job names:
| after_script | no | Define commands that run after each job's script |
| variables | no | Define build variables |
| cache | no | Define list of files that should be cached between subsequent runs |
+| coverage | no | Define coverage settings for all jobs |
### image and services
@@ -86,7 +87,7 @@ used for time of the build. The configuration of this feature is covered in
### before_script
`before_script` is used to define the command that should be run before all
-builds, including deploy builds. This can be an array or a multi-line string.
+builds, including deploy builds, but after the restoration of artifacts. This can be an array or a multi-line string.
### after_script
@@ -278,6 +279,23 @@ cache:
untracked: true
```
+### coverage
+
+`coverage` allows you to configure how coverage will be filtered out from the
+build outputs. Setting this up globally will make all the jobs to use this
+setting for output filtering and extracting the coverage information from your
+builds.
+
+Regular expressions are the only valid kind of value expected here. So, using
+surrounding `/` is mandatory in order to consistently and explicitly represent
+a regular expression string. You must escape special characters if you want to
+match them literally.
+
+A simple example:
+```yaml
+coverage: /\(\d+\.\d+\) covered\./
+```
+
## Jobs
`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job
@@ -319,6 +337,7 @@ job_name:
| before_script | no | Override a set of commands that are executed before build |
| after_script | no | Override a set of commands that are executed after build |
| environment | no | Defines a name of environment to which deployment is done by this build |
+| coverage | no | Define coverage settings for a given job |
### script
@@ -993,6 +1012,25 @@ job:
- execute this after my script
```
+### job coverage
+
+This entry is pretty much the same as described in the global context in
+[`coverage`](#coverage). The only difference is that, by setting it inside
+the job level, whatever is set in there will take precedence over what has
+been defined in the global level. A quick example of one overriding the
+other would be:
+
+```yaml
+coverage: /\(\d+\.\d+\) covered\./
+
+job1:
+ coverage: /Code coverage: \d+\.\d+/
+```
+
+In the example above, considering the context of the job `job1`, the coverage
+regex that would be used is `/Code coverage: \d+\.\d+/` instead of
+`/\(\d+\.\d+\) covered\./`.
+
## Git Strategy
> Introduced in GitLab 8.9 as an experimental feature. May change or be removed
View Lorry Controller Status