From 712d17684b2b9a8664cdff685c44fa59ea6fabbc Mon Sep 17 00:00:00 2001 From: Douwe Maan Date: Tue, 13 Oct 2015 13:07:59 +0200 Subject: Make Reply by email easier to configure --- doc/incoming_email/README.md | 299 ++++++++++++++++++------------------------- 1 file changed, 124 insertions(+), 175 deletions(-) (limited to 'doc') diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md index aafa2345fab..86d205ba7a5 100644 --- a/doc/incoming_email/README.md +++ b/doc/incoming_email/README.md @@ -4,9 +4,9 @@ GitLab can be set up to allow users to comment on issues and merge requests by r ## Get a mailbox -Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises. +Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises. -If you want to use Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255). +If you want to use Gmail / Google Apps with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255). To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md). @@ -14,30 +14,62 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these ### Omnibus package installations -1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature, enter the email address including a placeholder for the `key` that references the item being replied to and fill in the details for your specific IMAP server and email account: +1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature and fill in the details for your specific IMAP server and email account: ```ruby - # Postfix mail server, assumes mailbox incoming@gitlab.example.com + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com gitlab_rails['incoming_email_enabled'] = true + + # The email address including a placeholder for the key that references the item being replied to. + # The `%{key}` placeholder is added after the user part, before the `@`. gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com" - gitlab_rails['incoming_email_host'] = "gitlab.example.com" # IMAP server host - gitlab_rails['incoming_email_port'] = 143 # IMAP server port - gitlab_rails['incoming_email_ssl'] = false # Whether the IMAP server uses SSL - gitlab_rails['incoming_email_email'] = "incoming" # Email account username. Usually the full email address. - gitlab_rails['incoming_email_password'] = "[REDACTED]" # Email account password - gitlab_rails['incoming_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox". + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "incoming" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "gitlab.example.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 143 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = false + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" ``` ```ruby - # Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com gitlab_rails['incoming_email_enabled'] = true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" - gitlab_rails['incoming_email_host'] = "imap.gmail.com" # IMAP server host - gitlab_rails['incoming_email_port'] = 993 # IMAP server port - gitlab_rails['incoming_email_ssl'] = true # Whether the IMAP server uses SSL - gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" # Email account username. Usually the full email address. - gitlab_rails['incoming_email_password'] = "[REDACTED]" # Email account password - gitlab_rails['incoming_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox". + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "imap.gmail.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 993 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = true + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" ``` As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. @@ -64,229 +96,146 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these cd /home/git/gitlab ``` -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key` that references the item being replied to: +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: ```sh sudo editor config/gitlab.yml ``` ```yaml - # Postfix mail server, assumes mailbox incoming@gitlab.example.com + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com incoming_email: enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. address: "incoming+%{key}@gitlab.example.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "incoming" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "gitlab.example.com" + # IMAP server port + port: 143 + # Whether the IMAP server uses SSL + ssl: false + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` ```yaml - # Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com incoming_email: enabled: true - address: "gitlab-incoming+%{key}@gmail.com" - ``` - - As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. -2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`: + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + address: "gitlab-incoming+%{key}@gmail.com" - ```sh - sudo cp config/mail_room.yml.example config/mail_room.yml - ``` + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" -3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account: + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false - ```sh - sudo editor config/mail_room.yml - ``` - - ```yaml - # Postfix mail server - :mailboxes: - - - # IMAP server host - :host: "gitlab.example.com" - # IMAP server port - :port: 143 - # Whether the IMAP server uses SSL - :ssl: false - # Whether the IMAP server uses StartTLS - :start_tls: false - # Email account username. Usually the full email address. - :email: "incoming" - # Email account password - :password: "[REDACTED]" - - # The name of the mailbox where incoming mail will end up. Usually "inbox". - :name: "inbox" - - # Always "sidekiq". - :delivery_method: sidekiq - # Always true. - :delete_after_delivery: true - :delivery_options: - # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "resque:gitlab". - :namespace: resque:gitlab - # Always "incoming_email". - :queue: incoming_email - # Always "EmailReceiverWorker" - :worker: EmailReceiverWorker - - # Always "redis". - :arbitration_method: redis - :arbitration_options: - # The URL to the Redis server. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "mail_room:gitlab". - :namespace: mail_room:gitlab + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` - ```yaml - # Gmail / Google Apps - :mailboxes: - - - # IMAP server host - :host: "imap.gmail.com" - # IMAP server port - :port: 993 - # Whether the IMAP server uses SSL - :ssl: true - # Whether the IMAP server uses StartTLS - :start_tls: false - # Email account username. Usually the full email address. - :email: "gitlab-incoming@gmail.com" - # Email account password - :password: "[REDACTED]" - - # The name of the mailbox where incoming mail will end up. Usually "inbox". - :name: "inbox" - - # Always "sidekiq". - :delivery_method: sidekiq - # Always true. - :delete_after_delivery: true - :delivery_options: - # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "resque:gitlab". - :namespace: resque:gitlab - # Always "incoming_email". - :queue: incoming_email - # Always "EmailReceiverWorker" - :worker: EmailReceiverWorker - - # Always "redis". - :arbitration_method: redis - :arbitration_options: - # The URL to the Redis server. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "mail_room:gitlab". - :namespace: mail_room:gitlab - ``` + As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. -5. Edit the init script configuration at `/etc/default/gitlab` to enable `mail_room`: +1. Enable `mail_room` in the init script at `/etc/default/gitlab`: ```sh sudo mkdir -p /etc/default echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab ``` -6. Restart GitLab: +1. Restart GitLab: ```sh sudo service gitlab restart ``` -7. Verify that everything is configured correctly: +1. Verify that everything is configured correctly: ```sh sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production ``` -8. Reply by email should now be working. +1. Reply by email should now be working. ### Development 1. Go to the GitLab installation directory. -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key` that references the item being replied to: +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: ```yaml - # Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com incoming_email: enabled: true + + # The email address including a placeholder for the key that references the item being replied to. + # The `%{key}` placeholder is added after the user part, before the `@`. address: "gitlab-incoming+%{key}@gmail.com" - ``` - As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" -2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`: + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false - ```sh - sudo cp config/mail_room.yml.example config/mail_room.yml + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` -3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account: - - ```yaml - # Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com - :mailboxes: - - - # IMAP server host - :host: "imap.gmail.com" - # IMAP server port - :port: 993 - # Whether the IMAP server uses SSL - :ssl: true - # Whether the IMAP server uses StartTLS - :start_tls: false - # Email account username. Usually the full email address. - :email: "gitlab-incoming@gmail.com" - # Email account password - :password: "[REDACTED]" - - # The name of the mailbox where incoming mail will end up. Usually "inbox". - :name: "inbox" - - # Always "sidekiq". - :delivery_method: sidekiq - # Always true. - :delete_after_delivery: true - :delivery_options: - # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "resque:gitlab". - :namespace: resque:gitlab - # Always "incoming_email". - :queue: incoming_email - # Always "EmailReceiverWorker" - :worker: EmailReceiverWorker - - # Always "redis". - :arbitration_method: redis - :arbitration_options: - # The URL to the Redis server. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "mail_room:gitlab". - :namespace: mail_room:gitlab - ``` + As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. -4. Uncomment the `mail_room` line in your `Procfile`: +1. Uncomment the `mail_room` line in your `Procfile`: ```yaml mail_room: bundle exec mail_room -q -c config/mail_room.yml ``` -6. Restart GitLab: +1. Restart GitLab: ```sh bundle exec foreman start ``` -7. Verify that everything is configured correctly: +1. Verify that everything is configured correctly: ```sh bundle exec rake gitlab:incoming_email:check RAILS_ENV=development ``` -8. Reply by email should now be working. +1. Reply by email should now be working. -- cgit v1.2.1 From 97e02f4bc99d391c5e1dc0ce8e317be105e6d2b0 Mon Sep 17 00:00:00 2001 From: Yorick Peterse Date: Thu, 15 Oct 2015 12:15:18 +0200 Subject: Added documentation on the various profiling tools --- doc/development/profiling.md | 56 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 doc/development/profiling.md (limited to 'doc') diff --git a/doc/development/profiling.md b/doc/development/profiling.md new file mode 100644 index 00000000000..80c86ef921e --- /dev/null +++ b/doc/development/profiling.md @@ -0,0 +1,56 @@ +# Profiling + +To make it easier to track down performance problems GitLab comes with a set of +profiling tools, some of these are available by default while others need to be +explicitly enabled. + +## rack-mini-profiler + +This Gem is enabled by default in development only. It allows you to see the +timings of the various components that made up a web request (e.g. the SQL +queries executed and their execution timings). + +## Bullet + +Bullet is a Gem that can be used to track down N+1 query problems. Because +Bullet adds quite a bit of logging noise it's disabled by default. To enable +Bullet, set the environment variable `ENABLE_BULLET` to a non-empty value before +starting GitLab. For example: + + ENABLE_BULLET=true bundle exec rails s + +Bullet will log query problems to both the Rails log as well as the Chrome +console. + +## ActiveRecord Query Trace + +This Gem adds backtraces for every ActiveRecord query in the Rails console. This +can be useful to track down where a query was executed. Because this Gem adds +quite a bit of noise (5-10 extra lines per ActiveRecord query) it's disabled by +default. To use this Gem you'll need to set `ENABLE_QUERY_TRACE` to a non empty +file before starting GitLab. For example: + + ENABLE_QUERY_TRACE=true bundle exec rails s + +## rack-lineprof + +This is a Gem that can trace the execution time of code on a per line basis. +Because this Gem can add quite a bit of overhead it's disabled by default. To +enable it, set the environment variable `ENABLE_LINEPROF` to a non-empty value. +For example: + + ENABLE_LINEPROF=true bundle exec rails s + +Once enabled you'll need to add a query string parameter to a request to +actually profile code execution. The name of the parameter is `lineprof` and +should be set to a regular expression (minus the starting/ending slash) used to +select what files to profile. To profile all files containing "foo" somewhere in +the path you'd use the following parameter: + + ?lineprof=foo + +Or when filtering for files containing "foo" and "bar" in their path: + + ?lineprof=foo|bar + +Once set the profiling output will be displayed in your terminal. -- cgit v1.2.1 From f7e0357840bb0780ca3ba997aa1e2f4c045f08c7 Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Thu, 15 Oct 2015 13:22:28 +0200 Subject: Track compatible gitlab-git-http-server version --- doc/install/installation.md | 1 + doc/update/7.14-to-8.0.md | 1 + 2 files changed, 2 insertions(+) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 3c62b11988e..acc4e505971 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -325,6 +325,7 @@ GitLab Shell is an SSH access and repository management software developed speci cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git cd gitlab-git-http-server + sudo -u git -H git checkout 0.2.14 sudo -u git -H make ### Initialize Database and Activate Advanced Features diff --git a/doc/update/7.14-to-8.0.md b/doc/update/7.14-to-8.0.md index 7ad4935e839..305017b7048 100644 --- a/doc/update/7.14-to-8.0.md +++ b/doc/update/7.14-to-8.0.md @@ -84,6 +84,7 @@ Now we download `gitlab-git-http-server` and install it in `/home/git/gitlab-git cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git cd gitlab-git-http-server +sudo -u git -H git checkout 0.2.14 sudo -u git -H make ``` -- cgit v1.2.1 From 999051bc91f02292e28582ce538f0d147c7b665e Mon Sep 17 00:00:00 2001 From: Jacob Vosmaer Date: Thu, 15 Oct 2015 14:59:06 +0200 Subject: Use gitlab-git-http-server 0.3.0 --- doc/install/installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index acc4e505971..e09890dc296 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -325,7 +325,7 @@ GitLab Shell is an SSH access and repository management software developed speci cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git cd gitlab-git-http-server - sudo -u git -H git checkout 0.2.14 + sudo -u git -H git checkout 0.3.0 sudo -u git -H make ### Initialize Database and Activate Advanced Features -- cgit v1.2.1 From 6348f654cdd76b062c18ae1bce3eecf0b6dd0c6c Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 15 Oct 2015 15:59:06 -0400 Subject: Update Installation doc for 8-1-stable --- doc/install/installation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 3c62b11988e..b154d280471 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -211,9 +211,9 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-0-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-1-stable gitlab -**Note:** You can change `8-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `8-1-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It -- cgit v1.2.1 From facf8826daae0c5bad9854bb903101dadc04acb8 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 15 Oct 2015 16:06:34 -0400 Subject: Add 8.0-to-8.1 update guide --- doc/update/8.0-to-8.1.md | 228 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 228 insertions(+) create mode 100644 doc/update/8.0-to-8.1.md (limited to 'doc') diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md new file mode 100644 index 00000000000..34d7c25a461 --- /dev/null +++ b/doc/update/8.0-to-8.1.md @@ -0,0 +1,228 @@ +# From 8.0 to 8.1 + +### 0. Double-check your Git version + +**This notice applies only to /usr/local/bin/git** + +If you compiled Git from source on your GitLab server then please double-check +that you are using a version that protects against CVE-2014-9390. For six +months after this vulnerability became known the GitLab installation guide +still contained instructions that would install the outdated, 'vulnerable' Git +version 2.1.2. + +Run the following command to get your current Git version: + +```sh +/usr/local/bin/git --version +``` + +If you see 'No such file or directory' then you did not install Git according +to the outdated instructions from the GitLab installation guide and you can go +to the next step 'Stop server' below. + +If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4, +v2.2.1 or newer. You can use the [instructions in the GitLab source +installation +guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) +to install a newer version of Git. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-1-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-1-stable-ee +``` + +### 4. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch +sudo -u git -H git checkout v2.6.5 +``` + +### 5. Install gitlab-git-http-server + +First we download Go 1.5 and install it into `/usr/local/go`: + +```bash +curl -O --progress https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz +echo '5817fa4b2252afdb02e11e8b9dc1d9173ef3bd5a go1.5.linux-amd64.tar.gz' | shasum -c - && \ + sudo tar -C /usr/local -xzf go1.5.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.5.linux-amd64.tar.gz +``` + +Now we download `gitlab-git-http-server` and install it in `/home/git/gitlab-git-http-server`: + +```bash +cd /home/git +sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git +cd gitlab-git-http-server +sudo -u git -H make +``` + +Make sure your unicorn.rb file contains a 'listen' line for +'127.0.0.1:8080' and that this line is not commented out. + +``` +cd /home/git/gitlab +grep ^listen config/unicorn.rb + +# If there is no 'listen' line for 127.0.0.1:8080, add it: +sudo -u git tee -a config/unicorn.rb < true +EOF +``` + +If your Git repositories are in a directory other than `/home/git/repositories`, +you need to tell `gitlab-git-http-server` about it via `/etc/default/gitlab`. +See `lib/support/init.d/gitlab.default.example` for the options. + +### 6. Copy secrets + +The `secrets.yml` file is used to store keys to encrypt sessions and encrypt secure variables. +When you run migrations make sure to store it someplace safe. +Don't store it in the same place as your database backups, +otherwise your secrets are exposed if one of your backups is compromised. + +``` +cd /home/git/gitlab +sudo -u git -H cp config/secrets.yml.example config/secrets.yml +sudo -u git -H chmod 0600 config/secrets.yml +``` + +### 7. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +# Update init.d script +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +### 8. Update config files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-0-stable:config/gitlab.yml.example origin/8-1-stable:config/gitlab.yml.example +``` + +The new options include configuration of GitLab CI that are now being part of GitLab CE and EE. + +#### New Nginx configuration + +Because of the new `gitlab-git-http-server` you need to update your Nginx +configuration. If you skip this step 'git clone' and 'git push' over HTTP(S) +will stop working. + +View changes between the previous recommended Nginx configuration and the +current one: + +```sh +# For HTTPS configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab-ssl origin/8-1-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab origin/8-1-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache). +Also note that because Apache does not support upstreams behind Unix sockets you will need to let gitlab-git-http-server listen on a TCP port. You can do this via [/etc/default/gitlab](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-0-stable/lib/support/init.d/gitlab.default.example#L34). + +### 9. Migrate GitLab CI to GitLab CE/EE + +Now, GitLab CE and EE has CI integrated. However, migrations don't happen automatically and you need to do it manually. +Please follow the following guide [to migrate](../migrate_ci_to_ce/README.md) your GitLab CI instance to GitLab CE/EE. + +### 10. Use Redis v2.4.0+ + +Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but +Sidekiq jobs could fail due to lack of support for the SREM command. GitLab +8.0 now checks that Redis >= 2.4.0 is used. You can check your Redis version +with the following command: + + redis-cli info | grep redis_version + +### 11. Start application + + sudo service gitlab start + sudo service nginx restart + +### 12. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.0) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 7.14 to 8.0](7.14-to-8.0.md), except for the database migration +(The backup is already migrated to the previous version) + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +## Troubleshooting + +### "You appear to have cloned an empty repository." + +If you see this message when attempting to clone a repository hosted by GitLab, +this is likely due to an outdated Nginx or Apache configuration, or a missing or +misconfigured `gitlab-git-http-server` instance. Double-check that you correctly +completed [Step 5](#5-install-gitlab-git-http-server) to install the daemon and +[Step 8](#new-nginx-configuration) to reconfigure Nginx. -- cgit v1.2.1 From 0aa6061d6ab0ab921ad585329b43b84d20da873e Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Thu, 15 Oct 2015 15:08:31 +0200 Subject: Implement when syntax in .gitlab-ci.yml --- doc/ci/yaml/README.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4caeccacb7f..8507389f1ce 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -140,6 +140,7 @@ job_name: | except | optional | Defines a list of git refs for which build is not created | | tags | optional | Defines a list of tags which are used to select runner | | allow_failure | optional | Allow build to fail. Failed build doesn't contribute to commit status | +| when | optional | Define when to run build. Can be on_success, on_failure or always | ### script `script` is a shell script which is executed by runner. The shell script is prepended with `before_script`. @@ -196,6 +197,54 @@ job: The above specification will make sure that `job` is built by a runner that have `ruby` AND `postgres` tags defined. +### when +`when` is used to implement jobs that are run in case of failure or despite the failure. + +The `when` can be set to one of the following values: +1. `on_success` - execute build only when all builds from prior stages succeeded. This is default. +1. `on_failure` - execute build only when at least one of the build from prior stages failed. +1. `always` - execute build despite the status of builds from prior stages. + +``` +stages: +- build +- cleanup_build +- test +- deploy +- cleanup + +build: + stage: build + script: + - make build + +cleanup_build: + stage: cleanup_build + script: + - cleanup build when failed + when: on_failure + +test: + stage: test + script: + - make test + +deploy: + stage: deploy + script: + - make deploy + +cleanup: + stage: cleanup + script: + - cleanup after builds + when: always +``` + +The above script will: +1. Execute `cleanup_build` only when the `build` failed, +2. Always execute `cleanup` as the last step in pipeline. + ## Validate the .gitlab-ci.yml Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link to the Lint in the project's settings page or use short url `/lint`. -- cgit v1.2.1 From 374ea21f32f2977533be2c9b124867bf7c72407c Mon Sep 17 00:00:00 2001 From: Kamil Trzcinski Date: Thu, 15 Oct 2015 23:56:42 +0200 Subject: Backticks around when types --- doc/ci/yaml/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 8507389f1ce..d37e616bbac 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -140,7 +140,7 @@ job_name: | except | optional | Defines a list of git refs for which build is not created | | tags | optional | Defines a list of tags which are used to select runner | | allow_failure | optional | Allow build to fail. Failed build doesn't contribute to commit status | -| when | optional | Define when to run build. Can be on_success, on_failure or always | +| when | optional | Define when to run build. Can be `on_success`, `on_failure` or `always` | ### script `script` is a shell script which is executed by runner. The shell script is prepended with `before_script`. -- cgit v1.2.1 From 6232bb1ef39c310f5cd7fb6d11d75b9d51e55e58 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 15 Oct 2015 18:18:41 -0400 Subject: Remove notes about CI migration from 8.1 update guide [ci skip] --- doc/update/8.0-to-8.1.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) (limited to 'doc') diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index 34d7c25a461..43c06609f4d 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -148,8 +148,6 @@ There are new configuration options available for [`gitlab.yml`](config/gitlab.y git diff origin/8-0-stable:config/gitlab.yml.example origin/8-1-stable:config/gitlab.yml.example ``` -The new options include configuration of GitLab CI that are now being part of GitLab CE and EE. - #### New Nginx configuration Because of the new `gitlab-git-http-server` you need to update your Nginx @@ -170,12 +168,7 @@ git diff origin/8-0-stable:lib/support/nginx/gitlab origin/8-1-stable:lib/suppor If you are using Apache instead of NGINX please see the updated [Apache templates](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache). Also note that because Apache does not support upstreams behind Unix sockets you will need to let gitlab-git-http-server listen on a TCP port. You can do this via [/etc/default/gitlab](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-0-stable/lib/support/init.d/gitlab.default.example#L34). -### 9. Migrate GitLab CI to GitLab CE/EE - -Now, GitLab CE and EE has CI integrated. However, migrations don't happen automatically and you need to do it manually. -Please follow the following guide [to migrate](../migrate_ci_to_ce/README.md) your GitLab CI instance to GitLab CE/EE. - -### 10. Use Redis v2.4.0+ +### 9. Use Redis v2.4.0+ Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but Sidekiq jobs could fail due to lack of support for the SREM command. GitLab @@ -184,12 +177,12 @@ with the following command: redis-cli info | grep redis_version -### 11. Start application +### 10. Start application sudo service gitlab start sudo service nginx restart -### 12. Check application status +### 11. Check application status Check if GitLab and its environment are configured correctly: -- cgit v1.2.1 From c2c9f6d52d392be8bb6bce5366cdcbcfdf38c9e5 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 15 Oct 2015 18:49:37 -0400 Subject: Remove 8.0-only steps from the 8.1 update guide Also adds a preamble note that a working 8.0 installation (i.e., gitlab-git-http-server, updated Nginx/Apache configs) is required before proceeding. [ci skip] --- doc/update/8.0-to-8.1.md | 98 +++++------------------------------------------- 1 file changed, 9 insertions(+), 89 deletions(-) (limited to 'doc') diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index 43c06609f4d..4dacc97f7f7 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -1,5 +1,9 @@ # From 8.0 to 8.1 +**NOTE:** GitLab 8.0 introduced several significant changes related to +installation and configuration which *are not duplicated here*. Be sure you're +already running a working version of 8.0 before proceeding with this guide. + ### 0. Double-check your Git version **This notice applies only to /usr/local/bin/git** @@ -66,58 +70,7 @@ sudo -u git -H git fetch sudo -u git -H git checkout v2.6.5 ``` -### 5. Install gitlab-git-http-server - -First we download Go 1.5 and install it into `/usr/local/go`: - -```bash -curl -O --progress https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz -echo '5817fa4b2252afdb02e11e8b9dc1d9173ef3bd5a go1.5.linux-amd64.tar.gz' | shasum -c - && \ - sudo tar -C /usr/local -xzf go1.5.linux-amd64.tar.gz -sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ -rm go1.5.linux-amd64.tar.gz -``` - -Now we download `gitlab-git-http-server` and install it in `/home/git/gitlab-git-http-server`: - -```bash -cd /home/git -sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git -cd gitlab-git-http-server -sudo -u git -H make -``` - -Make sure your unicorn.rb file contains a 'listen' line for -'127.0.0.1:8080' and that this line is not commented out. - -``` -cd /home/git/gitlab -grep ^listen config/unicorn.rb - -# If there is no 'listen' line for 127.0.0.1:8080, add it: -sudo -u git tee -a config/unicorn.rb < true -EOF -``` - -If your Git repositories are in a directory other than `/home/git/repositories`, -you need to tell `gitlab-git-http-server` about it via `/etc/default/gitlab`. -See `lib/support/init.d/gitlab.default.example` for the options. - -### 6. Copy secrets - -The `secrets.yml` file is used to store keys to encrypt sessions and encrypt secure variables. -When you run migrations make sure to store it someplace safe. -Don't store it in the same place as your database backups, -otherwise your secrets are exposed if one of your backups is compromised. - -``` -cd /home/git/gitlab -sudo -u git -H cp config/secrets.yml.example config/secrets.yml -sudo -u git -H chmod 0600 config/secrets.yml -``` - -### 7. Install libs, migrations, etc. +### 5. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -138,7 +91,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` -### 8. Update config files +### 6. Update configuration files #### New configuration options for `gitlab.yml` @@ -148,41 +101,12 @@ There are new configuration options available for [`gitlab.yml`](config/gitlab.y git diff origin/8-0-stable:config/gitlab.yml.example origin/8-1-stable:config/gitlab.yml.example ``` -#### New Nginx configuration - -Because of the new `gitlab-git-http-server` you need to update your Nginx -configuration. If you skip this step 'git clone' and 'git push' over HTTP(S) -will stop working. - -View changes between the previous recommended Nginx configuration and the -current one: - -```sh -# For HTTPS configurations -git diff origin/8-0-stable:lib/support/nginx/gitlab-ssl origin/8-1-stable:lib/support/nginx/gitlab-ssl - -# For HTTP configurations -git diff origin/8-0-stable:lib/support/nginx/gitlab origin/8-1-stable:lib/support/nginx/gitlab -``` - -If you are using Apache instead of NGINX please see the updated [Apache templates](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache). -Also note that because Apache does not support upstreams behind Unix sockets you will need to let gitlab-git-http-server listen on a TCP port. You can do this via [/etc/default/gitlab](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-0-stable/lib/support/init.d/gitlab.default.example#L34). - -### 9. Use Redis v2.4.0+ - -Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but -Sidekiq jobs could fail due to lack of support for the SREM command. GitLab -8.0 now checks that Redis >= 2.4.0 is used. You can check your Redis version -with the following command: - - redis-cli info | grep redis_version - -### 10. Start application +### 7. Start application sudo service gitlab start sudo service nginx restart -### 11. Check application status +### 8. Check application status Check if GitLab and its environment are configured correctly: @@ -214,8 +138,4 @@ If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of ### "You appear to have cloned an empty repository." -If you see this message when attempting to clone a repository hosted by GitLab, -this is likely due to an outdated Nginx or Apache configuration, or a missing or -misconfigured `gitlab-git-http-server` instance. Double-check that you correctly -completed [Step 5](#5-install-gitlab-git-http-server) to install the daemon and -[Step 8](#new-nginx-configuration) to reconfigure Nginx. +See the [7.14 to 8.0 update guide](7.14-to-8.0.md#troubleshooting). -- cgit v1.2.1 From 6fe2a679a799c0914b8c32e011343939800c5480 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 15 Oct 2015 19:46:01 -0400 Subject: Update CI YAML docs --- doc/ci/yaml/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index d37e616bbac..ea8f72bc135 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -200,9 +200,10 @@ The above specification will make sure that `job` is built by a runner that have ### when `when` is used to implement jobs that are run in case of failure or despite the failure. -The `when` can be set to one of the following values: -1. `on_success` - execute build only when all builds from prior stages succeeded. This is default. -1. `on_failure` - execute build only when at least one of the build from prior stages failed. +`when` can be set to one of the following values: + +1. `on_success` - execute build only when all builds from prior stages succeeded. This is the default. +1. `on_failure` - execute build only when at least one build from prior stages failed. 1. `always` - execute build despite the status of builds from prior stages. ``` @@ -250,4 +251,4 @@ Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link to the Lint in the project's settings page or use short url `/lint`. ## Skipping builds -There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped \ No newline at end of file +There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped -- cgit v1.2.1 From a317c5296e9c142cfd04168eced7d7bfd9458ffa Mon Sep 17 00:00:00 2001 From: Ashley Hindle Date: Fri, 16 Oct 2015 07:36:20 +0100 Subject: Changed loose to lose --- doc/raketasks/backup_restore.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index db3f6bb40bd..06f582dcee8 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -16,7 +16,7 @@ You need to keep a separate copy of `/etc/gitlab/gitlab-secrets.json` from source). This file contains the database encryption key used for two-factor authentication. If you restore a GitLab backup without restoring the database encryption key, users who have two-factor -authentication enabled will loose access to your GitLab server. +authentication enabled will lose access to your GitLab server. If you are interested in GitLab CI backup please follow to the [CI backup documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/raketasks/backup_restore.md)* -- cgit v1.2.1 From 8479feeba27800ee2e8286d759247339703f83be Mon Sep 17 00:00:00 2001 From: Benny Schimmer Date: Mon, 19 Oct 2015 23:10:17 +0200 Subject: Simplify patch version updates (less steps) --- doc/update/patch_versions.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index da719229ab6..593722eb01f 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -49,9 +49,7 @@ sudo -u git -H bundle install --without development test mysql --deployment sudo -u git -H bundle install --without development test postgres --deployment sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production -sudo -u git -H bundle exec rake assets:clean RAILS_ENV=production -sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production -sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production ``` ### 5. Start application -- cgit v1.2.1 From 8052c4ef4e23ea4f4a221fbc6ead138f33fb0f91 Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Tue, 20 Oct 2015 16:09:11 +0000 Subject: Merge branch 'rs-8-1-update-guide-changes' into 'master' Update 8.0-to-8.1 update guide - User needs to update gitlab-git-http-server - User needs to update Nginx configuration [ci skip] See merge request !1625 --- doc/update/8.0-to-8.1.md | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index 4dacc97f7f7..d57c0d0674d 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -70,7 +70,16 @@ sudo -u git -H git fetch sudo -u git -H git checkout v2.6.5 ``` -### 5. Install libs, migrations, etc. +### 5. Update gitlab-git-http-server + +```bash +cd /home/git/gitlab-git-http-server +sudo -u git -H git fetch origin +sudo -u git -H git checkout 0.3.0 +sudo -u git -H make +``` + +### 6. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -91,7 +100,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` -### 6. Update configuration files +### 7. Update configuration files #### New configuration options for `gitlab.yml` @@ -101,12 +110,33 @@ There are new configuration options available for [`gitlab.yml`](config/gitlab.y git diff origin/8-0-stable:config/gitlab.yml.example origin/8-1-stable:config/gitlab.yml.example ``` -### 7. Start application +#### Nginx configuration + +View changes between the previous recommended Nginx configuration and the +current one: + +```sh +# For HTTPS configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab-ssl origin/8-1-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab origin/8-1-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-git-http-server listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-1-stable/lib/support/init.d/gitlab.default.example#L34 + +### 8. Start application sudo service gitlab start sudo service nginx restart -### 8. Check application status +### 9. Check application status Check if GitLab and its environment are configured correctly: -- cgit v1.2.1