Add #run_command! to task helpers to raise a TaskFailedError if status is not 0

Signed-off-by: Rémy Coutable <remy@rymai.me>

6 files changed, 246 insertions, 26 deletions

diff --git a/doc/install/installation.md b/doc/install/installation.md
index 25b186b63f7..de650f0900d 100644
--- a/doc/install/installation.md
+++ b/doc/install/installation.md
@@ -408,7 +408,7 @@ You can specify a different Git repository by providing `GITLAB_WORKHORSE_REPO`:
 
     cd /home/git/gitlab
 
-    sudo -u git -H bundle exec rake gitlab:workhorse:install[/home/git/gitlab-workhorse] GITLAB_WORKHORSE_REPO=https://gitlab.com/gitlab-org/gitlab-ce.git RAILS_ENV=production
+    sudo -u git -H bundle exec rake gitlab:workhorse:install[/home/git/gitlab-workhorse] GITLAB_WORKHORSE_REPO=https://example.com/gitlab-workhorse.git RAILS_ENV=production
 
 You can specify a different version to use by providing `GITLAB_WORKHORSE_VERSION`:
 
diff --git a/doc/update/8.14-to-8.15.md b/doc/update/8.14-to-8.15.md
new file mode 100644
index 00000000000..9e6a3b45394
--- /dev/null
+++ b/doc/update/8.14-to-8.15.md
@@ -0,0 +1,202 @@
+# From 8.14 to 8.15
+
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
+### 1. Stop server
+
+    sudo service gitlab stop
+
+### 2. Backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
+```
+
+### 3. Update Ruby
+
+We will continue supporting Ruby < 2.3 for the time being but we recommend you
+upgrade to Ruby 2.3 if you're running a source installation, as this is the same
+version that ships with our Omnibus package.
+
+You can check which version you are running with `ruby -v`.
+
+Download and compile Ruby:
+
+```bash
+mkdir /tmp/ruby && cd /tmp/ruby
+curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
+echo 'c39b4001f7acb4e334cb60a0f4df72d434bef711  ruby-2.3.1.tar.gz' | shasum --check - && tar xzf ruby-2.3.1.tar.gz
+cd ruby-2.3.1
+./configure --disable-install-rdoc
+make
+sudo make install
+```
+
+Install Bundler:
+
+```bash
+sudo gem install bundler --no-ri --no-rdoc
+```
+
+### 4. Get latest code
+
+```bash
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
+```
+
+For GitLab Community Edition:
+
+```bash
+sudo -u git -H git checkout 8-15-stable
+```
+
+OR
+
+For GitLab Enterprise Edition:
+
+```bash
+sudo -u git -H git checkout 8-15-stable-ee
+```
+
+### 5. Update gitlab-shell
+
+```bash
+cd /home/git/gitlab-shell
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v4.0.0
+```
+
+### 6. Update gitlab-workhorse
+
+Install and compile gitlab-workhorse. This requires
+[Go 1.5](https://golang.org/dl) which should already be on your system from
+GitLab 8.1.
+
+```bash
+sudo -u git -H bundle exec rake gitlab:workhorse:install[/home/git/gitlab-workhorse] RAILS_ENV=production
+```
+
+### 7. Install libs, migrations, etc.
+
+```bash
+cd /home/git/gitlab
+
+# MySQL installations (note: the line below states '--without postgres')
+sudo -u git -H bundle install --without postgres development test --deployment
+
+# PostgreSQL installations (note: the line below states '--without mysql')
+sudo -u git -H bundle install --without mysql development test --deployment
+
+# Optional: clean up old gems
+sudo -u git -H bundle clean
+
+# Run database migrations
+sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
+
+# Clean up assets and cache
+sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production
+```
+
+### 8. Update configuration files
+
+#### New configuration options for `gitlab.yml`
+
+There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`:
+
+```sh
+git diff origin/8-13-stable:config/gitlab.yml.example origin/8-15-stable:config/gitlab.yml.example
+```
+
+#### Git configuration
+
+Configure Git to generate packfile bitmaps (introduced in Git 2.0) on
+the GitLab server during `git gc`.
+
+```sh
+sudo -u git -H git config --global repack.writeBitmaps true
+```
+
+#### Nginx configuration
+
+Ensure you're still up-to-date with the latest NGINX configuration changes:
+
+```sh
+# For HTTPS configurations
+git diff origin/8-13-stable:lib/support/nginx/gitlab-ssl origin/8-15-stable:lib/support/nginx/gitlab-ssl
+
+# For HTTP configurations
+git diff origin/8-13-stable:lib/support/nginx/gitlab origin/8-15-stable:lib/support/nginx/gitlab
+```
+
+If you are using Apache instead of NGINX please see the updated [Apache templates].
+Also note that because Apache does not support upstreams behind Unix sockets you
+will need to let gitlab-workhorse listen on a TCP port. You can do this
+via [/etc/default/gitlab].
+
+[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
+[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-15-stable/lib/support/init.d/gitlab.default.example#L38
+
+#### SMTP configuration
+
+If you're installing from source and use SMTP to deliver mail, you will need to add the following line
+to config/initializers/smtp_settings.rb:
+
+```ruby
+ActionMailer::Base.delivery_method = :smtp
+```
+
+See [smtp_settings.rb.sample] as an example.
+
+[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-15-stable/config/initializers/smtp_settings.rb.sample#L13
+
+#### Init script
+
+Ensure you're still up-to-date with the latest init script changes:
+
+    sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
+
+For Ubuntu 16.04.1 LTS:
+
+    sudo systemctl daemon-reload
+
+### 9. Start application
+
+    sudo service gitlab start
+    sudo service nginx restart
+
+### 10. Check application status
+
+Check if GitLab and its environment are configured correctly:
+
+    sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
+
+To make sure you didn't miss anything run a more thorough check:
+
+    sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
+
+If all items are green, then congratulations, the upgrade is complete!
+
+## Things went south? Revert to previous version (8.14)
+
+### 1. Revert the code to the previous version
+
+Follow the [upgrade guide from 8.13 to 8.14](8.13-to-8.14.md), except for the
+database migration (the backup is already migrated to the previous version).
+
+### 2. Restore from the backup
+
+```bash
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
+```
+
+If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
diff --git a/lib/tasks/gitlab/task_helpers.rake b/lib/tasks/gitlab/task_helpers.rake
index 85c3a3a9b0f..c0759b96602 100644
--- a/lib/tasks/gitlab/task_helpers.rake
+++ b/lib/tasks/gitlab/task_helpers.rake
@@ -1,4 +1,5 @@
 module Gitlab
+  class TaskFailedError < StandardError; end
   class TaskAbortedByUserError < StandardError; end
 end
 
@@ -81,6 +82,18 @@ namespace :gitlab do
     '' # if the command does not exist, return an empty string
   end
 
+  # Runs the given command and raise a Gitlab::TaskFailedError exception if
+  # the command does not exit with 0
+  #
+  # Returns the output of the command otherwise
+  def run_command!(command)
+    output, status = Gitlab::Popen.popen(command)
+
+    raise Gitlab::TaskFailedError unless status.zero?
+
+    output
+  end
+
   def uid_for(user_name)
     run_command(%W(id -u #{user_name})).chomp.to_i
   end
@@ -145,11 +158,11 @@ namespace :gitlab do
   def checkout_or_clone_tag(tag:, repo:, target_dir:)
     if Dir.exist?(target_dir)
       Dir.chdir(target_dir) do
-        run_command(%W[#{Gitlab.config.git.bin_path} fetch --tags --quiet])
-        run_command(%W[#{Gitlab.config.git.bin_path} checkout --quiet #{tag}])
+        run_command!(%W[#{Gitlab.config.git.bin_path} fetch --tags --quiet])
+        run_command!(%W[#{Gitlab.config.git.bin_path} checkout --quiet #{tag}])
       end
     else
-      run_command(%W[#{Gitlab.config.git.bin_path} clone -- #{repo} #{target_dir}])
+      run_command!(%W[#{Gitlab.config.git.bin_path} clone -- #{repo} #{target_dir}])
     end
 
     # Make sure we're on the right tag
@@ -161,13 +174,18 @@ namespace :gitlab do
   end
 
   def reset_to_tag(tag_wanted)
-    tag, status = Gitlab::Popen.popen(%W[#{Gitlab.config.git.bin_path} describe -- #{tag_wanted}])
+    tag =
+      begin
+        run_command!(%W[#{Gitlab.config.git.bin_path} describe -- #{tag_wanted}])
+      rescue Gitlab::TaskFailedError
+        run_command!(%W[#{Gitlab.config.git.bin_path} fetch origin])
+        run_command!(%W[#{Gitlab.config.git.bin_path} describe -- origin/#{tag_wanted}])
+      end
 
-    unless status.zero?
-      run_command(%W(#{Gitlab.config.git.bin_path} fetch origin))
-      tag = run_command(%W[#{Gitlab.config.git.bin_path} describe -- origin/#{tag_wanted}])
+    if tag
+      run_command!(%W[#{Gitlab.config.git.bin_path} reset --hard #{tag.strip}])
+    else
+      raise Gitlab::TaskFailedError
     end
-
-    run_command(%W[#{Gitlab.config.git.bin_path} reset --hard #{tag.strip}])
   end
 end
diff --git a/lib/tasks/gitlab/workhorse.rake b/lib/tasks/gitlab/workhorse.rake
index 5964cb83b89..563744dd0c7 100644
--- a/lib/tasks/gitlab/workhorse.rake
+++ b/lib/tasks/gitlab/workhorse.rake
@@ -16,7 +16,7 @@ namespace :gitlab do
       command = status.zero? ? 'gmake' : 'make'
 
       Dir.chdir(args.dir) do
-        run_command([command])
+        run_command!([command])
       end
     end
   end
diff --git a/spec/tasks/gitlab/task_helpers_spec.rb b/spec/tasks/gitlab/task_helpers_spec.rb
index 2a2d4a39ba8..dccb3b4cf9a 100644
--- a/spec/tasks/gitlab/task_helpers_spec.rb
+++ b/spec/tasks/gitlab/task_helpers_spec.rb
@@ -15,7 +15,7 @@ describe 'gitlab:workhorse namespace rake task' do
     let(:tag) { 'v1.1.0' }
     before do
       FileUtils.rm_rf(clone_path)
-      allow_any_instance_of(Object).to receive(:run_command)
+      allow_any_instance_of(Object).to receive(:run_command!)
       expect_any_instance_of(Object).to receive(:reset_to_tag).with(tag)
     end
 
@@ -27,7 +27,7 @@ describe 'gitlab:workhorse namespace rake task' do
       it 'clones the repo, retrieve the tag from origin, and checkout the tag' do
         expect(Dir).to receive(:chdir).and_call_original
         expect_any_instance_of(Object).
-          to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} clone -- #{repo} #{clone_path}]) { FileUtils.mkdir_p(clone_path) } # Fake the cloning
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} clone -- #{repo} #{clone_path}]) { FileUtils.mkdir_p(clone_path) } # Fake the cloning
 
         checkout_or_clone_tag(tag: tag, repo: repo, target_dir: clone_path)
       end
@@ -41,9 +41,9 @@ describe 'gitlab:workhorse namespace rake task' do
       it 'fetch and checkout the tag' do
         expect(Dir).to receive(:chdir).twice.and_call_original
         expect_any_instance_of(Object).
-          to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} fetch --tags --quiet])
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} fetch --tags --quiet])
         expect_any_instance_of(Object).
-          to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} checkout --quiet #{tag}])
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} checkout --quiet #{tag}])
 
         checkout_or_clone_tag(tag: tag, repo: repo, target_dir: clone_path)
       end
@@ -54,20 +54,20 @@ describe 'gitlab:workhorse namespace rake task' do
     let(:tag) { 'v1.1.0' }
     before do
       expect_any_instance_of(Object).
-        to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} reset --hard #{tag}])
+        to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} reset --hard #{tag}])
     end
 
     context 'when the tag is not checked out locally' do
       before do
-        expect(Gitlab::Popen).
-          to receive(:popen).with(%W[#{Gitlab.config.git.bin_path} describe -- #{tag}]).and_return(['', 42])
+        expect_any_instance_of(Object).
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} describe -- #{tag}]).and_raise(Gitlab::TaskFailedError)
       end
 
       it 'fetch origin, ensure the tag exists, and resets --hard to the given tag' do
         expect_any_instance_of(Object).
-          to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} fetch origin])
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} fetch origin])
         expect_any_instance_of(Object).
-          to receive(:run_command).with(%W[#{Gitlab.config.git.bin_path} describe -- origin/#{tag}]).and_return(tag)
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} describe -- origin/#{tag}]).and_return(tag)
 
         reset_to_tag(tag)
       end
@@ -75,8 +75,8 @@ describe 'gitlab:workhorse namespace rake task' do
 
     context 'when the tag is checked out locally' do
       before do
-        expect(Gitlab::Popen).
-          to receive(:popen).with(%W[#{Gitlab.config.git.bin_path} describe -- #{tag}]).and_return([tag, 0])
+        expect_any_instance_of(Object).
+          to receive(:run_command!).with(%W[#{Gitlab.config.git.bin_path} describe -- #{tag}]).and_return(tag)
       end
 
       it 'resets --hard to the given tag' do
diff --git a/spec/tasks/gitlab/workhorse_rake_spec.rb b/spec/tasks/gitlab/workhorse_rake_spec.rb
index c8ad004282a..87bc1b128bf 100644
--- a/spec/tasks/gitlab/workhorse_rake_spec.rb
+++ b/spec/tasks/gitlab/workhorse_rake_spec.rb
@@ -88,12 +88,12 @@ describe 'gitlab:workhorse namespace rake task' do
       context 'gmake is available' do
         before do
           expect_any_instance_of(Object).to receive(:checkout_or_clone_tag)
-          allow_any_instance_of(Object).to receive(:run_command).with(['gmake']).and_return(true)
+          allow_any_instance_of(Object).to receive(:run_command!).with(['gmake']).and_return(true)
         end
 
         it 'calls gmake in the gitlab-workhorse directory' do
           expect(Gitlab::Popen).to receive(:popen).with(%w[which gmake]).and_return(['/usr/bin/gmake', 0])
-          expect_any_instance_of(Object).to receive(:run_command).with(['gmake']).and_return(true)
+          expect_any_instance_of(Object).to receive(:run_command!).with(['gmake']).and_return(true)
 
           run_rake_task('gitlab:workhorse:install', clone_path)
         end
@@ -102,12 +102,12 @@ describe 'gitlab:workhorse namespace rake task' do
       context 'gmake is not available' do
         before do
           expect_any_instance_of(Object).to receive(:checkout_or_clone_tag)
-          allow_any_instance_of(Object).to receive(:run_command).with(['make']).and_return(true)
+          allow_any_instance_of(Object).to receive(:run_command!).with(['make']).and_return(true)
         end
 
         it 'calls make in the gitlab-workhorse directory' do
           expect(Gitlab::Popen).to receive(:popen).with(%w[which gmake]).and_return(['', 42])
-          expect_any_instance_of(Object).to receive(:run_command).with(['make']).and_return(true)
+          expect_any_instance_of(Object).to receive(:run_command!).with(['make']).and_return(true)
 
           run_rake_task('gitlab:workhorse:install', clone_path)
         end