diff options
Diffstat (limited to 'doc/update')
| -rw-r--r-- | doc/update/10.4-to-10.5.md | 361 | ||||
| -rw-r--r-- | doc/update/mysql_to_postgresql.md | 264 |
2 files changed, 501 insertions, 124 deletions
diff --git a/doc/update/10.4-to-10.5.md b/doc/update/10.4-to-10.5.md new file mode 100644 index 00000000000..313419ed13d --- /dev/null +++ b/doc/update/10.4-to-10.5.md @@ -0,0 +1,361 @@ +--- +comments: false +--- + +# From 10.4 to 10.5 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz +echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz +cd ruby-2.3.6 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets. +We require a minimum version of node v6.0.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v6.0.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-5-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-5-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-4-stable:config/gitlab.yml.example origin/10-5-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-4-stable:lib/support/nginx/gitlab-ssl origin/10-5-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-4-stable:lib/support/nginx/gitlab origin/10-5-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-5-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-5-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-4-stable:lib/support/init.d/gitlab.default.example origin/10-5-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.4) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.3 to 10.4](10.3-to-10.4.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-5-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-5-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index fff47180099..44e9f6c5516 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -1,13 +1,15 @@ --- -last_updated: 2017-10-05 +last_updated: 2018-02-07 --- # Migrating from MySQL to PostgreSQL -> **Note:** This guide assumes you have a working Omnibus GitLab instance with +> **Note:** This guide assumes you have a working GitLab instance with > MySQL and want to migrate to bundled PostgreSQL database. -## Prerequisites +## Omnibus installation + +### Prerequisites First, we'll need to enable the bundled PostgreSQL database with up-to-date schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data @@ -19,7 +21,7 @@ Here's what you'll need to have installed: - Omnibus GitLab - MySQL -## Enable bundled PostgreSQL database +### Enable bundled PostgreSQL database 1. Stop GitLab: @@ -65,7 +67,7 @@ Here's what you'll need to have installed: After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. -## Migrate data from MySQL to PostgreSQL +### Migrate data from MySQL to PostgreSQL Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: @@ -104,122 +106,9 @@ the following: ----------------------------------------------- --------- --------- --------- -------------- public.abuse_reports 0 0 0 0.490s public.appearances 0 0 0 0.488s - public.approvals 0 0 0 0.273s - public.application_settings 1 1 0 0.266s - public.approvers 0 0 0 0.339s - public.approver_groups 0 0 0 0.357s - public.audit_events 1 1 0 0.410s - public.award_emoji 0 0 0 0.441s - public.boards 0 0 0 0.505s - public.broadcast_messages 0 0 0 0.498s - public.chat_names 0 0 0 0.576s - public.chat_teams 0 0 0 0.617s - public.ci_builds 0 0 0 0.611s - public.ci_group_variables 0 0 0 0.620s - public.ci_pipelines 0 0 0 0.599s - public.ci_pipeline_schedules 0 0 0 0.622s - public.ci_pipeline_schedule_variables 0 0 0 0.573s - public.ci_pipeline_variables 0 0 0 0.594s - public.ci_runners 0 0 0 0.533s - public.ci_runner_projects 0 0 0 0.584s - public.ci_sources_pipelines 0 0 0 0.564s - public.ci_stages 0 0 0 0.595s - public.ci_triggers 0 0 0 0.569s - public.ci_trigger_requests 0 0 0 0.596s - public.ci_variables 0 0 0 0.565s - public.container_repositories 0 0 0 0.605s - public.conversational_development_index_metrics 0 0 0 0.571s - public.deployments 0 0 0 0.607s - public.emails 0 0 0 0.602s - public.deploy_keys_projects 0 0 0 0.557s - public.events 160 160 0 0.677s - public.environments 0 0 0 0.567s - public.features 0 0 0 0.639s - public.events_for_migration 160 160 0 0.582s - public.feature_gates 0 0 0 0.579s - public.forked_project_links 0 0 0 0.660s - public.geo_nodes 0 0 0 0.686s - public.geo_event_log 0 0 0 0.626s - public.geo_repositories_changed_events 0 0 0 0.677s - public.geo_node_namespace_links 0 0 0 0.618s - public.geo_repository_renamed_events 0 0 0 0.696s - public.gpg_keys 0 0 0 0.704s - public.geo_repository_deleted_events 0 0 0 0.638s - public.historical_data 0 0 0 0.729s - public.geo_repository_updated_events 0 0 0 0.634s - public.index_statuses 0 0 0 0.746s - public.gpg_signatures 0 0 0 0.667s - public.issue_assignees 80 80 0 0.769s - public.identities 0 0 0 0.655s - public.issue_metrics 80 80 0 0.781s - public.issues 80 80 0 0.720s - public.labels 0 0 0 0.795s - public.issue_links 0 0 0 0.707s - public.label_priorities 0 0 0 0.793s - public.keys 0 0 0 0.734s - public.lfs_objects 0 0 0 0.812s - public.label_links 0 0 0 0.725s - public.licenses 0 0 0 0.813s - public.ldap_group_links 0 0 0 0.751s - public.members 52 52 0 0.830s - public.lfs_objects_projects 0 0 0 0.738s - public.merge_requests_closing_issues 0 0 0 0.825s - public.lists 0 0 0 0.769s - public.merge_request_diff_commits 0 0 0 0.840s - public.merge_request_metrics 0 0 0 0.837s - public.merge_requests 0 0 0 0.753s - public.merge_request_diffs 0 0 0 0.771s - public.namespaces 30 30 0 0.874s - public.merge_request_diff_files 0 0 0 0.775s - public.notes 0 0 0 0.849s - public.milestones 40 40 0 0.799s - public.oauth_access_grants 0 0 0 0.979s - public.namespace_statistics 0 0 0 0.797s - public.oauth_applications 0 0 0 0.899s - public.notification_settings 72 72 0 0.818s - public.oauth_access_tokens 0 0 0 0.807s - public.pages_domains 0 0 0 0.958s - public.oauth_openid_requests 0 0 0 0.832s - public.personal_access_tokens 0 0 0 0.965s - public.projects 8 8 0 0.987s - public.path_locks 0 0 0 0.925s - public.plans 0 0 0 0.923s - public.project_features 8 8 0 0.985s - public.project_authorizations 66 66 0 0.969s - public.project_import_data 8 8 0 1.002s - public.project_statistics 8 8 0 1.001s - public.project_group_links 0 0 0 0.949s - public.project_mirror_data 0 0 0 0.972s - public.protected_branch_merge_access_levels 0 0 0 1.017s - public.protected_branches 0 0 0 0.969s - public.protected_branch_push_access_levels 0 0 0 0.991s - public.protected_tags 0 0 0 1.009s - public.protected_tag_create_access_levels 0 0 0 0.985s - public.push_event_payloads 0 0 0 1.041s - public.push_rules 0 0 0 0.999s - public.redirect_routes 0 0 0 1.020s - public.remote_mirrors 0 0 0 1.034s - public.releases 0 0 0 0.993s - public.schema_migrations 896 896 0 1.057s - public.routes 38 38 0 1.021s - public.services 0 0 0 1.055s - public.sent_notifications 0 0 0 1.003s - public.slack_integrations 0 0 0 1.022s - public.spam_logs 0 0 0 1.024s - public.snippets 0 0 0 1.058s - public.subscriptions 0 0 0 1.069s - public.taggings 0 0 0 1.099s - public.timelogs 0 0 0 1.104s - public.system_note_metadata 0 0 0 1.038s - public.tags 0 0 0 1.034s - public.trending_projects 0 0 0 1.140s - public.uploads 0 0 0 1.129s - public.todos 80 80 0 1.085s - public.users_star_projects 0 0 0 1.153s - public.u2f_registrations 0 0 0 1.061s - public.web_hooks 0 0 0 1.179s - public.users 26 26 0 1.163s - public.user_agent_details 0 0 0 1.068s + . + . + . public.web_hook_logs 0 0 0 1.080s ----------------------------------------------- --------- --------- --------- -------------- COPY Threads Completion 4 4 0 2.008s @@ -240,9 +129,9 @@ the following: Now, you can verify that everything worked by visiting GitLab. -## Troubleshooting +### Troubleshooting -### Permissions +#### Permissions Note that the PostgreSQL user that you use for the above MUST have **superuser** privileges. Otherwise, you may see a similar message to the following: @@ -256,7 +145,7 @@ debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL; ``` -### Experiencing 500 errors after the migration +#### Experiencing 500 errors after the migration If you experience 500 errors after the migration, try to clear the cache: @@ -265,3 +154,130 @@ sudo gitlab-rake cache:clear ``` [reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure + +## Source installation + +### Prerequisites + +#### Install PostgreSQL and create database + +See [installation guide](../install/installation.md#6-database). + +#### Install [pgloader](http://pgloader.io) 3.4.1+ + +Install directly from your distro: +``` bash +sudo apt-get install pgloader +``` + +If this version is too old, use PostgreSQL's repository: +``` bash +# add repository +sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list' + +# add key +sudo apt-get install wget ca-certificates +wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - + +# install package +sudo apt-get update +sudo apt-get install pgloader +``` + +### Enable bundled PostgreSQL database + +1. Stop GitLab: + + ``` bash + sudo service gitlab stop + ``` + +1. Switch database from MySQL to PostgreSQL + + ``` bash + cd /home/git/gitlab + sudo -u git mv config/database.yml config/database.yml.bak + sudo -u git cp config/database.yml.postgresql config/database.yml + sudo -u git -H chmod o-rwx config/database.yml + ``` + +1. Run the following commands to prepare the schema: + + ``` bash + sudo -u git -H bundle exec rake db:create db:migrate RAILS_ENV=production + ``` + +After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. + +### Migrate data from MySQL to PostgreSQL + +Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: + +1. Save the following snippet in a `commands.load` file, and edit with your + MySQL `username`, `password` and `host`: + + ``` + LOAD DATABASE + FROM mysql://username:password@host/gitlabhq_production + INTO postgresql://postgres@unix://var/run/postgresql:/gitlabhq_production + + WITH include no drop, truncate, disable triggers, create no tables, + create no indexes, preserve index names, no foreign keys, + data only + + ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' + + ; + ``` + +1. Start the migration: + + ``` bash + sudo -u postgres pgloader commands.load + ``` + +1. Once the migration finishes, you should see a summary table that looks like +the following: + + + ``` + table name read imported errors total time + ----------------------------------------------- --------- --------- --------- -------------- + fetch meta data 119 119 0 0.388s + Truncate 119 119 0 1.134s + ----------------------------------------------- --------- --------- --------- -------------- + public.abuse_reports 0 0 0 0.490s + public.appearances 0 0 0 0.488s + . + . + . + public.web_hook_logs 0 0 0 1.080s + ----------------------------------------------- --------- --------- --------- -------------- + COPY Threads Completion 4 4 0 2.008s + Reset Sequences 113 113 0 0.304s + Install Comments 0 0 0 0.000s + ----------------------------------------------- --------- --------- --------- -------------- + Total import time 1894 1894 0 12.497s + ``` + + If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See + the [troubleshooting guide](#Troubleshooting) for more details. + +1. Start GitLab: + + ``` bash + sudo service gitlab start + ``` + +Now, you can verify that everything worked by visiting GitLab. + +### Troubleshooting + +#### Experiencing 500 errors after the migration + +If you experience 500 errors after the migration, try to clear the cache: + +``` bash +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + |