diff options
Diffstat (limited to 'doc/administration')
| -rw-r--r-- | doc/administration/auth/ldap.md | 6 | ||||
| -rw-r--r-- | doc/administration/high_availability/README.md | 4 | ||||
| -rw-r--r-- | doc/administration/index.md | 121 | ||||
| -rw-r--r-- | doc/administration/job_artifacts.md | 39 | ||||
| -rw-r--r-- | doc/administration/monitoring/github_imports.md | 101 | ||||
| -rw-r--r-- | doc/administration/pages/index.md | 3 | ||||
| -rw-r--r-- | doc/administration/raketasks/maintenance.md | 4 | ||||
| -rw-r--r-- | doc/administration/raketasks/storage.md | 97 | ||||
| -rw-r--r-- | doc/administration/repository_storage_types.md | 20 | ||||
| -rw-r--r-- | doc/administration/repository_storages.md | 4 | ||||
| -rw-r--r-- | doc/administration/troubleshooting/debug.md | 28 | ||||
| -rw-r--r-- | doc/administration/troubleshooting/sidekiq.md | 2 |
12 files changed, 400 insertions, 29 deletions
diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index ad903aef896..6b5a0f139c5 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -30,6 +30,12 @@ immediately block all access. >**Note**: GitLab EE supports a configurable sync time, with a default of one hour. +## Git password authentication + +LDAP-enabled users can always authenticate with Git using their GitLab username +or email and LDAP password, even if password authentication for Git is disabled +in the application settings. + ## Configuration To enable LDAP integration you need to add your LDAP server settings in diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 4d3be0ab8f6..a88e67bfeb5 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -53,7 +53,9 @@ or in different cloud availability zones. > **Note:** GitLab recommends against choosing this HA method because of the complexity of managing DRBD and crafting automatic failover. This is - *compatible* with GitLab, but not officially *supported*. + *compatible* with GitLab, but not officially *supported*. If you are + an EE customer, support will help you with GitLab related problems, but if the + root cause is identified as DRBD, we will not troubleshoot further. Components/Servers Required: 2 servers/virtual machines (one active/one passive) diff --git a/doc/administration/index.md b/doc/administration/index.md new file mode 100644 index 00000000000..c8d28d8485a --- /dev/null +++ b/doc/administration/index.md @@ -0,0 +1,121 @@ +# Administrator documentation + +Learn how to administer your GitLab instance (Community Edition and +[Enterprise Editions](https://about.gitlab.com/gitlab-ee/)). +Regular users don't have access to GitLab administration tools and settings. + +GitLab.com is administered by GitLab, Inc., therefore, only GitLab team members have +access to its admin configurations. If you're a GitLab.com user, please check the +[user documentation](../user/index.html). + +## Installing and maintaining GitLab + +Learn how to install, configure, update, and maintain your GitLab instance. + +### Installing GitLab + +- [Install](../install/README.md): Requirements, directory structures, and installation methods. +- [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability. + +### Configuring GitLab + +- [Adjust your instance's timezone](../workflow/timezone.md): Customize the default time zone of GitLab. +- [Header logo](../customization/branded_page_and_email_header.md): Change the logo on all pages and email headers. +- [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page. +- [System hooks](../system_hooks/system_hooks.md): Notifications when users, projects and keys are changed. +- [Security](../security/README.md): Learn what you can do to further secure your GitLab instance. +- [Usage statistics, version check, and usage ping](../user/admin_area/settings/usage_statistics.md): Enable or disable information about your instance to be sent to GitLab, Inc. +- [Polling](polling.md): Configure how often the GitLab UI polls for updates. +- [GitLab Pages configuration](pages/index.md): Enable and configure GitLab Pages. +- [GitLab Pages configuration for installations from the source](pages/source.md): Enable and configure GitLab Pages on +[source installations](../install/installation.md#installation-from-source). +- [Environment variables](environment_variables.md): Supported environment variables that can be used to override their defaults values in order to configure GitLab. + +### Maintaining GitLab + +- [Raketasks](../raketasks/README.md): Perform various tasks for maintenance, backups, automatic webhooks setup, etc. + - [Backup and restore](../raketasks/backup_restore.md): Backup and restore your GitLab instance. +- [Operations](operations.md): Keeping GitLab up and running (clean up Redis sessions, moving repositories, Sidekiq Job throttling, Sidekiq MemoryKiller, Unicorn). +- [Restart GitLab](restart_gitlab.md): Learn how to restart GitLab and its components. + +#### Updating GitLab + +- [GitLab versions and maintenance policy](../policy/maintenance.md): Understand GitLab versions and releases (Major, Minor, Patch, Security), as well as update recommendations. +- [Update GitLab](../update/README.md): Update guides to upgrade your installation to a new version. +- [Downtimeless updates](../update/README.md#upgrading-without-downtime): Upgrade to a newer major, minor, or patch version of GitLab without taking your GitLab instance offline. +- [Migrate your GitLab CI/CD data to another version of GitLab](../migrate_ci_to_ce/README.md): If you have an old GitLab installation (older than 8.0), follow this guide to migrate your existing GitLab CI/CD data to another version of GitLab. + +### Upgrading or downgrading GitLab + +- [Upgrade from GitLab CE to GitLab EE](../update/README.md#upgrading-between-editions): learn how to upgrade GitLab Community Edition to GitLab Enterprise Editions. +- [Downgrade from GitLab EE to GitLab CE](../downgrade_ee_to_ce/README.md): Learn how to downgrade GitLab Enterprise Editions to Community Edition. + +### GitLab platform integrations + +- [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/): Integrate with [Mattermost](https://about.mattermost.com/), an open source, private cloud workplace for web messaging. +- [PlantUML](integration/plantuml.md): Create simple diagrams in AsciiDoc and Markdown documents +created in snippets, wikis, and repos. +- [Web terminals](integration/terminal.md): Provide terminal access to your applications deployed to Kubernetes from within GitLab's CI/CD [environments](../ci/environments.md#web-terminals). + +## User settings and permissions + +- [Libravatar](../customization/libravatar.md): Use Libravatar instead of Gravatar for user avatars. +- [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains. +- [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). +- [Authentication/Authorization](../topics/authentication/index.md#gitlab-administrators): Enforce 2FA, configure external authentication with LDAP, SAML, CAS and additional Omniauth providers. +- [Reply by email](reply_by_email.md): Allow users to comment on issues and merge requests by replying to notification emails. + - [Postfix for Reply by email](reply_by_email_postfix_setup.md): Set up a basic Postfix mail +server with IMAP authentication on Ubuntu, to be used with Reply by email. +- [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. + +## Project settings + +- [Container Registry](container_registry.md): Configure Container Registry with GitLab. +- [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. +- [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service. +- [Default labels](../user/admin_area/labels.html): Create labels that will be automatically added to every new project. + +### Repository settings + +- [Repository checks](repository_checks.md): Periodic Git repository checks. +- [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories. +- [Repository storage rake tasks](raketasks/storage.md): A collection of rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage. + +## Continuous Integration settings + +- [Enable/disable GitLab CI/CD](../ci/enable_or_disable_ci.md#site-wide-admin-setting): Enable or disable GitLab CI/CD for your instance. +- [GitLab CI/CD admin settings](../user/admin_area/settings/continuous_integration.md): Define max artifacts size and expiration time. +- [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). +- [Artifacts size and expiration](../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size): Define maximum artifacts limits and expiration date. +- [Register Shared and specific Runners](../ci/runners/README.md#registering-a-shared-runner): Learn how to register and configure Shared and specific Runners to your own instance. +- [Shared Runners pipelines quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota): Limit the usage of pipeline minutes for Shared Runners. +- [Enable/disable Auto DevOps](../topics/autodevops/index.md#enabling-auto-devops): Enable or disable Auto DevOps for your instance. + +## Git configuration options + +- [Custom Git hooks](custom_hooks.md): Custom Git hooks (on the filesystem) for when webhooks aren't enough. +- [Git LFS configuration](../workflow/lfs/lfs_administration.md): Learn how to configure LFS for GitLab. +- [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast. + +## Monitoring GitLab + +- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. + - [IP whitelist](monitoring/ip_whitelist.md): Monitor endpoints that provide health check information when probed. +- [Monitoring GitHub imports](monitoring/github_imports.md): GitLab's GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer. +- [Conversational Development (ConvDev) Index](../user/admin_area/monitoring/convdev.md): Provides an overview of your entire instance's feature usage. + +### Performance Monitoring + +- [GitLab Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring. +- [GitLab performance monitoring with InfluxDB](monitoring/performance/introduction.md): Configure GitLab and InfluxDB for measuring performance metrics. + - [InfluxDB Schema](monitoring/performance/influxdb_schema.md): Measurements stored in InfluxDB. +- [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. +- [GitLab performance monitoring with Grafana](monitoring/prometheus/index.md): Configure GitLab to visualize time series metrics through graphs and dashboards. +- [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. +- [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page. + +## Troubleshooting + +- [Debugging tips](troubleshooting/debug.md): Tips to debug problems when things go wrong +- [Log system](logs.md): Where to look for logs. +- [Sidekiq Troubleshooting](troubleshooting/sidekiq.md): Debug when Sidekiq appears hung and is not processing jobs. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 59305774c4a..7f1a0486749 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -157,6 +157,45 @@ steps below. 1. Save the file and [restart GitLab][] for the changes to take effect. +## Validation for dependencies + +> Introduced in GitLab 10.3. + +To disable [the dependencies validation](../ci/yaml/README.md#when-a-dependent-job-will-fail), +you can flip the feature flag from a Rails console. + +--- + +**In Omnibus installations:** + +1. Enter the Rails console: + + ```sh + sudo gitlab-rails console + ``` + +1. Flip the switch and disable it: + + ```ruby + Feature.enable('ci_disable_validates_dependencies') + ``` +--- + +**In installations from source:** + +1. Enter the Rails console: + + ```sh + cd /home/git/gitlab + RAILS_ENV=production sudo -u git -H bundle exec rails console + ``` + +1. Flip the switch and disable it: + + ```ruby + Feature.enable('ci_disable_validates_dependencies') + ``` + ## Set the maximum file size of the artifacts Provided the artifacts are enabled, you can change the maximum file size of the diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md new file mode 100644 index 00000000000..5592e0a9e9a --- /dev/null +++ b/doc/administration/monitoring/github_imports.md @@ -0,0 +1,101 @@ +# Monitoring GitHub imports + +>**Note:** +Available since [GitLab 10.2][14731]. + +The GitHub importer exposes various Prometheus metrics that you can use to +monitor the health and progress of the importer. + +## Import Duration Times + +| Name | Type | +|------------------------------------------|-----------| +| `github_importer_total_duration_seconds` | histogram | + +This metric tracks the total time spent (in seconds) importing a project (from +project creation until the import process finishes), for every imported project. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported projects + +| Name | Type | +|-------------------------------------|---------| +| `github_importer_imported_projects` | counter | + +This metric tracks the total number of projects imported over time. This metric +does not expose any labels. + +## Number of GitHub API calls + +| Name | Type | +|---------------------------------|---------| +| `github_importer_request_count` | counter | + +This metric tracks the total number of GitHub API calls performed over time, for +all projects. This metric does not expose any labels. + +## Rate limit errors + +| Name | Type | +|-----------------------------------|---------| +| `github_importer_rate_limit_hits` | counter | + +This metric tracks the number of times we hit the GitHub rate limit, for all +projects. This metric does not expose any labels. + +## Number of imported issues + +| Name | Type | +|-----------------------------------|---------| +| `github_importer_imported_issues` | counter | + +This metric tracks the number of imported issues across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported pull requests + +| Name | Type | +|------------------------------------------|---------| +| `github_importer_imported_pull_requests` | counter | + +This metric tracks the number of imported pull requests across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported comments + +| Name | Type | +|----------------------------------|---------| +| `github_importer_imported_notes` | counter | + +This metric tracks the number of imported comments across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported pull request review comments + +| Name | Type | +|---------------------------------------|---------| +| `github_importer_imported_diff_notes` | counter | + +This metric tracks the number of imported comments across all projects. + +The name of the project is stored in the `project` label in the format +`namespace/name` (e.g. `gitlab-org/gitlab-ce`). + +## Number of imported repositories + +| Name | Type | +|-----------------------------------------|---------| +| `github_importer_imported_repositories` | counter | + +This metric tracks the number of imported repositories across all projects. This +metric does not expose any labels. + +[14731]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14731 diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 0c63b0b59a7..7d47aaac299 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -58,6 +58,9 @@ Before proceeding with the Pages configuration, you will need to: so that your users don't have to bring their own. 1. (Only for custom domains) Have a **secondary IP**. +NOTE: **Note:** +If your GitLab instance and the Pages daemon are deployed in a private network or behind a firewall, your GitLab Pages websites will only be accessible to devices/users that have access to the private network. + ### DNS configuration GitLab Pages expect to run on their own virtual host. In your DNS server/provider diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index 5b6ee354887..136192191f9 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -58,7 +58,9 @@ Runs the following rake tasks: It will check that each component was setup according to the installation guide and suggest fixes for issues found. -You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). +You may also have a look at our Trouble Shooting Guides: +- [Trouble Shooting Guide (GitLab)](http://docs.gitlab.com/ee/README.html#troubleshooting) +- [Trouble Shooting Guide (Omnibus Gitlab)](http://docs.gitlab.com/omnibus/README.html#troubleshooting) **Omnibus Installation** diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index bac8fa4bd9d..6ec5baeb6e3 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -1,10 +1,43 @@ # Repository Storage Rake Tasks This is a collection of rake tasks you can use to help you list and migrate -existing projects from Legacy storage to the new Hashed storage type. +existing projects and attachments associated with it from Legacy storage to +the new Hashed storage type. You can read more about the storage types [here][storage-types]. +## Migrate existing projects to Hashed storage + +Before migrating your existing projects, you should +[enable hashed storage][storage-migration] for the new projects as well. + +This task will schedule all your existing projects and attachments associated with it to be migrated to the +**Hashed** storage type: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:migrate_to_hashed +``` + +**Source Installation** + +```bash +rake gitlab:storage:migrate_to_hashed + +``` + +You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. +There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** + +After it reaches zero, you can confirm every project has been migrated by running the commands bellow. +If you find it necessary, you can run this migration script again to schedule missing projects. + +Any error or warning will be logged in the sidekiq's log file. + +You only need the `gitlab:storage:migrate_to_hashed` rake task to migrate your repositories, but we have additional +commands below that helps you inspect projects and attachments in both legacy and hashed storage. + ## List projects on Legacy storage To have a simple summary of projects using **Legacy** storage: @@ -73,35 +106,73 @@ rake gitlab:storage:list_hashed_projects ``` -## Migrate existing projects to Hashed storage +## List attachments on Legacy storage -Before migrating your existing projects, you should -[enable hashed storage][storage-migration] for the new projects as well. +To have a simple summary of project attachments using **Legacy** storage: -This task will schedule all your existing projects to be migrated to the -**Hashed** storage type: +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:legacy_attachments +``` + +**Source Installation** + +```bash +rake gitlab:storage:legacy_attachments + +``` + +------ + +To list project attachments using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:migrate_to_hashed +gitlab-rake gitlab:storage:list_legacy_attachments ``` **Source Installation** ```bash -rake gitlab:storage:migrate_to_hashed +rake gitlab:storage:list_legacy_attachments ``` -You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. -There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** +## List attachments on Hashed storage -After it reaches zero, you can confirm every project has been migrated by running the commands above. -If you find it necessary, you can run this migration script again to schedule missing projects. +To have a simple summary of project attachments using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:hashed_attachments +``` -Any error or warning will be logged in the sidekiq log file. +**Source Installation** + +```bash +rake gitlab:storage:hashed_attachments + +``` + +------ + +To list project attachments using **Hashed** storage: + +**Omnibus Installation** +```bash +gitlab-rake gitlab:storage:list_hashed_attachments +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_hashed_attachments + +``` [storage-types]: ../repository_storage_types.md [storage-migration]: ../repository_storage_types.md#how-to-migrate-to-hashed-storage diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index bc9b6253f1a..21184fed6e9 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -9,7 +9,7 @@ mapping structure from the projects URLs: * Project's repository: `#{namespace}/#{project_name}.git` * Project's wiki: `#{namespace}/#{project_name}.wiki.git` - + This structure made simple to migrate from existing solutions to GitLab and easy for Administrators to find where the repository is stored. @@ -27,7 +27,7 @@ of load in big installations, and can be even worst if they are using any type o Last, for GitLab Geo, this storage type means we have to synchronize the disk state, replicate renames in the correct order or we may end-up with wrong repository or missing data temporarily. -This pattern also exists in other objects stored in GitLab, like issue Attachments, GitLab Pages artifacts, +This pattern also exists in other objects stored in GitLab, like issue Attachments, GitLab Pages artifacts, Docker Containers for the integrated Registry, etc. ## Hashed Storage @@ -62,9 +62,9 @@ you will never mistakenly restore a repository in the wrong project (considering ### How to migrate to Hashed Storage -In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select +In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select "_Create new projects using hashed storage paths_". - + To migrate your existing projects to the new storage type, check the specific [rake tasks]. [ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283 @@ -79,14 +79,14 @@ coverage status below. Note that things stored in an S3 compatible endpoint will not have the downsides mentioned earlier, if they are not prefixed with `#{namespace}/#{project_name}`, which is true for CI Cache and LFS Objects. -| Storable Object | Legacy Storage | Hashed Storage | S3 Compatible | GitLab Version | -| ----------------| -------------- | -------------- | ------------- | -------------- | +| Storable Object | Legacy Storage | Hashed Storage | S3 Compatible | GitLab Version | +| --------------- | -------------- | -------------- | ------------- | -------------- | | Repository | Yes | Yes | - | 10.0 | | Attachments | Yes | Yes | - | 10.2 | -| Avatars | Yes | No | - | - | +| Avatars | Yes | No | - | - | | Pages | Yes | No | - | - | | Docker Registry | Yes | No | - | - | -| CI Build Logs | No | No | - | - | -| CI Artifacts | No | No | - | - | +| CI Build Logs | No | No | - | - | +| CI Artifacts | No | No | Yes (EEP) | - | | CI Cache | No | No | Yes | - | -| LFS Objects | Yes | No | Yes (EEP) | - | +| LFS Objects | Yes | No | Yes (EEP) | - | diff --git a/doc/administration/repository_storages.md b/doc/administration/repository_storages.md index 9d41ba77f34..cf6de15743f 100644 --- a/doc/administration/repository_storages.md +++ b/doc/administration/repository_storages.md @@ -1,3 +1 @@ -# Repository storages - -This document was moved to a [new location](repository_storage_paths.md). +This document was moved to [another location](repository_storage_paths.md). diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index be538ea250a..83a714810c1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -163,6 +163,34 @@ separate Rails process to debug the issue: 1. In a new window, run `top`. It should show this ruby process using 100% CPU. Write down the PID. 1. Follow step 2 from the previous section on using gdb. +### GitLab: API is not accessible + +This often occurs when gitlab-shell attempts to request authorization via the +internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and +something in the check fails. There are many reasons why this may happen: + +1. Timeout connecting to a database (e.g., PostgreSQL or Redis) +1. Error in Git hooks or push rules +1. Error accessing the repository (e.g., stale NFS handles) + +To diagnose this problem, try to reproduce the problem and then see if there +is a unicorn worker that is spinning via `top`. Try to use the `gdb` +techniques above. In addition, using `strace` may help isolate issues: + +```shell +strace -tt -T -f -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt +``` + +If you cannot isolate which Unicorn worker is the issue, try to run `strace` +on all the Unicorn workers to see where the `/internal/allowed` endpoint gets +stuck: + +```shell +ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs strace -tt -T -f -s 1024 -o /tmp/unicorn.txt +``` + +The output in `/tmp/unicorn.txt` may help diagnose the root cause. + # More information * [Debugging Stuck Ruby Processes](https://blog.newrelic.com/2013/04/29/debugging-stuck-ruby-processes-what-to-do-before-you-kill-9/) diff --git a/doc/administration/troubleshooting/sidekiq.md b/doc/administration/troubleshooting/sidekiq.md index b71f8fabbc8..9d157720ad2 100644 --- a/doc/administration/troubleshooting/sidekiq.md +++ b/doc/administration/troubleshooting/sidekiq.md @@ -11,7 +11,7 @@ troubleshooting steps that will help you diagnose the bottleneck. debug steps with GitLab Support so the backtraces can be analyzed by our team. It may reveal a bug or necessary improvement in GitLab. -> **Note:** In any of the backtraces, be weary of suspecting cases where every +> **Note:** In any of the backtraces, be wary of suspecting cases where every thread appears to be waiting in the database, Redis, or waiting to acquire a mutex. This **may** mean there's contention in the database, for example, but look for one thread that is different than the rest. This other thread |