summaryrefslogtreecommitdiff
path: root/app/workers
Commit message (Collapse)AuthorAgeFilesLines
* Further remove code branches by database typeAndreas Brandl2019-07-291-2/+0
| | | | | | | | We dropped MySQL support and a lot of mysql specific code has been removed in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29608. This comes in from the other direction and removes any `if postgresql?` branches.
* Remove unused Clusters::RefreshServiceTiger2019-07-292-8/+2
| | | | | | Also removes all logic from ClusterConfigureWorker and ClusterProjectConfigureWorker, which are also no longer used.
* Validate the existence of archived traces before removing live tracesafe-archiving-for-tracesShinya Maeda2019-07-232-2/+2
| | | | | Often live traces are removed even though the archived trace doesn't exist. This commit checkes the existence strictly.
* Merge branch 'an-sidekiq-chaos' into 'master'Lin Jen-Shin2019-07-187-0/+75
|\ | | | | | | | | | | | | Adds chaos endpoints to Sidekiq Closes #64663 See merge request gitlab-org/gitlab-ce!30814
| * Adds chaos endpoints to Sidekiqan-sidekiq-chaosAndrew Newdigate2019-07-187-0/+75
| | | | | | | | | | This allows the chaos endpoints to be invoked in Sidekiq so that this environment can be tested for resilience.
* | Update usage ping cron behaviorbjk/usage_pingBen Kochie2019-07-181-0/+6
|/ | | | | | | * Splay the start by sleeping up to one minute. * Adjust sideikiq retries to try up to approximately three hours. * Randomize the minute fully within the hour. * Psudo-randomize the day of the week based on the instance UUID.
* Remove auto ssl feature flagsVladimir Shushlin2019-07-122-3/+3
| | | | | * remove feature flag for admin settings * remove feature flag for domain settings
* Add a rubocop for Rails.loggerMayra Cabrera2019-07-1018-18/+22
| | | | | | Suggests to use a JSON structured log instead Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/54102
* Add queue_size method to ApplicationWorkerdm-queue-mirror-jobs-in-batches-drain-ceDouwe Maan2019-07-051-0/+6
|
* Allow asynchronous rebase operations to be monitoredNick Thomas2019-07-041-0/+2
| | | | | | | | | This MR introduces tracking of the `rebase_jid` for merge requests. As with `merge_ongoing?`, `rebase_in_progress?` will now return true if a rebase is proceeding in sidekiq. After one release, we should remove the Gitaly-based lookup of rebases. It is much better to track this kind of thing via the database.
* Includes logic to persist namespace statisticsMayra Cabrera2019-07-024-0/+102
| | | | | | | | | | | | | | | | | | | | | - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
* Use project depended feature flag for pages sslpages-ssl-project-aware-feature-flagVladimir Shushlin2019-06-252-6/+4
| | | | | Also add ::Gitlab::LetsEncrypt.enabled? shortcut and simplify it a lot
* Renew Let's Encrypt certificatesVladimir Shushlin2019-06-243-0/+31
| | | | | | | | Add index for pages domain ssl auto renewal Add PagesDomain.needs_ssl_renewal scope Add cron worker for ssl renewal Add worker for ssl renewal Add pages ssl renewal worker queues settings
* Merge branch 'remove_create_deployment_from_build_success_worker' into 'master'Douwe Maan2019-06-241-12/+0
|\ | | | | | | | | | | | | Remove deprecated call in BuildSuccessWorker Closes #63622 See merge request gitlab-org/gitlab-ce!29958
| * Remove deprecated call in BuildSuccessWorkerremove_create_deployment_from_build_success_workerThong Kuah2019-06-241-12/+0
| | | | | | | | | | We now create_deployment on Ci::Build create. No need to call create_deployment again
* | Merge branch '60617-enable-project-cluster-jit' into 'master'Thong Kuah2019-06-241-2/+0
|\ \ | |/ |/| | | | | Enable JIT Kubernetes resource creation for project level clusters See merge request gitlab-org/gitlab-ce!29515
| * Enable project-level JIT resource creation60617-enable-project-cluster-jitTiger2019-06-181-2/+0
| | | | | | | | | | | | | | | | | | | | Previously this behaviour was only available to group and instance-level clusters, as some project clusters relied on Kubernetes credentials being passed through to the runner instead of having their resources managed by GitLab (which is not available when using JIT). These clusters have been migrated to unmanaged, so resources can be created on demand for the remaining managed clusters.
* | Merge branch 'ce-12014-incremental-es-wiki-updates' into 'master'Nick Thomas2019-06-191-8/+15
|\ \ | | | | | | | | | | | | Backport of 12014-incremental-es-wiki-updates See merge request gitlab-org/gitlab-ce!29531
| * | Backport of 12014-incremental-es-wiki-updatesce-12014-incremental-es-wiki-updatesMario de la Ossa2019-06-181-8/+15
| |/ | | | | | | Bringing in the DB migration and some light changes for CE classes
* | Add 2nd response for container api bulk deleteSteve Abrams2019-06-181-26/+3
|/ | | | | | | The bulk delete api endpoint for container registries can only be called once per hour. If a user calls the endpoint more than once per hour, they will now receive a 400 error with a descriptive message.
* Delete unauthorized Todos when project is privateissue_49897Felipe Artur2019-06-051-2/+2
| | | | | Delete Todos for guest users when project visibility level is updated to private.
* Ensure DB is writable before continuing jobs11925-geo-sidekiq-nodes-try-to-run-jobs-even-thought-db-is-readonlyAsh McKenzie2019-06-052-0/+4
| | | | | | | In the context of a Geo setup, some jobs can be running on a Geo secondary where the database is read-only and therefore we should guard against various jobs attempting to write.
* Merge branch 'introduce-auto-merge-process-worker' into 'master'Thong Kuah2019-06-043-8/+17
|\ | | | | | | | | Introduce sidekiq worker for auto merge process See merge request gitlab-org/gitlab-ce!28780
| * Introduce sidekiq worker for auto merge processintroduce-auto-merge-process-workerShinya Maeda2019-06-043-8/+17
| | | | | | | | | | As we have a central domain for auto merge process today, we should use a single worker for any auto merge process.
* | Merge branch ↵Kamil Trzciński2019-06-042-39/+28
|\ \ | |/ |/| | | | | | | | | | | | | 'set-real-next-run-at-for-preventing-duplciate-pipeline-creations' into 'master' Make pipeline schedule worker resilient Closes gitlab-com/gl-infra/production#805 and #61955 See merge request gitlab-org/gitlab-ce!28407
| * Make pipeline schedule worker resilientset-real-next-run-at-for-preventing-duplciate-pipeline-creationsShinya Maeda2019-06-032-39/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently, pipeline schedule worker is unstable because it's sometimes killed by excessive memory consumption. In order to improve the performance, we add the following fixes: 1. next_run_at is always real_next_run, which means the value always takes into account of worker's cron schedule 1. Remove exlusive lock. This is already covered by real_next_run change. 1. Use RunPipelineScheduleWorker for avoiding memory killer. Memory consumption is spread to the multiple sidekiq worker.
* | Abstract auto merge processesShinya Maeda2019-06-031-3/+4
|/ | | | | | | | | | | | We have one auto merge strategy today - Merge When Pipeline Succeeds. In order to add more strategies for Merge Train feature, we abstract the architecture to be more extensible. Removed arguments Fix spec
* Add wiki size to project statisticsPeter Marko2019-05-292-1/+5
|
* Add GitDeduplicationService for deduplication housekeepingjc-git-deduplication-serviceJohn Cai2019-05-211-1/+3
| | | | | GitDeduplicationService performs idempotent operations on deduplicated projects.
* Expire pipeline cache on finishmc/bug/pipeline-cache-invalidMatija Čupić2019-05-211-0/+1
|
* Remove remove_disabled_domains feature flagVladimir Shushlin2019-05-201-2/+0
|
* Stop configuring group clusters on creation60379-remove-ci-preparing-state-feature-flagTiger2019-05-161-1/+1
| | | | | | | | | | | Immediate configuration is not ideal for group and instance level clusters as projects that may never be deployed would still have Kubernetes namespaces and service accounts created for them. As of https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25586 we now create only the resources that are required for the project being deployed, at the time of deployment.
* Include MR information if possible when emailing notification of closing an ↵Michał Zając2019-05-161-1/+1
| | | | issue
* Change DetectRepositoryLanguagesWorker to not receive userDiego Silva2019-05-091-4/+3
| | | | Fixes #60425
* Add gitlab-managed option to clusters formMayra Cabrera2019-05-031-1/+1
| | | | | | | | When this option is enabled, GitLab will create namespaces and service accounts as usual. When disabled, GitLab wont create any project specific kubernetes resources Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/56557
* Use git_garbage_collect_worker to run pack_refsJan Provaznik2019-05-021-2/+9
| | | | | | | | PackRefs is not an expensive gitaly call - we want to call it more often (than as part of full `gc`) because it helps to keep number of refs files small - too many refs file may be a problem for deployments with slow storage.
* Remove disabled pages domainsVladimir Shushlin2019-04-302-0/+17
| | | | | Domain will be removed by verification worker after 1 week of being disabled
* DELETE clusters/:id/:application endpointThong Kuah2019-04-292-0/+18
| | | | Add endpoint to delete/uninstall a cluster application
* Services to uninstall cluster applicationThong Kuah2019-04-292-0/+21
| | | | + to monitor progress of uninstallation pod
* Merge branch 'lock-pipeline-schedule-worker' into 'master'Stan Hu2019-04-291-11/+17
|\ | | | | | | | | | | | | Prevent concurrent execution of PipelineScheduleWorker Closes gitlab-com/gl-infra/production#805 See merge request gitlab-org/gitlab-ce!27781
| * Prevent concurrent execution of PipelineScheduleWorkerShinya Maeda2019-04-281-11/+17
| | | | | | | | | | | | Currently, PipelineScheduleWorker is fired in a short period on our production server. We can stop this behavior by locking the execution thread with in_lock method.
* | Port changes for design management to CEBob Van Landuyt2019-04-291-1/+3
| | | | | | | | | | | | This ports the changes from https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/10462/ to CE
* | Add deployment events to chat notification servicesJason Goodman2019-04-262-0/+14
|/ | | | | This enables sending a chat message to Slack or Mattermost upon a successful, failed, or canceled deployment
* Merge branch ↵Kamil Trzciński2019-04-172-5/+3
|\ | | | | | | | | | | | | | | | | '60500-disable-jit-kubernetes-resource-creation-for-project-level-clusters' into 'master' Disable JIT Kubernetes resource creation for project level clusters Closes #60500 See merge request gitlab-org/gitlab-ce!27352
| * Disable JIT resource creation for project clusters60500-disable-jit-kubernetes-resource-creation-for-project-level-clustersTiger2019-04-162-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | JIT resource creation blocks deployments if a user is self-managing their cluster, as it will fail the build if unable to create a namespace and service account. Using a custom namespace and service account was previously supported for project level clusters, so we should preserve this functionality. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27352
* | Properly expire all pipeline caches when pipeline is deletedsh-fix-pipeline-delete-cachingStan Hu2019-04-151-50/+1
|/ | | | | | | | | | | When deleting a pipeline, only some of the cache structures were being expired, but not the full pipeline list. We have to synchronously schedule a pipeline cache expiration because the pipeline will be deleted if the Sidekiq expiration job picks it up. To do this, properly extract all the logic buried in the Sidekiq worker into a service, and then call the service. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60469
* Merge branch 'limit-amount-of-created-pipelines' into 'master'Grzegorz Bizon2019-04-111-12/+15
|\ | | | | | | | | Process at most 4 pipelines during push See merge request gitlab-org/gitlab-ce!27205
| * Process at most 4 pipelines during pushlimit-amount-of-created-pipelinesKamil Trzciński2019-04-101-12/+15
| | | | | | | | | | | | | | | | This adds a limitation that we will try to create pipeline for at most 4 first changes (branches and tags). This does not affect processing of Pipelines for Merge Requests, as each updated MR will have associated pipeline created.
* | Externalize strings in app/workersMartin Wortschack2019-04-103-16/+16
|/ | | | | | - email_receiver_worker - object_storage/migrate_uploads_worker - Update PO file
* Merge branch '43263-git-push-option-to-create-mr' into 'master'Nick Thomas2019-04-091-1/+1
|\ | | | | | | | | | | | | Git push options to create a merge request, set target_branch and set merge when pipeline succeeds Closes #53198 and #43263 See merge request gitlab-org/gitlab-ce!26752