summaryrefslogtreecommitdiff
path: root/app/services/ci
Commit message (Collapse)AuthorAgeFilesLines
* New interruptible attribute supported in YAML parsing.Cédric Tabin2019-09-051-5/+15
| | | | | | | Since it is not possible to dynamically detect if a job is automatically cancellable or not, a this new attribute is necessary. Moreover, it let the maintainer of the repo to adjust the behaviour of the auto cancellation feature to match exactly what he needs.
* Merge branch 'security-add-job-activity-limit-ce' into 'master'GitLab Release Tools Bot2019-08-291-1/+2
|\ | | | | | | | | | | | | Introduce JobActivity limit for alive jobs Closes gitlab-ee#376 See merge request gitlab/gitlabhq!3339
| * Add active_jobs_limit to plans tableFabio Pitino2019-08-211-1/+2
| | | | | | | | | | | | | | This is a port from EE changes where we introduce a new limit for Plan model. https://dev.gitlab.org/gitlab/gitlab-ee/merge_requests/1182
* | Optimise build queue serviceoptimise-build-queue-serviceKamil Trzciński2019-08-221-0/+4
|/ | | | | | | | This makes BuildQueueService to force refresh runners that are considered to have recent queue. Such runners are the ones that connected within online interval + time to expire runner cache.
* Merge branch 'enable-dag-support-by-default' into 'master'Kamil Trzciński2019-08-161-2/+2
|\ | | | | | | | | | | | | Enable DAG support by default Closes #65457 See merge request gitlab-org/gitlab-ce!31814
| * Enable DAG support by defaultenable-dag-support-by-defaultKamil Trzciński2019-08-141-2/+2
| | | | | | | | | | | | This toggles the ci_dag_support flag to be on by default. This relies on ci_dag_limit_needs to be present to reduce amount of inter-dependencies between jobs
* | Optimise DAG processingoptimise-dag-processingKamil Trzciński2019-08-141-14/+13
|/
* Merge branch 'prefer-to-use-process-pipeline-worker' into 'master'Sean McGivern2019-08-051-5/+11
|\ | | | | | | | | Extend PipelineProcessWorker to accept a list of builds See merge request gitlab-org/gitlab-ce!31425
| * Extend PipelineProcessWorker to accept a list of buildsprefer-to-use-process-pipeline-workerKamil Trzciński2019-08-021-5/+11
| | | | | | | | | | | | | | This changes used worker from `BuildProcessWorker` to `PipelineProcessWorker` to make pipeline processing much simpler. We process `pipeline_id`, based on some triggers.
* | Properly process `needs:` with `when:`properly-process-all-needsKamil Trzciński2019-08-021-17/+29
|/ | | | | | | | | | Currently, some of the jobs with `needs:` would be processed in stages, it means that `when:` for such jobs would not be respected. This changes the behavior to have a separate execution paths for jobs with `needs:`.
* Add support for DAGKamil Trzciński2019-08-012-5/+37
| | | | | | This implements the support for `needs:` keyword as part of GitLab CI. That makes some of the jobs to be run out of order.
* Add Job specific variablesMatija Čupić2019-07-291-2/+2
| | | | | Adds Job specific variables to facilitate specifying variables when running manual jobs.
* Prefer `flat_map` over `map` + `flatten`Peter Leitzen2019-07-261-2/+2
| | | | | Convert several occurrences of `map` + `flatten` to `flat_map` where applicable.
* Validate the existence of archived traces before removing live tracesafe-archiving-for-tracesShinya Maeda2019-07-231-4/+24
| | | | | Often live traces are removed even though the archived trace doesn't exist. This commit checkes the existence strictly.
* Upgrade to Rails 5.2upgrade-rails-5-2-ceHeinrich Lee Yu2019-07-121-2/+2
| | | | Updates changed method names and fixes spec failures
* Add a rubocop for Rails.loggerMayra Cabrera2019-07-101-2/+2
| | | | | | Suggests to use a JSON structured log instead Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/54102
* Adds 15m bucket for CI job queueElliot Rushton2019-07-091-1/+1
|
* Refactor pipeline errors_messagerefactor-pipeline-errors_message-ceShinya Maeda2019-07-041-1/+1
| | | | Use the shared method in Ci::Pipeline
* Limit amount of JUnit tests returnedKamil Trzciński2019-07-031-1/+5
| | | | | | | | | | Currently, we do not cap amount of tests returned to frontend, thus in some extreme cases we can see a MBs of data stored in Redis. This adds an upper limit of 100 tests per-suite. We will continue showing the total counters correctly, but we will limit amount of tests that will be presented.
* Revert concurrent pipeline schedule creationrevert-concurrent-pipeline-schedule-creationShinya Maeda2019-06-181-1/+12
| | | | | | This commit reverts the previously introduced concurrent pipeline schedule creation which was a viable solution for mitigating inconsistent pipeline schedule by Sidekiq Memory Killer.
* Fix pipeline schedule when owner is nilfix-pipeline-schedule-owner-is-nilShinya Maeda2019-06-111-1/+1
| | | | Fixing the bug
* Make pipeline schedule worker resilientset-real-next-run-at-for-preventing-duplciate-pipeline-creationsShinya Maeda2019-06-031-0/+13
| | | | | | | | | | | | | Currently, pipeline schedule worker is unstable because it's sometimes killed by excessive memory consumption. In order to improve the performance, we add the following fixes: 1. next_run_at is always real_next_run, which means the value always takes into account of worker's cron schedule 1. Remove exlusive lock. This is already covered by real_next_run change. 1. Use RunPipelineScheduleWorker for avoiding memory killer. Memory consumption is spread to the multiple sidekiq worker.
* Add 60s & 5m monitor buckets for CI queuesElliot Rushton2019-05-221-1/+1
|
* Added filtering jobs by age to jobs/request API endpointDmitry Chepurovskiy2019-05-201-0/+5
|
* Fix schedule head pipeline updatefix-schedule-head-pipeline-update-methodShinya Maeda2019-05-061-7/+1
| | | | | | | Currently, schedule head pipeline update method which executed after pipeline creation does not take into account of merge reqeust pipelines. We should use dedicated `all_merge_requests` method in this case.
* Adds a way to start multiple manual jobs in stageMayra Cabrera2019-05-021-0/+29
| | | | | | | | | | - Adds an endpoint on PipelinesController - Adds a service that iterates over every build in a stage and plays it. - Includes 'play_manual' details on EntitySerializer - Builds a new Stage state: PlayManual. An stage can take this status if it has manual builds or an skipped, scheduled or manual status - Includes FE modifications and specs
* Fix environment automatic on_stop triggerShinya Maeda2019-04-301-5/+11
| | | | | | | Due to the nature of pipelines for merge requests, deployments.ref can be a merge request ref instead of a branch name. We support the environment auto-stop hook for this case
* Rewind IID on Ci::Pipelinesrewind-iid-on-pipelinesKamil Trzciński2019-04-161-0/+4
| | | | | | | If no pipeline is created we currently have IID gap. This is due to fact that we generate IID not on save, but rather ahead of time. This results, us, losing IIDs.
* Merge branch 'sh-fix-pipeline-delete-caching' into 'master'Lin Jen-Shin2019-04-152-2/+64
|\ | | | | | | | | | | | | Properly expire all pipeline caches when pipeline is deleted Closes #60469 See merge request gitlab-org/gitlab-ce!27334
| * Properly expire all pipeline caches when pipeline is deletedsh-fix-pipeline-delete-cachingStan Hu2019-04-152-2/+64
| | | | | | | | | | | | | | | | | | | | | | When deleting a pipeline, only some of the cache structures were being expired, but not the full pipeline list. We have to synchronously schedule a pipeline cache expiration because the pipeline will be deleted if the Sidekiq expiration job picks it up. To do this, properly extract all the logic buried in the Sidekiq worker into a service, and then call the service. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60469
* | Rescue and log errors raised when preparing builds60480-rescue-build-preparing-errorsTiger2019-04-151-3/+5
|/ | | | https://gitlab.com/gitlab-org/gitlab-ce/issues/60480
* Use Gitlab::PushOptions for `ci.skip` push optionLuke Duncalfe2019-04-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the raw push option Array was sent to Pipeline::Chain::Skip. This commit updates this class (and the chain of classes that pass the push option parameters from the API internal `post_receive` endpoint to that class) to treat push options as a Hash of options parsed by GitLab::PushOptions. The GitLab::PushOptions class takes options like this: -o ci.skip -o merge_request.create -o merge_request.target=branch and turns them into a Hash like this: { ci: { skip: true }, merge_request: { create: true, target: 'branch' } } This now how Pipeline::Chain::Skip is determining if the `ci.skip` push option was used.
* Clear pipeline status cache after destruction of pipelineStan Hu2019-03-261-0/+2
| | | | | | | | | `project.pipeline_status.has_status?` is cached, which can lead to Error 500s in the UI if the this condition is used to check whether a pipeline exists for a commit. We now expire the cache to ensure that the information is consistent. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/59453
* Create framework for build prerequisitesTiger2019-03-201-0/+25
| | | | | | | | | Introduces the concept of Prerequisites for a CI build. If a build has unmet prerequisites it will go through the :preparing state before being made available to a runner. There are no actual prerequisites yet, so current behaviour is unchanged.
* Persist source sha and target sha for merge pipelinesShinya Maeda2019-02-271-1/+3
| | | | source_sha and target_sha are used for merge request pipelines
* Merge branch 'fix-misspellings-app-comments' into 'master'Rémy Coutable2019-02-262-4/+4
| | | | | Fix misspellings in app/spec comments See merge request gitlab-org/gitlab-ce!25517
* Revert "Merge branch 'fix-misspellings-app-comments' into 'master'"Michael Kozono2019-02-252-4/+4
| | | | | This reverts commit 9202bbd129537a698b986e6295d0c783b5a84815, reversing changes made to 4b282e9ce1ae246c4538b3ede18d1380ea778029.
* Fix misspellings in app/spec commentsTakuya Noguchi2019-02-252-4/+4
| | | | Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
* Move ChatOps to CoreJames Fargher2019-02-201-0/+1
| | | | ChatOps used to be in the Ultimate tier.
* Fix a Ruby 2.4 incompatibility in Ci::CreatePipelineServiceRémy Coutable2019-02-121-2/+7
| | | | Signed-off-by: Rémy Coutable <remy@rymai.me>
* Reduce the diff with EE in Ci::CreatePipelineServiceRémy Coutable2019-02-041-3/+11
| | | | Signed-off-by: Rémy Coutable <remy@rymai.me>
* Reduce diff with EE in Ci::PipelineTriggerServiceRémy Coutable2019-02-011-0/+10
| | | | Signed-off-by: Rémy Coutable <remy@rymai.me>
* Revert removing overridden method from build serviceGrzegorz Bizon2019-01-251-1/+5
|
* Make it clear that pipeline can process processablesGrzegorz Bizon2019-01-251-7/+7
|
* Simplify pipeline processing serviceGrzegorz Bizon2019-01-251-5/+1
|
* Segregate interface require to make CI/CD entity processableGrzegorz Bizon2019-01-251-1/+1
| | | | | | | This commit segregates interface that is require to make CI/CD entity processable, like `Ci::Build`. With this change it is not clear and explicit what methods need to be implement to pass an object to pipeline processing service.
* Remove expired artifacts periodicallyShinya Maeda2019-01-241-0/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rename Introduce Destroy expired job artifacts service Revert a bit Add changelog Use expired Improve Fix spec Fix spec Use bang for destroy Introduce iteration limit Update comment Simplify more Refacor Remove unnecessary thing Fix comments Fix coding offence Make loop helper exception free
* Disable audit event logging for pipeline destructionStan Hu2019-01-101-2/+0
| | | | | | | | | | | AuditEventService isn't equipped to handle logging of the destruction of entities such as CI pipelines. It's a project-level event that operates on a pipeline. The current log doesn't even indicate that the pipeline is being destroyed. This is a CE backport of https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9105. We're removing the auditing call because it breaks the EE implementation.
* Add config_options|variables to BuildMetadatakamil-refactor-ci-builds-v5Kamil Trzciński2019-01-041-1/+1
| | | | | | | | | | | These are data columns that store runtime configuration of build needed to execute it on runner and within pipeline. The definition of this data is that once used, and when no longer needed (due to retry capability) they can be freely removed. They use `jsonb` on PostgreSQL, and `text` on MySQL (due to lacking support for json datatype on old enough version).
* Add support for Git push options, specifically ci.skipJonathon Reinhart2018-12-311-1/+2
| | | | | | | | | | | | | | | | | gitlab-org/gitlab-shell!166 added support for collecting push options from the environment, and passing them along to the /internal/post_receive API endpoint. This change handles the new push_options JSON element in the payload, and passes them on through to the GitPushService and GitTagPushService services. Futhermore, it adds support for the first push option, ci.skip. With this change, one can use 'git push -o ci.skip' to skip CI pipe execution. Note that the pipeline is still created, but in the "skipped" state, just like with the 'ci skip' commit message text. Implements #18667