| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
This makes BuildQueueService to force refresh runners
that are considered to have recent queue.
Such runners are the ones that connected within online
interval + time to expire runner cache.
|
| |\
| |
| |
| |
| |
| |
| | |
Resolve "Improve system notes for Zoom links"
Closes #65427
See merge request gitlab-org/gitlab-ce!31410
|
| | |
| |
| |
| |
| |
| |
| | |
changes: @user a Zoom call was added to this issue
into: @user added a Zoom call to this issue
Same concept appleis for "removed"
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31741 introduced
a regression where not all the right parameters would be passed into
`Ci::CreatePipelineService`. We fix this by breaking out the pipeline
parameters and reusing a method from `Gitlab::DataBuilder::Push`.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66196
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Look up upstream commits once before queuing ProcessCommitWorkers
Closes #65464
See merge request gitlab-org/gitlab-ce!31871
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Instead of checking if a commit already exists in the upstream project
in its ProcessCommitWorker and bailing out if it does, we check the
existence of all commits in bulk in Git::BranchHooksService, so that we
can skip scheduling ProcessCommitWorker jobs for those commits
that already exist upstream entirely.
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Previously `ProjectCacheWorker` would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. `ProjectCacheWorker` would expire
three items:
1. Repository size: This only needs to be updated once per push.
2. Commit count: This only needs to be updated if the default branch
is updated.
3. Project method caches: This only needs to be updated if the default
branch changes, but only if certain files change (e.g. README,
CHANGELOG, etc.).
Because the third item requires looking at the actual changes in the
commit deltas, we schedule one `ProjectCacheWorker` to handle the first
two cases, and schedule a separate `ProjectCacheWorker` for the third
case if it is needed. As a result, this brings down the number of
`ProjectCacheWorker` jobs from N to 2.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Enable DAG support by default
Closes #65457
See merge request gitlab-org/gitlab-ce!31814
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This toggles the ci_dag_support flag to be on by default.
This relies on ci_dag_limit_needs to be present to reduce
amount of inter-dependencies between jobs
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Prior to 12.1, rebase status was looked up directly from Gitaly. In
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14417 , a DB
column was added to track the status instead. However, we couldn't stop
looking at the gitaly status immediately, since some rebases may been
running across the upgrade.
Now that we're in 12.3, it is safe to remove the direct-to-gitaly
lookup. This also happens to fix a 500 error that is seen when viewing
an MR for a fork where the source project has been removed.
We still look at the Gitaly status in the service, just in case Gitaly
and Sidekiq get out of sync - I assume this is possible, and it's a
relatively cheap check.
Since we atomically check and set `merge_requests.rebase_jid`, we
should never enqueue two `RebaseWorker` jobs in parallel.
|
| | |_|/
|/| |
| | |
| | |
| | |
| | | |
- Adds UI to configure in group and project settings
- Removes notification configuration for users when
disabled at group or project level
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Reduce Gitaly calls in PostReceive
Closes #65878
See merge request gitlab-org/gitlab-ce!31741
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This commit reduces I/O load and memory utilization during PostReceive
for the common case when no project hooks or services are set up.
We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches
are pushed. We can reduce this overhead in the common case because we
observe that most new projects do not have any Web hooks or services,
especially when they are first created. Previously, `BaseHooksService`
unconditionally iterated through the last 20 commits of each ref to
build the `push_data` structure. The `push_data` structured was used in
numerous places:
1. Building the push payload in `EventCreateService`
2. Creating a CI pipeline
3. Executing project Web or system hooks
4. Executing project services
5. As the return value of `BaseHooksService#execute`
6. `BranchHooksService#invalidated_file_types`
We only need to generate the full `push_data` for items 3, 4, and 6.
Item 1: `EventCreateService` only needs the last commit and doesn't
actually need the commit deltas.
Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of
the parameters.
Item 5: The return value of `BaseHooksService#execute` also wasn't being
used anywhere.
Item 6: This is only used when pushing to the default branch, so if
many tags are pushed we can save significant I/O here.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878
Fic
|
| |\ \ \ \
| |_|/ /
|/| | |
| | | |
| | | | |
Optimise dag processing
See merge request gitlab-org/gitlab-ce!31768
|
| | |/ / |
|
| |\ \ \
| | | |
| | | |
| | | |
| | | | |
Expand variables only when needed
See merge request gitlab-org/gitlab-ce!31772
|
| | |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This makes us to expand variables only when needed,
instead of requesting all variables each time.
This specifically helps in situation when explicit name
of `environment: production` is used.
|
| | |/
|/|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
**Prevention of running 2 simultaneous updates**
Instead of using `RemoteMirror#update_status` and raise an error if
it's already running to prevent the same mirror being updated at the
same time we now use `Gitlab::ExclusiveLease` for that.
When we fail to obtain a lease in 3 tries, 30 seconds apart, we bail
and reschedule. We'll reschedule faster for the protected branches.
If the mirror already ran since it was scheduled, the job will be
skipped.
**Error handling: Remote side**
When an update fails because of a `Gitlab::Git::CommandError`, we
won't track this error in sentry, this could be on the remote side:
for example when branches have diverged.
In this case, we'll try 3 times scheduled 1 or 5 minutes apart.
In between, the mirror is marked as "to_retry", the error would be
visible to the user when they visit the settings page.
After 3 tries we'll mark the mirror as failed and notify the user.
We won't track this error in sentry, as it's not likely we can help
it.
The next event that would trigger a new refresh.
**Error handling: our side**
If an unexpected error occurs, we mark the mirror as failed, but we'd
still retry the job based on the regular sidekiq retries with
backoff. Same as we used to
The error would be reported in sentry, since its likely we need to do
something about it.
|
| |\ \
| |/
|/|
| |
| | |
Only expire branch cache once per push
See merge request gitlab-org/gitlab-ce!31653
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Whenever `PostReceive` is enqueued, `UpdateMergeRequestsWorker`
is enqueued and `MergeRequests::RefreshService` is called, it'll
check if the source branch of each MR asssociated to the push exists
or not via `MergeRequest#source_branch_exists?`. The said method will
call `Repository#branch_exists?` which is cached in `Rails.cache`.
When the cache contains outdated data and the source branch actually
exists, the `MergeRequests#RefreshService` job will close associated
MRs which is not correct.
The fix is to expire the branches cache of the project so we have
updated data during the post receive hook which will help in the
accuracy of the check if we need to close associated MRs or not.
|
| | |
| |
| |
| | |
Standardize punctuation and format
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
'master'
Fix :wiki_can_not_be_created_total counter
See merge request gitlab-org/gitlab-ce!31673
|
| | |/ |
|
| |\ \
| |/
|/|
| |
| | |
Backport EE changes to Members::BaseService
See merge request gitlab-org/gitlab-ce!31581
|
| | |
| |
| |
| |
| | |
EE made some small changes to this class, but these changes were not
backported to CE.
|
| |/
|
|
| |
Generalize wiki page counter for other page types to extend to.
|
| |
|
|
| |
- Original EE MR: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14827
|
| |
|
|
|
| |
- This will make it easy to identify the project even if admins change
the name of the project or move it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://gitlab.com/gitlab-org/gitlab-ce/issues/62971
Adds support to EnvironmentsController#metrics_dashboard
for the following params: group, title, y_label
These params are used to uniquely identify a panel on
the metrics dashboard.
Metrics are stored in several places, so this adds
utilities to find a specific panel from the database
or filesystem depending on the metric specified.
Also moves some shared utilities into separate classes,
notably default values and errors.
|
| |\
| |
| |
| |
| | |
Don't attempt to contact registry if it is disabled
See merge request gitlab-org/gitlab-ce!31553
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21679 moved the
deletion of registry tags outside of a transaction, but introduced an
issue where Sidekiq would attempt to contact the container registry
during project destruction even if it were disabled.
Relates to:
* https://gitlab.com/charts/gitlab/issues/1455
* https://gitlab.com/gitlab-org/gitlab-ce/issues/45941
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Kubernetes deployments on new clusters will now have
a separate namespace per project environment, instead
of sharing a single namespace for the project.
Behaviour of existing clusters is unchanged.
All new functionality is controlled by the
:kubernetes_namespace_per_environment feature flag,
which is safe to enable/disable at any time.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Closes #60024
- Change PrometheusClient.new to accept a base url instead of an
already created RestClient
- Use Gitlab::HTTP in PrometheusClient instead of creating RestClient
in PrometheusService
- Move http_options from PrometheusService to
PrometheusClient (follow_redirects: false)
- ensure that base urls don't have the trailing slash
- Created a `PrometheusClient#url` method that might not be strictly
required
- Change rescued exceptions from RestClient::* to
HTTParty::ResponseError where possible and StandardError for the
rest
|
| |\
| |
| |
| |
| | |
Extend PipelineProcessWorker to accept a list of builds
See merge request gitlab-org/gitlab-ce!31425
|
| | |
| |
| |
| |
| |
| |
| | |
This changes used worker from `BuildProcessWorker`
to `PipelineProcessWorker` to make pipeline
processing much simpler. We process `pipeline_id`,
based on some triggers.
|
| |\ \
| | |
| | |
| | |
| | | |
Add outbound setting for system hooks
See merge request gitlab-org/gitlab-ce!31177
|
| | |/
| |
| |
| |
| |
| |
| | |
This MR adds new application setting to network section
`allow_local_requests_from_system_hooks`. Prior to this change
system hooks were allowed to do local network requests by default
and we are adding an ability for admins to control it.
|
| |/
|
|
|
|
|
|
|
|
| |
Currently, some of the jobs with `needs:`
would be processed in stages, it means
that `when:` for such jobs would not be
respected.
This changes the behavior to have a separate
execution paths for jobs with `needs:`.
|
| |\
| |
| |
| |
| | |
Backport of https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3809
See merge request gitlab-org/gitlab-ce!31375
|
| | |
| |
| |
| | |
Introducing Docker Registry replication
|
| |\ \
| | |
| | |
| | |
| | | |
Add exclusive lease to mergeability check process
See merge request gitlab-org/gitlab-ce!31082
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Concurrent calls to UserMergeToRef RPC updating a single ref
can lead to an opaque fail that is being rescued at Gitaly.
So this commit adds an exclusive lease to the mergeability
check process with the key as the current MR ID.
|
| | |/
|/|
| |
| |
| |
| | |
This implements the support for `needs:` keyword
as part of GitLab CI. That makes some of the jobs
to be run out of order.
|
| | |
| |
| |
| |
| |
| | |
- Add to whitelist so that even if local requests from hooks and
services are not allowed, the prometheus manual configuration will
still succeed.
|
| |/ |
|
| |
|
|
|
|
|
| |
We should just ignore these errors and move along with the deletion
since the repositories are going to be trashed anyway.
Closes https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31164
|
| |\
| |
| |
| |
| |
| |
| |
| |
| | |
'63547-add-system-notes-for-when-a-zoom-call-was-added-removed-from-an-issue' into 'master'
Resolve "Add system notes for when a zoom call was added/removed from an issue"
Closes #63547
See merge request gitlab-org/gitlab-ce!30857
|
| | |
| |
| |
| |
| | |
Add a zoom link added / removed system note when a zoom link is being
added / removed to the issue description.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
In preparation for embedding specific metrics in issues
https://gitlab.com/gitlab-org/gitlab-ce/issues/62971,
this commit moves the BaseService for metrics dashboards
to a new services subdirectory. This is purely for the sake
of organization and maintainability.
|
| | |
| |
| |
| |
| | |
Extends the quick actions "commands applied" banner to show
the quick action preview text, but with everything in past tense.
|
