| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
Transform `CancelledError` into `JobRetry::Skip`
|
| |
|
|
|
|
|
|
|
|
| |
This makes:
- very shallow `Middleware::Monitor` to only request tracking
of sidekiq jobs,
- `SidekiqStatus::Monitor` to be responsible to maintain persistent
connection to receive messages,
- `SidekiqStatus::Monitor` to always use structured logging
and instance variables
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a middleware to track all threads
for running jobs.
This makes sidekiq to watch for redis-delivered notifications.
This makes be able to send notification to interrupt
running sidekiq jobs.
This does not take into account any native code,
as `Thread.raise` generates exception once the control gets
back to Ruby.
The separate measure should be taken to interrupt gRPC, shellouts,
or anything else that escapes Ruby.
|
| |
|
|
|
|
|
|
| |
Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses
`remote_ip` and `path` for the same fields. Let's standardize these
namings to make it easier for people working with the data.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167
|
| |\
| |
| |
| |
| | |
Update qa/Dockerfile to be built from the project root context
See merge request gitlab-org/gitlab-ce!31533
|
| | |
| |
| |
| |
| | |
For the QA tests to use the new injection methods, we must require the
initializer and ensure that the "constantize" method is available.
|
| |/
|
|
|
|
|
| |
After moving the multiproc dir cleanup into `config.ru`:`warmup`, we
stopped cleaning Sidekiq metrics dir which is not correct.
This MR intended to fix that. More details:
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31668
|
| |\
| |
| |
| |
| |
| |
| |
| |
| | |
'47003-user-onboarding-replace-current-email-confirmation-flow-with-a-soft-email-confirmation-flow' into 'master'
Soft email confirmation flow
Closes #47003
See merge request gitlab-org/gitlab-ce!31245
|
| | |
| |
| |
| | |
to 30 days
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
'46548-open-source-alternative-to-recaptcha-for-gitlab-com-registration' into 'master'
Open source alternative to reCAPTCHA for GitLab.com registration
See merge request gitlab-org/gitlab-ce!31625
|
| | | |
| | |
| | |
| | |
| | |
| | | |
With a time treshold of 4 seconds
and a firstname and lastname honeypot
input fields when signing up
|
| |\ \ \
| | | |
| | | |
| | | |
| | | | |
Elasticsearch versioned schema for Snippet
See merge request gitlab-org/gitlab-ce!31465
|
| | |/ / |
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| | |
When we hit our app with the initial request, in `warmup`,
some metrics already being created as well as corresponding files.
If we do `multiproc_file_dir` cleanup after that, we delete the files
from the dir while keeping them in memory which leads to the incorrect
behavior: the metric is being updated in in-memory, while is not present
in the db, not sent to Prometheus as the result.
|
| |/
|
|
|
|
|
|
|
|
| |
This will help identify Sidekiq jobs that invoke excessive number of
filesystem access.
The timing data is stored in `RequestStore`, but this is only active
within the middleware and is not directly accessible to the Sidekiq
logger. However, it is possible for the middleware to modify the job
hash to pass this data along to the logger.
|
| |\
| |
| |
| |
| |
| |
| | |
Add support for Content-Security-Policy
Closes #65330
See merge request gitlab-org/gitlab-ce!31402
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A nonce-based Content-Security-Policy thwarts XSS attacks by allowing
inline JavaScript to execute if the script nonce matches the header
value. Rails 5.2 supports nonce-based Content-Security-Policy headers,
so provide configuration to enable this and make it work.
To support this, we need to change all `:javascript` HAML filters to the
following form:
```
= javascript_tag nonce: true do
:plain
...
```
We use `%script` throughout our HAML to store JSON and other text, but
since this doesn't execute, browsers don't appear to block this content
from being used and require the nonce value to be present.
|
| |/
|
|
|
|
| |
These were disabled in production mode, but that also broke the rest of
the performance bar. As they were only enabled in development mode, we
can just remove them for now.
|
| |
|
| |
This reverts merge request !31379
|
| |\
| |
| |
| |
| | |
Support X_if_ee methods for QA tests
See merge request gitlab-org/gitlab-ce!31379
|
| | |
| |
| |
| |
| | |
For the QA tests to use the new injection methods, we must require the
initializer and ensure that the "constantize" method is available.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, both InfluxSampler and RubySampler were relying on the
`GC::Profiler.total_time` data which is the sum over the list
of captured GC events. Also, both samplers asynchronously called
`GC::Profiler.clear` which led to incorrect metric data because
each sampler has the wrong assumption it is the only object who calls
`GC::Profiler.clear` and thus could rely on the gathered results between
such calls.
We should ensure that `GC::Profiler.total_time` is called only in one
place making it possible to rely on accumulated data between such wipes.
Also, we need to track the amount of profiler reports we lost.
|
| |/
|
|
| |
Introducing Docker Registry replication
|
| |
|
|
|
|
|
|
|
|
|
| |
This adds the methods prepend_if_ee, extend_if_ee, and include_if_ee
that can be used to inject EE specific modules in EE.
These methods are exposed as an initializer that is loaded as soon as
possible. For tests that use fast_spec_helper.rb we must load this
initializer manually, as the Rails environment is not loaded. This is
not the most pretty setup, but unfortunately there is no alternative
that we can use.
|
| |
|
|
|
|
|
|
|
| |
1. The output isn't great. It can be hard to find hotspots and, even
when you do find them, to find why those are hotspots.
2. It uses some jQuery-specific frontend code which we can remove now
that we don't have this any more.
3. It's only possible to profile the initial request, not any subsequent
AJAX requests.
|
| |\ |
|
| | |\
| | |
| | |
| | |
| | | |
Do not allow localhost url redirection in GitHub Integration
See merge request gitlab/gitlabhq!3188
|
| | | | |
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Replace peek-pg with our own implementation
Closes #44441
See merge request gitlab-org/gitlab-ce!31187
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This uses an ActiveRecord subscriber to get queries and calculate the
total query time from that. This means that the total will always be
consistent with the queries in the table. It does however mean that we
could potentially miss some queries that don't go through ActiveRecord.
Making this change also allows us to unify the response JSON a little
bit, making the frontend slightly simpler as a result.
|
| |/ / /
| | |
| | |
| | |
| | | |
This adds diirect monitoring for sidekiq metrics. This is done via
sidekiq middleware and a sampler to pull from sidekiqs api.
|
| | | |
| | |
| | |
| | |
| | | |
Changes all calls to data_source_exists? to table_exists?
since that is the intent of these calls
|
| |\ \ \
| |/ /
|/| |
| | |
| | | |
Backport of https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14597
See merge request gitlab-org/gitlab-ce!30520
|
| | | |
| | |
| | |
| | |
| | | |
This is the first part of Docker Registry replication
for secondary Geo node.
|
| |/ /
| |
| |
| |
| |
| | |
This will help diagnose the source of excessive I/O from Rugged
calls. To implement this, we need to obtain the full list of arguments
sent to each request method.
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Remove dead mysql code
Closes #63191
See merge request gitlab-org/gitlab-ce!29608
|
| | | |
| | |
| | |
| | | |
None of this code can be reached any more, so it can all be removed
|
| |/ /
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is no longer needed with Rails 5.2. opclass is the attribute used
per https://github.com/rails/rails/pull/19090/files.
Now that we've removed the monkey patch and restored the Rails schema
dumper, it appears Rails has dropped the inclusion of `using: :btree` as
well (https://github.com/rails/rails/pull/27981).
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64529
|
| | | |
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Add Rugged calls and duration to API and Rails logs
Closes #64676
See merge request gitlab-org/gitlab-ce!30871
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This adds `rugged_duration_ms` and `rugged_calls` fields to
`api_json.log` and `production_json.log`. This will make it easier to
identify performance issues caused by excessive I/O.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64676
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | | |
Use relative worker identifier for metrics (instead of Process.pid) and
identify when Unicorn/Puma/Sidekiq is used.
Previously, it was assumed that all metrics are gathered from Unicorn
due to hardcoded implementation which was incorrect.
|
| |\ \ \
| |/ /
|/| |
| | |
| | |
| | |
| | | |
Fix inconsistency in Redis performance bar stats
Closes #64707
See merge request gitlab-org/gitlab-ce!30866
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
peek-redis resets its counters at the start of an ActionController
notification (`start_processing.action_controller`), which causes it to
miss some Redis queries that precede it, such as the database load
balancer and Rack Attack queries. This produces inconsistencies in the
performance bar between the number of calls and their durations with the
actual calls in the detailed view.
We fix this by getting rid of peek-redis in favor of consolidating all
logic into the `RedisDetailed` view, which tracks Redis queries using
`RequestStore`. This has the nice property of removing thread-specific
counters as well.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64707
|
| |\ \ \
| |/ /
|/| |
| | |
| | | |
Add EE-only class instrumentation
See merge request gitlab-org/gitlab-ce!30802
|
| | | | |
|
| |/ / |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
By default, httpclient (and hence anything that uses rack-oauth2)
ignores the system-wide SSL certificate configuration in favor of its
own `cacert.pem`. This makes it impossible to use custom certificates
without patching that file. Until
https://github.com/nahi/httpclient/pull/386 is merged, we work around
this limitation by forcing the `HTTPClient` SSL store to use the default
system configuration.
Closes https://gitlab.com/charts/gitlab/issues/1436
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Use transactions in JS feature specs
Closes #60207
See merge request gitlab-org/gitlab-ce!27496
|
| | | |
| | |
| | |
| | |
| | | |
Uses Rails transactional tests instead of DatabaseCleaner
transaction strategy because that doesn't work with JS tests
|
