summaryrefslogtreecommitdiff
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
* Standardize remote_ip and path keys for auth.log and api_json.logStan Hu2019-08-203-3/+19
| | | | | | | | Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses `remote_ip` and `path` for the same fields. Let's standardize these namings to make it easier for people working with the data. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167
* Merge branch 'bvl-mr-commit-note-counter' into 'master'Douwe Maan2019-08-201-4/+4
|\ | | | | | | | | Count notes for commits and merge requests See merge request gitlab-org/gitlab-ce!31912
| * Count comments on notes and merge requestsBob Van Landuyt2019-08-161-4/+4
| | | | | | | | | | This extends our existing `Gitlab::UsageDataCounters::NoteCounter` to also count notes on commits and merge requests
* | Allow measurement for Sidekiq jobs taking > 2.5sAndrew Newdigate2019-08-201-1/+5
| | | | | | | | Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/66319.
* | Merge branch 'sh-fix-issues-api-gitaly-nplusone' into 'master'Sean McGivern2019-08-202-3/+8
|\ \ | | | | | | | | | | | | | | | | | | Fix Gitaly N+1 calls with listing issues/MRs via API Closes #66202 See merge request gitlab-org/gitlab-ce!31938
| * | Fix Gitaly N+1 calls with listing issues/MRs via APIsh-fix-issues-api-gitaly-nplusoneStan Hu2019-08-172-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In GitLab 9.0, https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9661 removed the `subscribed` flag from the API when the user requested a list of issues or merge requests since calculating this value triggers extensive Markdown processing. In GitLab 12.0 via a4fbf39e, we accidentally reintroduced this performance regression by changing `IssueBasic` to `Issue` in `entities.rb`. This showed up as a Gitaly N+1 issue since the Markdown processing would attempt to extract a commit if it detected a regex that matched a commit. We restore the prior behavior by once again removing the `subscribed` flag for the bulk list of issues and merge requests and add a test to ensure they aren't reintroduced. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66202
* | | Use ActiveModel's type instead of virtusPeter Leitzen2019-08-201-7/+0
| | | | | | | | | | | | | | | | | | The virtus project has been discontinued: https://github.com/solnic/virtus/commit/a6f896984
* | | Add missing SAST environment variablesCameron Swords2019-08-201-0/+3
| | |
* | | Add support for sentry_extra_data in exceptionsAlex Kalderimis2019-08-191-1/+16
| | | | | | | | | | | | | | | This allows exceptions to advertise their support for sentry and provide structured data.
* | | Merge branch '64251-branch-name-set-cache' into 'master'Robert Speicher2019-08-192-0/+120
|\ \ \ | | | | | | | | | | | | | | | | Cache branch and tag names as Redis sets See merge request gitlab-org/gitlab-ce!30476
| * | | Cache branch and tag names as Redis setsNick Thomas2019-08-162-0/+120
| | | | | | | | | | | | | | | | | | | | | | | | This allows us to check inclusion for the *_exists? methods without downloading the full list of branch names, which is over 100KiB in size for gitlab-ce at the moment.
* | | | Merge branch '39-count-unique-users-for-more-accurate-smau-reporting' into ↵Nick Thomas2019-08-191-2/+2
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'master' Allow UsageData.count to use count_by: See merge request gitlab-org/gitlab-ce!30770
| * | | | Allow UsageData.count to use count_by:Ash McKenzie2019-07-181-2/+2
| | | | |
* | | | | Clean Sidekiq metrics from multiproc dir on startAleksei Lipniagov2019-08-191-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After moving the multiproc dir cleanup into `config.ru`:`warmup`, we stopped cleaning Sidekiq metrics dir which is not correct. This MR intended to fix that. More details: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31668
* | | | | Fix pipelines not always being created after a pushsh-fix-pipelines-not-being-createdStan Hu2019-08-171-2/+0
| |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31741 introduced a regression where not all the right parameters would be passed into `Ci::CreatePipelineService`. We fix this by breaking out the pipeline parameters and reusing a method from `Gitlab::DataBuilder::Push`. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66196
* | | | Merge branch 'legacy-attachments-migrate-fix' into 'master'Michael Kozono2019-08-164-0/+202
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | Migrate legacy uploads rake tasks See merge request gitlab-org/gitlab-ce!29409
| * | | | Add rake tasks for migrating leacy uploadslegacy-attachments-migrate-fixJarka Košanová2019-08-164-0/+202
| | | | | | | | | | | | | | | | | | | | | | | | | - move uploads created by AttachmentUploader - handle also files created for legacy_diff_notes
* | | | | Expire project caches once per push instead of once per refStan Hu2019-08-161-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously `ProjectCacheWorker` would be scheduled once per ref, which would generate unnecessary I/O and load on Sidekiq, especially if many tags or branches were pushed at once. `ProjectCacheWorker` would expire three items: 1. Repository size: This only needs to be updated once per push. 2. Commit count: This only needs to be updated if the default branch is updated. 3. Project method caches: This only needs to be updated if the default branch changes, but only if certain files change (e.g. README, CHANGELOG, etc.). Because the third item requires looking at the actual changes in the commit deltas, we schedule one `ProjectCacheWorker` to handle the first two cases, and schedule a separate `ProjectCacheWorker` for the third case if it is needed. As a result, this brings down the number of `ProjectCacheWorker` jobs from N to 2. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
* | | | | Remove Security Dashboard feature flagrossfuhrman2019-08-161-3/+1
| |_|_|/ |/| | | | | | | | | | | This removes the group_overview_security_dashboard feature flag
* | | | Merge branch 'mc/feature/pipeline-tracking-config-ce' into 'master'Kamil Trzciński2019-08-162-2/+6
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add `needs:` CI config option CE Closes gitlab-ee#12334 See merge request gitlab-org/gitlab-ce!31346
| * | | | Port changes from EEmc/feature/pipeline-tracking-config-ceMatija Čupić2019-08-042-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Ports changes from https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/12343
* | | | | Merge branch 'fix/commits-api-empty-refname' into 'master'Nick Thomas2019-08-161-1/+1
|\ \ \ \ \ | |_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | fix handling of empty ref_name parameter string in commits api Closes #64745 See merge request gitlab-org/gitlab-ce!31687
| * | | | fix handling of empty ref_name parameter string in commits apiArmin Hohenegger2019-08-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when params[:ref_name] is set to "" by passing an empty query parameter to the api it is evaluated as false by the || operator. The use of active support core extensions presence method fixes the original implemantation. https://guides.rubyonrails.org/active_support_core_extensions.html#presence
* | | | | Merge branch '24705-multi-selection-for-delete-on-registry-page' into 'master'Thong Kuah2019-08-151-0/+7
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Resolve "Multi selection for delete on registry page" Closes #24705 See merge request gitlab-org/gitlab-ce!30837
| * | | | | Validates tag names and tags#bulk_destroyGiorgenes Gelatti2019-07-301-0/+7
| | | | | |
* | | | | | Support query parameters in metrics embedsSarah Yasonik2019-08-152-13/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://gitlab.com/gitlab-org/gitlab-ce/issues/62971 Adds support for embedding specific charts from the metrics dashboard. Expected parameters are dashboard, title, group, and y_label.
* | | | | | Squash project templates on updateHordur Freyr Yngvason2019-08-152-14/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As per https://gitlab.com/gitlab-org/gitlab-ce/issues/46043, project templates should be squashed before updating, so that repositories created from these templates don't include the full history of the backing repository.
* | | | | | Removes db/fixtures from database filesMayra Cabrera2019-08-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will avoid Danger to suggest a database review for files inside db/fixtures
* | | | | | Allow disabling group/project email notificationsBrett Walker2019-08-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Adds UI to configure in group and project settings - Removes notification configuration for users when disabled at group or project level
* | | | | | Merge branch 'sh-fix-discussions-api-perf' into 'master'Douglas Barbosa Alexandre2019-08-151-10/+16
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Eliminate many Gitaly calls in discussions API Closes #65957 See merge request gitlab-org/gitlab-ce!31834
| * | | | | | Eliminate many Gitaly calls in discussions APIStan Hu2019-08-151-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, the API to retrieve discussions from merge requests often generated hundreds of Gitaly calls to determine whether a system note should be shown to the user. It did this by: 1. Rendering the Markdown 2. Extracting cross-references from the Markdown 3. For cross-references that were commits, a Gitaly FindCommit RPC would be issued to validate that the commit exists. The last step is unnecessary because we don't need to display a commit if the user doesn't have access to the project in the first place. `RendersNotes#prepare_notes_for_rendering` is already used in `MergeRequestsController`, which is why we don't see N+1 Gitaly calls there. We use it here to optimize the note redaction process. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65957
* | | | | | | Merge branch 'fix-broken-ee-master-prepend-todos-api' into 'master'Stan Hu2019-08-151-0/+2
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds EE prepend, which for some reason was missing See merge request gitlab-org/gitlab-ce!31851
| * | | | | | | Adds EE prepend, which for some reason was missingfix-broken-ee-master-prepend-todos-apiAlex Kalderimis2019-08-151-0/+2
| | | | | | | |
* | | | | | | | Merge branch '65278-livesum-puma-phase' into 'master'Kamil Trzciński2019-08-151-3/+0
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove :puma_phase metrics See merge request gitlab-org/gitlab-ce!31773
| * | | | | | | | Remove :puma_phase metrics65278-livesum-puma-phaseAleksei Lipniagov2019-08-141-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't use phase restarts, as we use `preload_app`: https://github.com/puma/puma/blob/master/README.md#clustered-mode `:puma_phase` values will always be zero.
* | | | | | | | | Remove prepend_if_ee from CE codebaseLuke Duncalfe2019-08-151-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was added in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31789
* | | | | | | | | CE-specific changes to allow design Todos13494-todo-s-not-rendering-when-there-is-a-design-management-related-todo-ceLuke Duncalfe2019-08-151-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | CE-specific changes for: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/15129 Co-Authored-By: Alex Kalderimis <akalderimis@gitlab.com> Co-Authored-By: Luke Duncalfe <lduncalfe@eml.cc>
* | | | | | | | | Make ruby 2.6 the officially supported versionHordur Freyr Yngvason2019-08-141-1/+1
| | | | | | | | |
* | | | | | | | | Make use of Gitlab::KubernetesJoão Cunha2019-08-142-2/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - refactor Knative and Prometheus
* | | | | | | | | Migrates Snowplow backend from EE to CEJeremy Jackson2019-08-142-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This introduces several changes, but these are all just ported from the EE project.
* | | | | | | | | Track page views for cycle analytics show pageAdam Hegyi2019-08-142-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a new counter 'cycle_analytics_views' to the usage data metrics to count the page views for cycle analytics show page.
* | | | | | | | | Merge branch 'limit-amount-of-needs' into 'master'Grzegorz Bizon2019-08-141-1/+19
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add `ci_dag_limit_needs` See merge request gitlab-org/gitlab-ce!31803
| * | | | | | | | | Add `ci_dag_limit_needs` feature flaglimit-amount-of-needsKamil Trzciński2019-08-141-1/+19
| |/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This makes to limit `needs:` to 5 by default. Allow to increase the limit to 50 with disable of FF.
* | | | | | | | | Merge branch 'sh-optimize-commit-deltas-post-receive' into 'master'Nick Thomas2019-08-141-2/+3
|\ \ \ \ \ \ \ \ \ | |/ / / / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reduce Gitaly calls in PostReceive Closes #65878 See merge request gitlab-org/gitlab-ce!31741
| * | | | | | | | Reduce Gitaly calls in PostReceivesh-optimize-commit-deltas-post-receiveStan Hu2019-08-121-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit reduces I/O load and memory utilization during PostReceive for the common case when no project hooks or services are set up. We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches are pushed. We can reduce this overhead in the common case because we observe that most new projects do not have any Web hooks or services, especially when they are first created. Previously, `BaseHooksService` unconditionally iterated through the last 20 commits of each ref to build the `push_data` structure. The `push_data` structured was used in numerous places: 1. Building the push payload in `EventCreateService` 2. Creating a CI pipeline 3. Executing project Web or system hooks 4. Executing project services 5. As the return value of `BaseHooksService#execute` 6. `BranchHooksService#invalidated_file_types` We only need to generate the full `push_data` for items 3, 4, and 6. Item 1: `EventCreateService` only needs the last commit and doesn't actually need the commit deltas. Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of the parameters. Item 5: The return value of `BaseHooksService#execute` also wasn't being used anywhere. Item 6: This is only used when pushing to the default branch, so if many tags are pushed we can save significant I/O here. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878 Fic
* | | | | | | | | Merge branch 'require-needs-to-be-present' into 'master'Grzegorz Bizon2019-08-144-20/+47
|\ \ \ \ \ \ \ \ \ | |_|_|/ / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Require `needs:` to be present Closes #65839 See merge request gitlab-org/gitlab-ce!31761
| * | | | | | | | Require `needs:` to be presentrequire-needs-to-be-presentKamil Trzciński2019-08-134-20/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the `needs:` logic to require that all jobs to be present. Instead of skipping do fail the pipeline creation if `needs:` dependency is not found.
* | | | | | | | | Add usage pings for source code pushesIgor2019-08-132-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Source Code Usage Ping for Create SMAU
* | | | | | | | | Merge branch 'bump_helm_kubectl_gitlab' into 'master'Douglas Barbosa Alexandre2019-08-131-2/+2
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bump Helm to 2.14.3 and kubectl to 1.11.10 See merge request gitlab-org/gitlab-ce!31716
| * | | | | | | | | Bump Helm to 2.14.3 and kubectl to 1.11.10bump_helm_kubectl_gitlabThong Kuah2019-08-131-2/+2
| | |_|_|_|/ / / / | |/| | | | | | |