summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Use CTEs for nested groups and authorizationsYorick Peterse2017-05-1751-464/+949
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit introduces the usage of Common Table Expressions (CTEs) to efficiently retrieve nested group hierarchies, without having to rely on the "routes" table (which is an _incredibly_ inefficient way of getting the data). This requires a patch to ActiveRecord (found in the added initializer) to work properly as ActiveRecord doesn't support WITH statements properly out of the box. Unfortunately MySQL provides no efficient way of getting nested groups. For example, the old routes setup could easily take 5-10 seconds depending on the amount of "routes" in a database. Providing vastly different logic for both MySQL and PostgreSQL will negatively impact the development process. Because of this the various nested groups related methods return empty relations when used in combination with MySQL. For project authorizations the logic is split up into two classes: * Gitlab::ProjectAuthorizations::WithNestedGroups * Gitlab::ProjectAuthorizations::WithoutNestedGroups Both classes get the fresh project authorizations (= as they should be in the "project_authorizations" table), including nested groups if PostgreSQL is used. The logic of these two classes is quite different apart from their public interface. This complicates development a bit, but unfortunately there is no way around this. This commit also introduces Gitlab::GroupHierarchy. This class can be used to get the ancestors and descendants of a base relation, or both by using a UNION. This in turn is used by methods such as: * Namespace#ancestors * Namespace#descendants * User#all_expanded_groups Again this class relies on CTEs and thus only works on PostgreSQL. The Namespace methods will return an empty relation when MySQL is used, while User#all_expanded_groups will return only the groups a user is a direct member of. Performance wise the impact is quite large. For example, on GitLab.com Namespace#descendants used to take around 580 ms to retrieve data for a particular user. Using CTEs we are able to reduce this down to roughly 1 millisecond, returning the exact same data. == On The Fly Refreshing Refreshing of authorizations on the fly (= when users.authorized_projects_populated was not set) is removed with this commit. This simplifies the code, and ensures any queries used for authorizations are not mutated because they are executed in a Rails scope (e.g. Project.visible_to_user). This commit includes a migration to schedule refreshing authorizations for all users, ensuring all of them have their authorizations in place. Said migration schedules users in batches of 5000, with 5 minutes between every batch to smear the load around a bit. == Spec Changes This commit also introduces some changes to various specs. For example, some specs for ProjectTeam assumed that creating a personal project would _not_ lead to the owner having access, which is incorrect. Because we also no longer refresh authorizations on the fly for new users some code had to be added to the "empty_project" factory. This chunk of code ensures that the owner's permissions are refreshed after creating the project, something that is normally done in Projects::CreateService.
* Merge branch 'allow_numeric_values_in_gitlab_ci_yml' into 'master' Kamil Trzciński2017-05-156-4/+30
|\ | | | | | | | | | | | | Allow numeric values in gitlab-ci.yml Closes #30017 See merge request !10607
| * Allow numeric values in gitlab-ci.ymlblackst0ne2017-05-076-4/+30
| |
* | Merge branch 'rename-column-concurrently-defaults' into 'master' Rémy Coutable2017-05-152-3/+11
|\ \ | | | | | | | | | | | | Fix adding defaults for concurrent column renames See merge request !11335
| * | Fix adding defaults for concurrent column renamesrename-column-concurrently-defaultsYorick Peterse2017-05-122-3/+11
| | | | | | | | | | | | | | | | | | By adding the default value _after_ adding the column we avoid updating all rows in a table, saving a lot of time and unnecessary work in the process.
* | | Merge branch 'add-balsamiq-integration-test' into 'master' Phil Hughes2017-05-158-48/+119
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Added balsamiq jasmine integration test Closes #31889 See merge request !11257
| * | | Added balsamiq jasmine integration testLuke "Jared" Bennett2017-05-158-48/+119
|/ / /
* | | Merge branch 'dturner/username' into 'master' Rémy Coutable2017-05-154-0/+8
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | add username field to push webhook Closes #32136 See merge request !11272
| * | | Add username field to push webhookDavid Turner2017-05-104-0/+8
| | | | | | | | | | | | | | | | Signed-off-by: David Turner <novalis@novalis.org>
* | | | Merge branch 'master' into 'master' Rémy Coutable2017-05-151-1/+1
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | Fix typo in code snippet to match text below See merge request !11101
| * | | | Fix typo in code snippet to match text belowSteve Hall2017-05-041-1/+1
| | | | |
* | | | | Merge branch 'patch-18' into 'master' Rémy Coutable2017-05-151-1/+1
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Fix the GitLab EE link See merge request !11021
| * | | | | Fix the GitLab EE linkSota Yamashita2017-05-021-1/+1
| | | | | |
* | | | | | Merge branch 'patch-8' into 'master' Rémy Coutable2017-05-151-1/+1
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | fix $CI_JOB_ID variable See merge request !10966
| * | | | | | fix $CI_JOB_ID variable Elan Ruusamäe2017-04-271-1/+1
| | | | | | | | | | | | | | | | | | | | | introduced with rename variables commit: a084437b5c1b7226ce64e74bab1972bb1dbd49e0
* | | | | | | Merge branch 'fix/support-commit-deltas-only-for-gitaly' into 'master' Robert Speicher2017-05-147-58/+99
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fetch commit deltas from Gitaly Closes gitaly#199 See merge request !11122
| * | | | | | | Add support for deltas_only under GitalyAhmad Sherif2017-05-137-58/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Closes gitaly#199
* | | | | | | | Merge branch 'use-common-vue-for-pipeline-schedule-bundles' into 'master' Filipa Lacerda2017-05-143-4/+7
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensure schedule_form and schedules_index both utilize common_vue bundle See merge request !11274
| * | | | | | | | ensure schedule_form and schedules_index both utilize common_vue bundleuse-common-vue-for-pipeline-schedule-bundlesMike Greiling2017-05-123-4/+7
| | |_|_|_|_|/ / | |/| | | | | |
* | | | | | | | Merge branch '26325-system-hooks' into 'master' Robert Speicher2017-05-1319-11/+230
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Backport New SystemHook: `repository_update` Closes #26325 See merge request !11140
| * | | | | | | | CodestyleGabriel Mazetto2017-05-131-1/+1
| | | | | | | | |
| * | | | | | | | Minor cosmetic fixes in hooks admin screenGabriel Mazetto2017-05-122-4/+4
| | | | | | | | |
| * | | | | | | | Documentation for repository_update_eventsGabriel Mazetto2017-05-121-1/+49
| | | | | | | | |
| * | | | | | | | ChangelogGabriel Mazetto2017-05-121-0/+4
| | | | | | | | |
| * | | | | | | | Update SystemHooks API to expose and handle new repository_update_eventsGabriel Mazetto2017-05-123-3/+5
| | | | | | | | |
| * | | | | | | | Make the new repository_update_events configurable in System Hooks UIGabriel Mazetto2017-05-126-2/+54
| | | | | | | | |
| * | | | | | | | Added repository_update hookGabriel Mazetto2017-05-1210-4/+117
| | | | | | | | |
* | | | | | | | | Update CHANGELOG.md for 9.1.4Felipe Artur2017-05-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | [ci skip]
* | | | | | | | | Update CHANGELOG.md for 9.1.4Felipe Artur2017-05-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | [ci skip]
* | | | | | | | | Merge branch 'dm-wait-for-requests' into 'master' Douwe Maan2017-05-122-8/+3
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Wait for requests after each Spinach scenario instead of feature See merge request !11349
| * | | | | | | | | Wait for requests after each Spinach scenario instead of featuredm-wait-for-requestsDouwe Maan2017-05-122-8/+3
| | | | | | | | | |
* | | | | | | | | | Update CHANGELOG.md for 9.1.4Felipe Artur2017-05-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [ci skip]
* | | | | | | | | | Update CHANGELOG.md for 9.1.4Felipe Artur2017-05-129-33/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [ci skip]
* | | | | | | | | | Merge branch 'add-problematic-mysql-examples' into 'master' Sid Sijbrandij2017-05-121-2/+5
|\ \ \ \ \ \ \ \ \ \ | |_|/ / / / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Link to a case where we can't support MySQL Closes #32181 See merge request !11334
| * | | | | | | | | Link to a case where we can't support MySQLadd-problematic-mysql-examplesYorick Peterse2017-05-121-2/+5
| | |/ / / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This updates the requirements guide to include a link to an issue where we discuss dropping MySQL support for nested groups. This should give users a better understanding of why we highly recommend them to use PostgreSQL instead. [ci skip]
* | | | | | | | | Merge branch 'sh-remove-trailing-comma' into 'master' Douwe Maan2017-05-121-1/+1
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove trailing comma in dependency linker to satisfy Rubocop See merge request !11348
| * | | | | | | | | Remove trailing comma in dependency linker to satisfy RubocopStan Hu2017-05-121-1/+1
| | | | | | | | | |
* | | | | | | | | | Merge branch '30975-fixed-top-nav-POC' into 'master' Annabel Dunstone Gray2017-05-1224-35/+70
|\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add fixed positioning to top level navbar Closes #30975 See merge request !10751
| * | | | | | | | | | Fix specsAnnabel Dunstone Gray2017-05-123-4/+4
| | | | | | | | | | |
| * | | | | | | | | | Clean up fixed navbarAnnabel Dunstone Gray2017-05-123-6/+9
| | | | | | | | | | |
| * | | | | | | | | | Fix specsAnnabel Dunstone Gray2017-05-1213-25/+30
| | | | | | | | | | |
| * | | | | | | | | | Fade out border on scrollAnnabel Dunstone Gray2017-05-124-11/+14
| | | | | | | | | | |
| * | | | | | | | | | Add animation to navbar borderAnnabel Dunstone Gray2017-05-121-2/+3
| | | | | | | | | | |
| * | | | | | | | | | Add nav border on scrollAnnabel Dunstone Gray2017-05-122-0/+15
| | | | | | | | | | |
| * | | | | | | | | | Remove trailing semicolonAnnabel Dunstone Gray2017-05-121-1/+1
| | | | | | | | | | |
| * | | | | | | | | | Add fixed positioning to top level navbarAnnabel Dunstone Gray2017-05-129-8/+16
| | | | | | | | | | |
* | | | | | | | | | | Merge branch 'sh-relax-backup-rake-spec' into 'master' Stan Hu2017-05-121-1/+1
|\ \ \ \ \ \ \ \ \ \ \ | |_|/ / / / / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Relax rake backup regex to handle CE and EE RCs See merge request !11345
| * | | | | | | | | | Relax rake backup regex to handle CE and EE RCssh-relax-backup-rake-specStan Hu2017-05-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See http://rubular.com/r/uTDlT85zW1
* | | | | | | | | | | Merge branch 'fix-conflict-resolution-with-corrupt-repos' into 'master' Douwe Maan2017-05-1215-200/+268
|\ \ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix conflict resolution from corrupted upstream Closes gitlab-ee#2128 See merge request !11298
| * | | | | | | | | | | Fix conflict resolution from corrupted upstreamfix-conflict-resolution-with-corrupt-reposSean McGivern2017-05-1215-200/+268
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I don't know why this happens exactly, but given an upstream and fork repository from a customer, both of which required GC, resolving conflicts would corrupt the fork so badly that it couldn't be cloned. This isn't a perfect fix for that case, because the MR may still need to be merged manually, but it does ensure that the repository is at least usable. My best guess is that when we generate the index for the conflict resolution (which we previously did in the target project), we obtain a reference to an OID that doesn't exist in the source, even though we already fetch the refs from the target into the source. Explicitly setting the source project as the place to get the merge index from seems to prevent repository corruption in this way.