Contains a large list of files we expect to find for an Auto-DevOps
project.

Look up upstream commits once before queuing ProcessCommitWorkers

Closes #65464

See merge request gitlab-org/gitlab-ce!31871

Previously `ProjectCacheWorker` would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. `ProjectCacheWorker` would expire
three items:

1. Repository size: This only needs to be updated once per push.
2. Commit count: This only needs to be updated if the default branch
   is updated.
3. Project method caches: This only needs to be updated if the default
   branch changes, but only if certain files change (e.g. README,
   CHANGELOG, etc.).

Because the third item requires looking at the actual changes in the
commit deltas, we schedule one `ProjectCacheWorker` to handle the first
two cases, and schedule a separate `ProjectCacheWorker` for the third
case if it is needed. As a result, this brings down the number of
`ProjectCacheWorker` jobs from N to 2.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046

Instead of checking if a commit already exists in the upstream project
in its ProcessCommitWorker and bailing out if it does, we check the
existence of all commits in bulk in Git::BranchHooksService, so that we
can skip scheduling ProcessCommitWorker jobs for those commits
that already exist upstream entirely.

Enable DAG support by default

Closes #65457

See merge request gitlab-org/gitlab-ce!31814

Prior to 12.1, rebase status was looked up directly from Gitaly. In
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14417 , a DB
column was added to track the status instead. However, we couldn't stop
looking at the gitaly status immediately, since some rebases may been
running across the upgrade.

Now that we're in 12.3, it is safe to remove the direct-to-gitaly
lookup. This also happens to fix a 500 error that is seen when viewing
an MR for a fork where the source project has been removed.

We still look at the Gitaly status in the service, just in case Gitaly
and Sidekiq get out of sync - I assume this is possible, and it's a
relatively cheap check.

Since we atomically check and set `merge_requests.rebase_jid`, we
should never enqueue two `RebaseWorker` jobs in parallel.

- Adds UI to configure in group and project settings
- Removes notification configuration for users when
disabled at group or project level

Reduce Gitaly calls in PostReceive

Closes #65878

See merge request gitlab-org/gitlab-ce!31741

This toggles the ci_dag_support flag to be on by default.
This relies on ci_dag_limit_needs to be present to reduce
amount of inter-dependencies between jobs

Optimise dag processing

See merge request gitlab-org/gitlab-ce!31768