This implements the support for `needs:` keyword
as part of GitLab CI. That makes some of the jobs
to be run out of order.

Adds Job specific variables to facilitate specifying variables when
running manual jobs.

Using the sed script from
https://gitlab.com/gitlab-org/gitlab-ce/issues/59758

Partial CE backport of https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/10565

Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>

Currently, pipeline schedule worker is unstable because it's
sometimes killed by excessive memory consumption.
In order to improve the performance, we add the following fixes:

1. next_run_at is always real_next_run, which means the value
   always takes into account of worker's cron schedule
1. Remove exlusive lock. This is already covered by real_next_run
   change.
1. Use RunPipelineScheduleWorker for avoiding memory killer.
   Memory consumption is spread to the multiple sidekiq worker.

We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.

Add env_var and file as supported types for CI variables. Variables of
type file expose to users existing gitlab-runner behaviour - save
variable value into a temp file and set the path to this file in an ENV
var named after the variable key.

Resolves https://gitlab.com/gitlab-org/gitlab-ce/issues/46806.