Add rake task to clean orphan artifact filestc-rake-orphan-artifacts

This adds the rake task rake
gitlab:cleanup:orphan_job_artifact_files. This rake task cleans all
orphan job artifact files it can find on disk.

 It performs a search on the complete folder of all artifacts on
disk. Then it filters out all the job artifact ID for which it could
not find a record with matching ID in the database. For these, the
file is deleted from disk.

2 files changed, 54 insertions, 0 deletions

diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md
index 5394e6dd763..5bd6cc81362 100644
--- a/doc/administration/geo/replication/troubleshooting.md
+++ b/doc/administration/geo/replication/troubleshooting.md
@@ -504,6 +504,15 @@ To resolve this, run the following command:
 sudo gitlab-rake geo:db:refresh_foreign_tables
 ```
 
+## Expired artifacts
+
+If you notice for some reason there are more artifacts on the Geo
+secondary node than on the Geo primary node, you can use the rake task
+to [cleanup orphan artifact files](../../../raketasks/cleanup.md#remove-orphan-artifact-files).
+
+On a Geo **secondary** node, this command will also clean up all Geo
+registry record related to the orphan files on disk.
+
 ## Fixing common errors
 
 This section documents common errors reported in the Admin UI and how to fix them.
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index f5c788af578..f880f31c39e 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -92,3 +92,48 @@ I, [2018-08-02T10:26:47.598424 #45087]  INFO -- : Looking for orphaned remote up
 I, [2018-08-02T10:26:47.753131 #45087]  INFO -- : Moved to lost and found: @hashed/6b/DSC_6152.JPG -> lost_and_found/@hashed/6b/DSC_6152.JPG
 I, [2018-08-02T10:26:47.764356 #45087]  INFO -- : Moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg -> lost_and_found/@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg
 ```
+
+## Remove orphan artifact files
+
+When you notice there are more job artifacts files on disk than there
+should be, you can run:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files
+```
+
+This command:
+
+- Scans through the entire artifacts folder.
+- Checks which files still have a record in the database.
+- If no database record is found, the file is deleted from disk.
+
+By default, this task does not delete anything but shows what it can
+delete. Run the command with `DRY_RUN=false` if you actually want to
+delete the files:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false
+```
+
+You can also limit the number of files to delete with `LIMIT`:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100`
+```
+
+This will only delete up to 100 files from disk. You can use this to
+delete a small set for testing purposes.
+
+If you provide `DEBUG=1`, you'll see the full path of every file that
+is detected as being an orphan.
+
+If `ionice` is installed, the tasks uses it to ensure the command is
+not causing too much load on the disk. You can configure the niceness
+level with `NICENESS`. Below are the valid levels, but consult
+`man 1 ionice` to be sure.
+
+- `0` or `None`
+- `1` or `Realtime`
+- `2` or `Best-effort` (default)
+- `3` or `Idle`