<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/gitlab/gitlab-ce.git/spec/uploaders, branch scripts-differences</title>
<subtitle>gitlab.com: gitlab-org/gitlab-ce.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/'/>
<entry>
<title>Queries for Upload should be scoped by model</title>
<updated>2019-07-11T13:52:55+00:00</updated>
<author>
<name>Adam Hegyi</name>
<email>ahegyi@gitlab.com</email>
</author>
<published>2019-07-11T04:48:50+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=dfe906209e2238b82c84c9fb435498cae2f3d43e'/>
<id>dfe906209e2238b82c84c9fb435498cae2f3d43e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Use #filename when generating upload URLs</title>
<updated>2019-07-03T19:46:26+00:00</updated>
<author>
<name>Heinrich Lee Yu</name>
<email>heinrich@gitlab.com</email>
</author>
<published>2019-06-20T21:01:14+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=76b0518f334090294d4ad7db7be64846f6018e80'/>
<id>76b0518f334090294d4ad7db7be64846f6018e80</id>
<content type='text'>
We don't need to find the filename from the remote URL
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We don't need to find the filename from the remote URL
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix order-dependent spec failure in appearance_spec.rb</title>
<updated>2019-07-03T18:27:16+00:00</updated>
<author>
<name>Stan Hu</name>
<email>stanhu@gmail.com</email>
</author>
<published>2019-07-03T18:27:16+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=3b924c13460d4afe26c386633241d1362d10e824'/>
<id>3b924c13460d4afe26c386633241d1362d10e824</id>
<content type='text'>
When file_mover_spec.rb ran, it would initialize fog-aws with
`Fog::AWS::Storage::Real` service instead of `Fog::AWS::Storage::Mock`
because `Fog.mock!` was not called. Ensure that we use
`stub_uploads_object_storage` to prevent that from happening.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64083
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When file_mover_spec.rb ran, it would initialize fog-aws with
`Fog::AWS::Storage::Real` service instead of `Fog::AWS::Storage::Mock`
because `Fog.mock!` was not called. Ensure that we use
`stub_uploads_object_storage` to prevent that from happening.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64083
</pre>
</div>
</content>
</entry>
<entry>
<title>Support object storage at FileMover class</title>
<updated>2019-06-30T21:00:00+00:00</updated>
<author>
<name>Oswaldo Ferreira</name>
<email>oswaldo@gitlab.com</email>
</author>
<published>2019-06-28T21:28:52+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=114dd976426a26c19d5e2a350a2c41bda35ddf54'/>
<id>114dd976426a26c19d5e2a350a2c41bda35ddf54</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Persist tmp snippet uploads</title>
<updated>2019-06-17T14:25:40+00:00</updated>
<author>
<name>Oswaldo Ferreira</name>
<email>oswaldo@gitlab.com</email>
</author>
<published>2019-06-04T23:50:57+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=44e1915d4f0e20cf445196ccc7e1d279c75ef0ce'/>
<id>44e1915d4f0e20cf445196ccc7e1d279c75ef0ce</id>
<content type='text'>
It persist temporary personal snippets under
user/:id namespaces temporarily while creating
a upload record to track it. If an user gets removed
while it's still a tmp upload, it also gets removed.
If the tmp upload is sent, the upload gets moved to
personal_snippets/:id as before. The upload record
also gets updated to the new model type as well.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It persist temporary personal snippets under
user/:id namespaces temporarily while creating
a upload record to track it. If an user gets removed
while it's still a tmp upload, it also gets removed.
If the tmp upload is sent, the upload gets moved to
personal_snippets/:id as before. The upload record
also gets updated to the new model type as well.
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove legacy artifact related code</title>
<updated>2019-05-31T03:49:17+00:00</updated>
<author>
<name>Shinya Maeda</name>
<email>shinya@gitlab.com</email>
</author>
<published>2019-03-22T12:38:45+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=387a4f4b2cc9cffe2a21ef1060fe35c60d1ac769'/>
<id>387a4f4b2cc9cffe2a21ef1060fe35c60d1ac769</id>
<content type='text'>
We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix incorrect prefix used in new uploads for personal snippets</title>
<updated>2019-05-17T06:36:55+00:00</updated>
<author>
<name>Stan Hu</name>
<email>stanhu@gmail.com</email>
</author>
<published>2019-05-15T16:26:18+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=a1f44c1b4969847fa80d6c53d6bd70813d273d6c'/>
<id>a1f44c1b4969847fa80d6c53d6bd70813d273d6c</id>
<content type='text'>
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24550 fixed the
case where the wrong path on disk was being searched, but it
inadvertently ommitted the `/uploads/-/system` prefix when rendering the
Markdown for personal snippet uploads when they were stored directly in
object storage.

A personal snippet path is stored using FileUploader#upload_path.
The format for the path:

Local storage: :random_hex/:filename.
Object storage: personal_snippet/:id/:random_hex/:filename.

upload_paths represent the possible paths for a given identifier,
which will vary depending on whether the file is stored in local or
object storage. upload_path should match an element in upload_paths.

base_dir represents the path seen by the user in Markdown, and it
should always be prefixed with uploads/-/system.

store_dirs represent the paths that are actually used on disk. For
object storage, this should omit the prefix /uploads/-/system.

For example, consider the requested path
/uploads/-/system/personal_snippet/172/ff4ad5c2/file.png.

For local storage:

base_dir: uploads/-/system/personal_snippet/172
upload_path: ff4ad5c2/file.png
upload_paths: ["ff4ad5c2/file.png", "personal_snippet/172/ff4ad5c2/file.png"].
store_dirs: {1=&gt;"uploads/-/system/personal_snippet/172/ff4ad5c2",
             2=&gt;"personal_snippet/172/ff4ad5c2"}

For object storage:

upload_path: personal_snippet/172/ff4ad5c2/file.png

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61671
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24550 fixed the
case where the wrong path on disk was being searched, but it
inadvertently ommitted the `/uploads/-/system` prefix when rendering the
Markdown for personal snippet uploads when they were stored directly in
object storage.

A personal snippet path is stored using FileUploader#upload_path.
The format for the path:

Local storage: :random_hex/:filename.
Object storage: personal_snippet/:id/:random_hex/:filename.

upload_paths represent the possible paths for a given identifier,
which will vary depending on whether the file is stored in local or
object storage. upload_path should match an element in upload_paths.

base_dir represents the path seen by the user in Markdown, and it
should always be prefixed with uploads/-/system.

store_dirs represent the paths that are actually used on disk. For
object storage, this should omit the prefix /uploads/-/system.

For example, consider the requested path
/uploads/-/system/personal_snippet/172/ff4ad5c2/file.png.

For local storage:

base_dir: uploads/-/system/personal_snippet/172
upload_path: ff4ad5c2/file.png
upload_paths: ["ff4ad5c2/file.png", "personal_snippet/172/ff4ad5c2/file.png"].
store_dirs: {1=&gt;"uploads/-/system/personal_snippet/172/ff4ad5c2",
             2=&gt;"personal_snippet/172/ff4ad5c2"}

For object storage:

upload_path: personal_snippet/172/ff4ad5c2/file.png

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61671
</pre>
</div>
</content>
</entry>
<entry>
<title>Optimise upload path calls</title>
<updated>2019-05-14T16:01:48+00:00</updated>
<author>
<name>Sean McGivern</name>
<email>sean@gitlab.com</email>
</author>
<published>2019-05-11T12:06:44+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=79dd92c8189600f24a29fe2da8d391cdb021f8fd'/>
<id>79dd92c8189600f24a29fe2da8d391cdb021f8fd</id>
<content type='text'>
String#underscore isn't particularly slow, but it's possible for us to
call it many times in a users autocomplete request, with mostly-static
values ('User', 'Group', etc.). We can memoise this and save a
surprising amount of time (around 10% of the total request time in some
cases).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
String#underscore isn't particularly slow, but it's possible for us to
call it many times in a users autocomplete request, with mostly-static
values ('User', 'Group', etc.). We can memoise this and save a
surprising amount of time (around 10% of the total request time in some
cases).
</pre>
</div>
</content>
</entry>
<entry>
<title>Clean up CarrierWave's import/export files</title>
<updated>2019-05-07T08:27:50+00:00</updated>
<author>
<name>Stan Hu</name>
<email>stanhu@gmail.com</email>
</author>
<published>2019-04-18T12:10:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=ec341a2bbd81eac9639faf5c5e8e8b670f535096'/>
<id>ec341a2bbd81eac9639faf5c5e8e8b670f535096</id>
<content type='text'>
Unlike uploads that have been uploaded with Tempfile, the project
import/export archives are stored in a temporary cache directory and
remain there if:

1. Object storage is enabled
2. `move_to_store` is set to `true`.

CarrierWave will leave these files there until disk space runs out or a
clean step is run manually.

If `move_to_store` is set to `false`, CarrierWave will remove the files
after storing them. However, unlike a local file, with object storage,
the file is still copied, so setting `move_to_store` to `true`
doesn't buy us anything.

To ensure files are cleaned up, we can just inherit from the
GitlabUploader implementation of `move_to_store`, which returns `true`
if it's a local file, `false` otherwise.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60656
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Unlike uploads that have been uploaded with Tempfile, the project
import/export archives are stored in a temporary cache directory and
remain there if:

1. Object storage is enabled
2. `move_to_store` is set to `true`.

CarrierWave will leave these files there until disk space runs out or a
clean step is run manually.

If `move_to_store` is set to `false`, CarrierWave will remove the files
after storing them. However, unlike a local file, with object storage,
the file is still copied, so setting `move_to_store` to `true`
doesn't buy us anything.

To ensure files are cleaned up, we can just inherit from the
GitlabUploader implementation of `move_to_store`, which returns `true`
if it's a local file, `false` otherwise.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60656
</pre>
</div>
</content>
</entry>
<entry>
<title>Disable method replacement in avatar loading</title>
<updated>2019-04-30T04:29:25+00:00</updated>
<author>
<name>Stan Hu</name>
<email>stanhu@gmail.com</email>
</author>
<published>2019-04-30T00:07:42+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=25818bd7ae765422c934d0a32efb4ba353d11183'/>
<id>25818bd7ae765422c934d0a32efb4ba353d11183</id>
<content type='text'>
We've seen a significant performance penalty when using
`BatchLoader#__replace_with!`. This defines methods on the batch loader
that proxy to the 'real' object using send. The alternative is
`method_missing`, which is slower.  However, we've noticed that
`method_missing` can be faster if:

1. The objects being loaded have a large interface.
2. We don't call too many methods on the loaded object.

Avatar uploads meet both criteria above, so let's use the newly-released
feature in https://github.com/exAspArk/batch-loader/pull/45.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60903
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
We've seen a significant performance penalty when using
`BatchLoader#__replace_with!`. This defines methods on the batch loader
that proxy to the 'real' object using send. The alternative is
`method_missing`, which is slower.  However, we've noticed that
`method_missing` can be faster if:

1. The objects being loaded have a large interface.
2. We don't call too many methods on the loaded object.

Avatar uploads meet both criteria above, so let's use the newly-released
feature in https://github.com/exAspArk/batch-loader/pull/45.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60903
</pre>
</div>
</content>
</entry>
</feed>
