<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/gitlab/gitlab-ce.git/lib, branch background-migrations</title>
<subtitle>gitlab.com: gitlab-org/gitlab-ce.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/'/>
<entry>
<title>Support for post deployment migrations</title>
<updated>2016-10-31T11:54:48+00:00</updated>
<author>
<name>Yorick Peterse</name>
<email>yorickpeterse@gmail.com</email>
</author>
<published>2016-10-27T12:36:53+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=83c8241160ed48ab066e2c5bd58d0914a745197c'/>
<id>83c8241160ed48ab066e2c5bd58d0914a745197c</id>
<content type='text'>
These are regular Rails migrations that are executed by default. A user
can opt-out of these migrations by setting an environment variable
during the deployment process.

Fixes gitlab-org/gitlab-ce#22133
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
These are regular Rails migrations that are executed by default. A user
can opt-out of these migrations by setting an environment variable
during the deployment process.

Fixes gitlab-org/gitlab-ce#22133
</pre>
</div>
</content>
</entry>
<entry>
<title>Ignore teaspoon tasks in production environment</title>
<updated>2016-10-29T11:22:32+00:00</updated>
<author>
<name>winniehell</name>
<email>git@winniehell.de</email>
</author>
<published>2016-10-29T11:22:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=8a9c7b1e04461f3dffa156a56bc25e77eda2fd9c'/>
<id>8a9c7b1e04461f3dffa156a56bc25e77eda2fd9c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'dz-internal-api-fullpath' into 'master'</title>
<updated>2016-10-28T21:36:40+00:00</updated>
<author>
<name>Dmitriy Zaporozhets</name>
<email>dmitriy.zaporozhets@gmail.com</email>
</author>
<published>2016-10-28T21:36:40+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=b7d3fc15a45a7f3ca0a293a23edf7e74ad80aa78'/>
<id>b7d3fc15a45a7f3ca0a293a23edf7e74ad80aa78</id>
<content type='text'>

Make internal api work with full repo path and name

## What does this MR do?

Make internal api work with full repo path and name

## Why was this MR needed?

So we can pass full repository path on filesystem from gitlab-shell instead of extracted one. We need this for nested groups support where project is can be nested under several groups. 

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/2772

See merge request !7148</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

Make internal api work with full repo path and name

## What does this MR do?

Make internal api work with full repo path and name

## Why was this MR needed?

So we can pass full repository path on filesystem from gitlab-shell instead of extracted one. We need this for nested groups support where project is can be nested under several groups. 

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/2772

See merge request !7148</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'adam-fix-labels-find-or-create' into 'master'</title>
<updated>2016-10-28T15:01:59+00:00</updated>
<author>
<name>Douwe Maan</name>
<email>douwe@gitlab.com</email>
</author>
<published>2016-10-28T15:01:59+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=44cbfeaba87b2b659f22c802920004d6dff1b53a'/>
<id>44cbfeaba87b2b659f22c802920004d6dff1b53a</id>
<content type='text'>

Pass user instance to Labels::FindOrCreateService or skip_authorization: true

## What does this MR do?

It fixes a bug described in #23694 when `project.owner` was passed to `Labels::FindOrCreateService`. `Labels::FindOrCreateService` expected a user instance and `project.owner` may return a group as well. This MR makes sure that we either pass a user instance or `skip_authorization: true`.

## Are there points in the code the reviewer needs to double check?

- places where we pass `skip_authorization: true`

## Does this MR meet the acceptance criteria?

- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Fixes #23694

See merge request !7093</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

Pass user instance to Labels::FindOrCreateService or skip_authorization: true

## What does this MR do?

It fixes a bug described in #23694 when `project.owner` was passed to `Labels::FindOrCreateService`. `Labels::FindOrCreateService` expected a user instance and `project.owner` may return a group as well. This MR makes sure that we either pass a user instance or `skip_authorization: true`.

## Are there points in the code the reviewer needs to double check?

- places where we pass `skip_authorization: true`

## Does this MR meet the acceptance criteria?

- Tests
  - [x] Added for this feature/bug
  - [ ] All builds are passing
- [ ] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Fixes #23694

See merge request !7093</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'use-optimistic-locking' into 'master'</title>
<updated>2016-10-28T14:41:24+00:00</updated>
<author>
<name>Stan Hu</name>
<email>stanhu@gmail.com</email>
</author>
<published>2016-10-28T14:41:24+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=d306b0d7c2c1f9384382c2a90a9d7c43bd20573c'/>
<id>d306b0d7c2c1f9384382c2a90a9d7c43bd20573c</id>
<content type='text'>

Use optimistic locking

## What does this MR do?
Removes the usage of pessimistic locking in favor of optimistic which is way cheaper and doesn't block database operation.

Since this is very simple change it should be safe. If we receive `StaleObjectError` message we will reload object a retry operations in lock.

However, I still believe that we need this one: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7005 as this will reduce a load on Database and FS.
This changes a behavior from:

### Pesimistic locking (previous behavior)

#### For updating
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. enqueue: (use: transition :created -&gt; :pending)
5. [state_machine] we are in  state created, we can go to pending
6. [state_machine] ci_pipeline.status = created
7. [state_machine] ci_pipeline.save
8. [state_machine] after_transition: (if for success): PipelineSuccessWorker on Sidekiq
9. release DB lock

#### If no update is required
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. we are in pending, we can't transition to pending, because it's forbidden
5. release DB lock

### Optimistic locking (implemented by this MR)

#### For updating
1. latest_build_status
2. enqueue: (use `transition :created -&gt; :pending`)
3. [state_machine] we are in state created, we can go to pending
4. [state_machine] ci_pipeline.status = created
5. [state_machine] ci_pipeline.save
6. [state_machine] [save] where(lock_version: ci_pipeline.lock_version).update_all(status: :created, updated_at: Time.now)
7. [state_machine] [save] unless we_updated_row then raise ObjectInconsistentError

#### If no update is required
1. we update ci_pipeline
2. latest_build_status
3. we are in pending, we can't transition to pending, because it's forbidden

## Why was this MR needed?
We have been seeing a number of problems when we migrated Pipeline/Build processing to Sidekiq. Especially we started seeing a lot of blocking queries.

We used a pessimistic locking which doesn't seem to be required. This effectively allows us to fix our issues with blocked queries by using more efficient method of operation.

## What are the relevant issue numbers?
Issues: https://gitlab.com/gitlab-com/infrastructure/issues/623 and https://gitlab.com/gitlab-com/infrastructure/issues/584, but also there's a bunch of Merge Requests that try to improve behavior of scheduled jobs.

cc @pcarranza @yorickpeterse @stanhu

See merge request !7040</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

Use optimistic locking

## What does this MR do?
Removes the usage of pessimistic locking in favor of optimistic which is way cheaper and doesn't block database operation.

Since this is very simple change it should be safe. If we receive `StaleObjectError` message we will reload object a retry operations in lock.

However, I still believe that we need this one: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7005 as this will reduce a load on Database and FS.
This changes a behavior from:

### Pesimistic locking (previous behavior)

#### For updating
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. enqueue: (use: transition :created -&gt; :pending)
5. [state_machine] we are in  state created, we can go to pending
6. [state_machine] ci_pipeline.status = created
7. [state_machine] ci_pipeline.save
8. [state_machine] after_transition: (if for success): PipelineSuccessWorker on Sidekiq
9. release DB lock

#### If no update is required
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. we are in pending, we can't transition to pending, because it's forbidden
5. release DB lock

### Optimistic locking (implemented by this MR)

#### For updating
1. latest_build_status
2. enqueue: (use `transition :created -&gt; :pending`)
3. [state_machine] we are in state created, we can go to pending
4. [state_machine] ci_pipeline.status = created
5. [state_machine] ci_pipeline.save
6. [state_machine] [save] where(lock_version: ci_pipeline.lock_version).update_all(status: :created, updated_at: Time.now)
7. [state_machine] [save] unless we_updated_row then raise ObjectInconsistentError

#### If no update is required
1. we update ci_pipeline
2. latest_build_status
3. we are in pending, we can't transition to pending, because it's forbidden

## Why was this MR needed?
We have been seeing a number of problems when we migrated Pipeline/Build processing to Sidekiq. Especially we started seeing a lot of blocking queries.

We used a pessimistic locking which doesn't seem to be required. This effectively allows us to fix our issues with blocked queries by using more efficient method of operation.

## What are the relevant issue numbers?
Issues: https://gitlab.com/gitlab-com/infrastructure/issues/623 and https://gitlab.com/gitlab-com/infrastructure/issues/584, but also there's a bunch of Merge Requests that try to improve behavior of scheduled jobs.

cc @pcarranza @yorickpeterse @stanhu

See merge request !7040</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'clean-up-issue_spec.js' into 'master'</title>
<updated>2016-10-28T13:37:19+00:00</updated>
<author>
<name>Rémy Coutable</name>
<email>remy@rymai.me</email>
</author>
<published>2016-10-28T13:37:19+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=2672f44e25b6b2638b850208a1ef4a97983f28ed'/>
<id>2672f44e25b6b2638b850208a1ef4a97983f28ed</id>
<content type='text'>

Replace static fixture by generated one in issue_spec.js

## What does this MR do?

- clean up `issue_spec.js`
- introduce an alternative approach to #19445
  - rename `rake teaspoon` to `rake teaspoon:tests`
  - introduce `rake teaspoon:fixtures` which generates fixtures using RSpec
  - introduce `rake teaspoon` which runs `rake teaspoon:fixtures` and `rake teaspoon:tests`

## Why was this MR needed?

- many duplications
- missing existence checks
- missing conditions
- static fixtures don't match real views

## Reasoning

I want to explain some of my decisions here, so that they stay visible for future discussions.

### Why not HAML?

- same number of HAML templates as number of fixtures (many input files)
- embedded logic less readable
- can not be rendered by JavaScript (because of inline Ruby)

### Why RSpec?

- real controllers for fixtures
- spys available for mocking
- easily report failed fixture generations

### Why not magic_lamp? (#19445)

- introduces another dependency/tool
- needs to run a server concurrently to teaspoon
- makes it harder to use a JavaScript test runner
- static HTML files serve faster

See merge request !6059</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

Replace static fixture by generated one in issue_spec.js

## What does this MR do?

- clean up `issue_spec.js`
- introduce an alternative approach to #19445
  - rename `rake teaspoon` to `rake teaspoon:tests`
  - introduce `rake teaspoon:fixtures` which generates fixtures using RSpec
  - introduce `rake teaspoon` which runs `rake teaspoon:fixtures` and `rake teaspoon:tests`

## Why was this MR needed?

- many duplications
- missing existence checks
- missing conditions
- static fixtures don't match real views

## Reasoning

I want to explain some of my decisions here, so that they stay visible for future discussions.

### Why not HAML?

- same number of HAML templates as number of fixtures (many input files)
- embedded logic less readable
- can not be rendered by JavaScript (because of inline Ruby)

### Why RSpec?

- real controllers for fixtures
- spys available for mocking
- easily report failed fixture generations

### Why not magic_lamp? (#19445)

- introduces another dependency/tool
- needs to run a server concurrently to teaspoon
- makes it harder to use a JavaScript test runner
- static HTML files serve faster

See merge request !6059</pre>
</div>
</content>
</entry>
<entry>
<title>Refactor storage path extraction from full repo path</title>
<updated>2016-10-28T12:55:55+00:00</updated>
<author>
<name>Dmitriy Zaporozhets</name>
<email>dmitriy.zaporozhets@gmail.com</email>
</author>
<published>2016-10-28T12:55:55+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=440604ad1ba67dcbdd23633765b9140fae4bd4b9'/>
<id>440604ad1ba67dcbdd23633765b9140fae4bd4b9</id>
<content type='text'>
Signed-off-by: Dmitriy Zaporozhets &lt;dmitriy.zaporozhets@gmail.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Signed-off-by: Dmitriy Zaporozhets &lt;dmitriy.zaporozhets@gmail.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch 'fix/make-github-import-retryable' into 'master'</title>
<updated>2016-10-28T12:44:32+00:00</updated>
<author>
<name>Sean McGivern</name>
<email>sean@mcgivern.me.uk</email>
</author>
<published>2016-10-28T12:44:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=4fd015183cdb280083384c69261c2ab5d475a54b'/>
<id>4fd015183cdb280083384c69261c2ab5d475a54b</id>
<content type='text'>

Modify GitHub importer to be retryable

Closes #23533

See merge request !7003</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

Modify GitHub importer to be retryable

Closes #23533

See merge request !7003</pre>
</div>
</content>
</entry>
<entry>
<title>Replace static issue fixtures by script (!6059)</title>
<updated>2016-10-28T12:02:17+00:00</updated>
<author>
<name>winniehell</name>
<email>git@winniehell.de</email>
</author>
<published>2016-08-29T16:18:59+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=0f060bf4fccc3119dc4b7d6e296c2f0fc994bb4e'/>
<id>0f060bf4fccc3119dc4b7d6e296c2f0fc994bb4e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge branch '23890-api-should-accepts-boolean' into 'master'</title>
<updated>2016-10-28T11:42:01+00:00</updated>
<author>
<name>Sean McGivern</name>
<email>sean@mcgivern.me.uk</email>
</author>
<published>2016-10-28T11:42:01+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/gitlab/gitlab-ce.git/commit/?id=5c5f34dd13d43e6ae5c15feb8e1b22db54fff606'/>
<id>5c5f34dd13d43e6ae5c15feb8e1b22db54fff606</id>
<content type='text'>

API: Fix booleans not recognized as such when using the `to_boolean` helper

Fixes #22831
Fixes #23890

See merge request !7149</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>

API: Fix booleans not recognized as such when using the `to_boolean` helper

Fixes #22831
Fixes #23890

See merge request !7149</pre>
</div>
</content>
</entry>
</feed>
