Merge branch 'master' into merge-if-green

5 files changed, 58 insertions, 44 deletions

diff --git a/doc/development/architecture.md b/doc/development/architecture.md
index c00d290371e..6101a71a8de 100644
--- a/doc/development/architecture.md
+++ b/doc/development/architecture.md
@@ -146,7 +146,7 @@ nginx
 
 Apache httpd
 
-- [Explanation of Apache logs](http://httpd.apache.org/docs/2.2/logs.html).
+- [Explanation of Apache logs](https://httpd.apache.org/docs/2.2/logs.html).
 - `/var/log/apache2/` contains error and output logs (on Ubuntu).
 - `/var/log/httpd/` contains error and output logs (on RHEL).
 
diff --git a/doc/development/profiling.md b/doc/development/profiling.md
index 80c86ef921e..e244ad4e881 100644
--- a/doc/development/profiling.md
+++ b/doc/development/profiling.md
@@ -4,11 +4,15 @@ To make it easier to track down performance problems GitLab comes with a set of
 profiling tools, some of these are available by default while others need to be
 explicitly enabled.
 
-## rack-mini-profiler
+## Sherlock
 
-This Gem is enabled by default in development only. It allows you to see the
-timings of the various components that made up a web request (e.g. the SQL
-queries executed and their execution timings).
+Sherlock is a custom profiling tool built into GitLab. Sherlock is _only_
+available when running GitLab in development mode _and_ when setting the
+environment variable `ENABLE_SHERLOCK` to a non empty value. For example:
+
+    ENABLE_SHERLOCK=1 bundle exec rails s
+
+Recorded transactions can be found by navigating to `/sherlock/transactions`.
 
 ## Bullet
 
@@ -21,36 +25,3 @@ starting GitLab. For example:
 
 Bullet will log query problems to both the Rails log as well as the Chrome
 console.
-
-## ActiveRecord Query Trace
-
-This Gem adds backtraces for every ActiveRecord query in the Rails console. This
-can be useful to track down where a query was executed. Because this Gem adds
-quite a bit of noise (5-10 extra lines per ActiveRecord query) it's disabled by
-default. To use this Gem you'll need to set `ENABLE_QUERY_TRACE` to a non empty
-file before starting GitLab. For example:
-
-    ENABLE_QUERY_TRACE=true bundle exec rails s
-
-## rack-lineprof
-
-This is a Gem that can trace the execution time of code on a per line basis.
-Because this Gem can add quite a bit of overhead it's disabled by default. To
-enable it, set the environment variable `ENABLE_LINEPROF` to a non-empty value.
-For example:
-
-    ENABLE_LINEPROF=true bundle exec rails s
-
-Once enabled you'll need to add a query string parameter to a request to
-actually profile code execution. The name of the parameter is `lineprof` and
-should be set to a regular expression (minus the starting/ending slash) used to
-select what files to profile. To profile all files containing "foo" somewhere in
-the path you'd use the following parameter:
-
-    ?lineprof=foo
-
-Or when filtering for files containing "foo" and "bar" in their path:
-
-    ?lineprof=foo|bar
-
-Once set the profiling output will be displayed in your terminal.
diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md
index a4a980cf0e0..9f3fd69fc4e 100644
--- a/doc/development/rake_tasks.md
+++ b/doc/development/rake_tasks.md
@@ -9,7 +9,7 @@ bundle exec rake setup
 ```
 
 The `setup` task is a alias for `gitlab:setup`.
-This tasks calls `db:setup` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and fianlly it calls `db:seed_fu` to seed the database.
+This tasks calls `db:setup` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database.
 Note: `db:setup` calls `db:seed` but this does nothing.
 
 ## Run tests
diff --git a/doc/development/shared_files.md b/doc/development/shared_files.md
new file mode 100644
index 00000000000..fcd905b54a4
--- /dev/null
+++ b/doc/development/shared_files.md
@@ -0,0 +1,33 @@
+# Shared files
+
+Historically, GitLab has been storing shared files in many different
+directories: `public/uploads`, `builds`, `tmp/repositories`, `tmp/rebase` (EE),
+etc. Having so many shared directories makes it difficult to deploy GitLab on
+shared storage (e.g. NFS). Working towards GitLab 9.0 we are consolidating
+these different directories under the `shared` directory.
+
+This means that if GitLab will start storing puppies in some future version
+then we should put them in `shared/puppies`. Temporary puppy files should be
+stored in `shared/tmp`.
+
+In the GitLab application code you can get the full path to the `shared`
+directory with `Gitlab.config.shared.path`.
+
+## What is not a 'shared file'
+
+Files that belong to only one process, or on only one server, should not go in
+`shared`. Examples include PID files and sockets.
+
+## Temporary files and shared storage
+
+Sometimes you create a temporary file on disk with the intention of it becoming
+'official'. For example you might be first streaming an upload from a user to
+disk in a temporary file so you can perform some checks on it. When the checks
+pass, you make the file official. In scenarios like this please follow these
+rules:
+
+- Store the temporary file under `shared/tmp`, i.e. on the same filesystem you
+  want the official file to be on.
+- Use move/rename operations when operating on the file instead of copy
+  operations where possible, because renaming a file is much faster than
+  copying it.
diff --git a/doc/development/shell_commands.md b/doc/development/shell_commands.md
index 2d1d0fb4154..65cdd74bdb6 100644
--- a/doc/development/shell_commands.md
+++ b/doc/development/shell_commands.md
@@ -35,6 +35,16 @@ Gitlab::Popen.popen(%W(find /some/path -not -path /some/path -mmin +120 -delete)
 
 This coding style could have prevented CVE-2013-4490.
 
+## Always use the configurable git binary path for git commands
+
+```ruby
+# Wrong
+system(*%W(git branch -d -- #{branch_name}))
+
+# Correct
+system(*%W(#{Gitlab.config.git.bin_path} branch -d -- #{branch_name}))
+```
+
 ## Bypass the shell by splitting commands into separate tokens
 
 When we pass shell commands as a single string to Ruby, Ruby will let `/bin/sh` evaluate the entire string. Essentially, we are asking the shell to evaluate a one-line script. This creates a risk for shell injection attacks. It is better to split the shell command into tokens ourselves. Sometimes we use the scripting capabilities of the shell to change the working directory or set environment variables. All of this can also be achieved securely straight from Ruby
@@ -81,9 +91,9 @@ In the GitLab codebase, we avoid the option/argument ambiguity by _always_ using
 
 ```ruby
 # Wrong
-system(*%W(git branch -d #{branch_name}))
+system(*%W(#{Gitlab.config.git.bin_path} branch -d #{branch_name}))
 # Correct
-system(*%W(git branch -d -- #{branch_name}))
+system(*%W(#{Gitlab.config.git.bin_path} branch -d -- #{branch_name}))
 ```
 
 This coding style could have prevented CVE-2013-4582.
@@ -94,9 +104,9 @@ Capturing the output of shell commands with backticks reads nicely, but you are
 
 ```ruby
 # Wrong
-logs = `cd #{repo_dir} && git log`
+logs = `cd #{repo_dir} && #{Gitlab.config.git.bin_path} log`
 # Correct
-logs, exit_status = Gitlab::Popen.popen(%W(git log), repo_dir)
+logs, exit_status = Gitlab::Popen.popen(%W(#{Gitlab.config.git.bin_path} log), repo_dir)
 
 # Wrong
 user = `whoami`
@@ -108,7 +118,7 @@ In other repositories, such as gitlab-shell you can also use `IO.popen`.
 
 ```ruby
 # Safe IO.popen example
-logs = IO.popen(%W(git log), chdir: repo_dir) { |p| p.read }
+logs = IO.popen(%W(#{Gitlab.config.git.bin_path} log), chdir: repo_dir) { |p| p.read }
 ```
 
 Note that unlike `Gitlab::Popen.popen`, `IO.popen` does not capture standard error.