diff options
author | Kirill Smelkov <kirr@nexedi.com> | 2015-11-06 13:41:53 +0300 |
---|---|---|
committer | Kirill Smelkov <kirr@nexedi.com> | 2015-11-10 20:33:24 +0300 |
commit | 184385ac5b15ee8b7dc6fa5278f7e711de275921 (patch) | |
tree | 77bfd186bf4cd4f892326250e310db2bd46b3b92 | |
parent | 79fdf65c71e90773fbf52d6832b74cf5a7124755 (diff) | |
download | gitlab-shell-184385ac5b15ee8b7dc6fa5278f7e711de275921.tar.gz |
Add support to connect gitlab-shell to Unicorn via UNIX socket
It is well known that UNIX sockets are faster than TCP over loopback.
E.g. on my machine according to lmbench[1] they have ~ 2 times
lower latency and ~ 2-3 times more throughput compared to TCP over
loopback:
*Local* Communication latencies in microseconds - smaller is better
---------------------------------------------------------------------
Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP
ctxsw UNIX UDP TCP conn
--------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
teco Linux 4.2.0-1 13.8 29.2 26.8 45.0 47.9 48.5 55.5 45.
*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------------------------
Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem
UNIX reread reread (libc) (hand) read write
--------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
teco Linux 4.2.0-1 1084 4353 1493 2329.1 3720.7 1613.8 1109.2 3402 1404.
The same ratio usually holds for servers.
Also UNIX sockets, since they reside on filesystem, besides being faster with
less latency, have one another nice property: access permissions to them are
managed the same way access to files is.
Because of lower latencies and higher throughput - for performance reasons, and
for easier security, it makes sense to interconnect services on one machine via
UNIX sockets and talk via TCP only to outside world.
All internal services inside GitLab can talk to each other via UNIX socket
already and only gitlab-shell was missing support to talk to Unicorn via UNIX
socket.
Let's teach gitlab-shell to talk via UNIX sockets.
[1] http://www.bitmover.com/lmbench/
~~~~
In this patch we
- add URI::HTTPUNIX to handle http+unix:// URI scheme
- add Net::HTTPUNIX to handle "connect via unix socket and then talk http"
- adjust GitlabNet#http_client_for() accordingly
- adjust documentation in config.yml.example
The http+unix:// scheme is not reinvented anew: the idea about its structure is
quite logical an was already established at least in requests-unixsocket python
package:
http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/
https://github.com/msabramo/requests-unixsocket
-rw-r--r-- | config.yml.example | 4 | ||||
-rw-r--r-- | lib/gitlab_net.rb | 7 | ||||
-rw-r--r-- | lib/httpunix.rb | 54 | ||||
-rw-r--r-- | spec/httpunix_spec.rb | 55 |
4 files changed, 118 insertions, 2 deletions
diff --git a/config.yml.example b/config.yml.example index 43d6e85..e7ecc01 100644 --- a/config.yml.example +++ b/config.yml.example @@ -10,7 +10,9 @@ user: git # Default: http://localhost:8080/ # You only have to change the default if you have configured Unicorn # to listen on a custom port, or if you have configured Unicorn to -# only listen on a Unix domain socket. +# only listen on a Unix domain socket. For Unix domain sockets use +# "http+unix://<urlquoted-path-to-socket>/", e.g. +# "http+unix://%2Fpath%2Fto%2Fsocket/" gitlab_url: "http://localhost:8080/" # See installation.md#using-https for additional HTTPS configuration details. diff --git a/lib/gitlab_net.rb b/lib/gitlab_net.rb index 8eb63ae..6f47938 100644 --- a/lib/gitlab_net.rb +++ b/lib/gitlab_net.rb @@ -5,6 +5,7 @@ require 'json' require_relative 'gitlab_config' require_relative 'gitlab_logger' require_relative 'gitlab_access' +require_relative 'httpunix' class GitlabNet class ApiUnreachableError < StandardError; end @@ -63,7 +64,11 @@ class GitlabNet end def http_client_for(uri) - http = Net::HTTP.new(uri.host, uri.port) + if uri.is_a?(URI::HTTPUNIX) + http = Net::HTTPUNIX.new(uri.hostname) + else + http = Net::HTTP.new(uri.host, uri.port) + end if uri.is_a?(URI::HTTPS) http.use_ssl = true diff --git a/lib/httpunix.rb b/lib/httpunix.rb new file mode 100644 index 0000000..12787ee --- /dev/null +++ b/lib/httpunix.rb @@ -0,0 +1,54 @@ +# support for http+unix://... connection scheme +# +# The URI scheme has the same structure as the similar one for python requests. See: +# http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/ +# https://github.com/msabramo/requests-unixsocket + +require 'uri' +require 'net/http' + +module URI + class HTTPUNIX < HTTP + def hostname + # decode %XX from path to file + v = self.host + URI.decode(v) + end + + # port is not allowed in URI + DEFAULT_PORT = nil + def set_port(v) + return v unless v + raise InvalidURIError, "http+unix:// cannot contain port" + end + end + @@schemes['HTTP+UNIX'] = HTTPUNIX +end + +# Based on: +# - http://stackoverflow.com/questions/15637226/ruby-1-9-3-simple-get-request-to-unicorn-through-socket +# - Net::HTTP::connect +module Net + class HTTPUNIX < HTTP + def initialize(socketpath, port=nil) + super(socketpath, port) + @port = nil # HTTP will set it to default - override back -> set DEFAULT_PORT + end + + # override to prevent ":<port>" being appended to HTTP_HOST + def addr_port + address + end + + def connect + D "opening connection to #{address} ..." + s = UNIXSocket.new(address) + D "opened" + @socket = BufferedIO.new(s) + @socket.read_timeout = @read_timeout + @socket.continue_timeout = @continue_timeout + @socket.debug_output = @debug_output + on_connect + end + end +end diff --git a/spec/httpunix_spec.rb b/spec/httpunix_spec.rb new file mode 100644 index 0000000..cd2ede9 --- /dev/null +++ b/spec/httpunix_spec.rb @@ -0,0 +1,55 @@ +require_relative 'spec_helper' +require_relative '../lib/httpunix' +require 'webrick' + +describe URI::HTTPUNIX do + describe :parse do + uri = URI::parse('http+unix://%2Fpath%2Fto%2Fsocket/img.jpg') + subject { uri } + + it { should be_an_instance_of(URI::HTTPUNIX) } + its(:scheme) { should eq('http+unix') } + its(:hostname) { should eq('/path/to/socket') } + its(:path) { should eq('/img.jpg') } + end +end + + +# like WEBrick::HTTPServer, but listens on UNIX socket +class HTTPUNIXServer < WEBrick::HTTPServer + def listen(address, port) + socket = Socket.unix_server_socket(address) + socket.autoclose = false + server = UNIXServer.for_fd(socket.fileno) + socket.close + @listeners << server + end +end + +def tmp_socket_path + File.join(ROOT_PATH, 'tmp', 'socket') +end + +describe Net::HTTPUNIX do + # "hello world" over unix socket server in background thread + FileUtils.mkdir_p(File.dirname(tmp_socket_path)) + server = HTTPUNIXServer.new(:BindAddress => tmp_socket_path) + server.mount_proc '/' do |req, resp| + resp.body = "Hello World (at #{req.path})" + end + Thread.start { server.start } + + it "talks via HTTP ok" do + VCR.turned_off do + begin + WebMock.allow_net_connect! + http = Net::HTTPUNIX.new(tmp_socket_path) + expect(http.get('/').body).to eq('Hello World (at /)') + expect(http.get('/path').body).to eq('Hello World (at /path)') + + ensure + WebMock.disable_net_connect! + end + end + end +end |