summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* [RELEASE] Released version 1.5.0v1.5.0Willy Tarreau2014-06-196-6/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Released version 1.5.0 with the following main changes : - MEDIUM: ssl: ignored file names ending as '.issuer' or '.ocsp'. - MEDIUM: ssl: basic OCSP stapling support. - MINOR: ssl/cli: Fix unapropriate comment in code on 'set ssl ocsp-response' - MEDIUM: ssl: add 300s supported time skew on OCSP response update. - MINOR: checks: mysql-check: Add support for v4.1+ authentication - MEDIUM: ssl: Add the option to use standardized DH parameters >= 1024 bits - MEDIUM: ssl: fix detection of ephemeral diffie-hellman key exchange by using the cipher description. - MEDIUM: http: add actions "replace-header" and "replace-values" in http-req/resp - MEDIUM: Break out check establishment into connect_chk() - MEDIUM: Add port_to_str helper - BUG/MEDIUM: fix ignored values for half-closed timeouts (client-fin and server-fin) in defaults section. - BUG/MEDIUM: Fix unhandled connections problem with systemd daemon mode and SO_REUSEPORT. - MINOR: regex: fix a little configuration memory leak. - MINOR: regex: Create JIT compatible function that return match strings - MEDIUM: regex: replace all standard regex function by own functions - MEDIUM: regex: Remove null terminated strings. - MINOR: regex: Use native PCRE API. - MINOR: missing regex.h include - DOC: Add Exim as Proxy Protocol implementer. - BUILD: don't use type "uint" which is not portable - BUILD: stats: workaround stupid and bogus -Werror=format-security behaviour - BUG/MEDIUM: http: clear CF_READ_NOEXP when preparing a new transaction - CLEANUP: http: don't clear CF_READ_NOEXP twice - DOC: fix proxy protocol v2 decoder example - DOC: fix remaining occurrences of "pattern extraction" - MINOR: log: allow the HTTP status code to be logged even in TCP frontends - MINOR: logs: don't limit HTTP header captures to HTTP frontends - MINOR: sample: improve sample_fetch_string() to report partial contents - MINOR: capture: extend the captures to support non-header keys - MINOR: tcp: prepare support for the "capture" action - MEDIUM: tcp: add a new tcp-request capture directive - MEDIUM: session: allow shorter retry delay if timeout connect is small - MEDIUM: session: don't apply the retry delay when redispatching - MEDIUM: session: redispatch earlier when possible - MINOR: config: warn when tcp-check rules are used without option tcp-check - BUG/MINOR: connection: make proxy protocol v1 support the UNKNOWN protocol - DOC: proxy protocol example parser was still wrong - DOC: minor updates to the proxy protocol doc - CLEANUP: connection: merge proxy proto v2 header and address block - MEDIUM: connection: add support for proxy protocol v2 in accept-proxy - MINOR: tools: add new functions to quote-encode strings - DOC: clarify the CSV format - MEDIUM: stats: report the last check and last agent's output on the CSV status - MINOR: freq_ctr: introduce a new averaging method - MEDIUM: session: maintain per-backend and per-server time statistics - MEDIUM: stats: report per-backend and per-server time stats in HTML and CSV outputs - BUG/MINOR: http: fix typos in previous patch - DOC: remove the ultra-obsolete TODO file - DOC: update roadmap - DOC: minor updates to the README - DOC: mention the maxconn limitations with the select poller - DOC: commit a few old design thoughts files
* DOC: commit a few old design thoughts filesWilly Tarreau2014-06-194-0/+705
| | | | | | | These ones were design notes and ideas collected during the 1.5 development phase lying on my development machine. There might still be some value in keeping them for future reference since they mention certain corner cases.
* DOC: mention the maxconn limitations with the select pollerWilly Tarreau2014-06-191-1/+5
| | | | | | Select()'s safe area is limited to 1024 FDs, and anything higher than this will report "select: FAILED" on startup in debug mode, so better document it.
* DOC: minor updates to the READMEWilly Tarreau2014-06-191-18/+19
| | | | | - mention that this is a stable version - stop mentionning the outdated contrib.html page.
* DOC: update roadmapWilly Tarreau2014-06-191-123/+31
| | | | 1.5-dev is closed, it will not get any more updates.
* DOC: remove the ultra-obsolete TODO fileWilly Tarreau2014-06-191-181/+0
| | | | This one was still written in french and outdated since 1.4 or so.
* MEDIUM: ssl: add 300s supported time skew on OCSP response update.Emeric Brun2014-06-192-1/+4
| | | | | OCSP_MAX_RESPONSE_TIME_SKEW can be set to a different value at compilation (default is 300 seconds).
* MINOR: ssl/cli: Fix unapropriate comment in code on 'set ssl ocsp-response'Emeric Brun2014-06-191-1/+1
|
* MINOR: missing regex.h includeThierry FOURNIER2014-06-191-0/+1
|
* MEDIUM: ssl: basic OCSP stapling support.Emeric Brun2014-06-185-1/+439
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The support is all based on static responses. This doesn't add any request / response logic to HAProxy, but allows a way to update information through the socket interface. Currently certificates specified using "crt" or "crt-list" on "bind" lines are loaded as PEM files. For each PEM file, haproxy checks for the presence of file at the same path suffixed by ".ocsp". If such file is found, support for the TLS Certificate Status Request extension (also known as "OCSP stapling") is automatically enabled. The content of this file is optional. If not empty, it must contain a valid OCSP Response in DER format. In order to be valid an OCSP Response must comply with the following rules: it has to indicate a good status, it has to be a single response for the certificate of the PEM file, and it has to be valid at the moment of addition. If these rules are not respected the OCSP Response is ignored and a warning is emitted. In order to identify which certificate an OCSP Response applies to, the issuer's certificate is necessary. If the issuer's certificate is not found in the PEM file, it will be loaded from a file at the same path as the PEM file suffixed by ".issuer" if it exists otherwise it will fail with an error. It is possible to update an OCSP Response from the unix socket using: set ssl ocsp-response <response> This command is used to update an OCSP Response for a certificate (see "crt" on "bind" lines). Same controls are performed as during the initial loading of the response. The <response> must be passed as a base64 encoded string of the DER encoded response from the OCSP server. Example: openssl ocsp -issuer issuer.pem -cert server.pem \ -host ocsp.issuer.com:80 -respout resp.der echo "set ssl ocsp-response $(base64 -w 10000 resp.der)" | \ socat stdio /var/run/haproxy.stat This feature is automatically enabled on openssl 0.9.8h and above. This work was performed jointly by Dirkjan Bussink of GitHub and Emeric Brun of HAProxy Technologies.
* MEDIUM: ssl: ignored file names ending as '.issuer' or '.ocsp'.Emeric Brun2014-06-181-0/+4
| | | | | | | We don't want to load these files found in directories specified in "crt" or "crt-list". These suffixes are reserved for OCSP stapling.
* MINOR: regex: Use native PCRE API.Thierry FOURNIER2014-06-182-10/+14
| | | | | | | | | | | | The pcreposix layer (in the pcre projetc) execute strlen to find thlength of the string. When we are using the function "regex_exex*2", the length is used to add a final \0, when pcreposix is executed a strlen is executed to compute the length. If we are using a native PCRE api, the length is provided as an argument, and these operations disappear. This is useful because PCRE regex are more used than POSIC regex.
* MEDIUM: regex: Remove null terminated strings.Thierry FOURNIER2014-06-181-73/+21
| | | | | | | | | | | | The new regex function can use string and length. The HAproxy buffer are not null-terminated, and the use of the regex_exec* functions implies the add of this null character. This patch replace these function by the functions which takes a string and length as input. Just the file "proto_http.c" is change because this one is more executed than other. The file "checks.c" have a very low usage, and it is not interesting to change it. Furthermore, the buffer used by "checks.c" are null-terminated.
* MEDIUM: regex: replace all standard regex function by own functionsThierry FOURNIER2014-06-189-75/+77
| | | | | | | | This patch remove all references of standard regex in haproxy. The last remaining references are only in the regex.[ch] files. In the file src/checks.c, the original function uses a "pmatch" array. In fact this array is unused. This patch remove it.
* MINOR: regex: Create JIT compatible function that return match stringsThierry FOURNIER2014-06-183-4/+152
| | | | | | | | | | | | | | | | | | | | | | | | | | This patchs rename the "regex_exec" to "regex_exec2". It add a new "regex_exec", "regex_exec_match" and "regex_exec_match2" function. This function can match regex and return array containing matching parts. Otherwise, this function use the compiled method (JIT or PCRE or POSIX). JIT require a subject with length. PCREPOSIX and native POSIX regex require a null terminted subject. The regex_exec* function are splited in two version. The first version take a null terminated string, but it execute strlen() on the subject if it is compiled with JIT. The second version (terminated by "2") take the subject and the length. This version adds a null character in the subject if it is compiled with PCREPOSIX or native POSIX functions. The documentation of posix regex and pcreposix says that the function returns 0 if the string matche otherwise it returns REG_NOMATCH. The REG_NOMATCH macro take the value 1 with posix regex and the value 17 with the pcreposix. The documentaion of the native pcre API (used with JIT) returns a negative number if no match, otherwise, it returns 0 or a positive number. This patch fix also the return codes of the regex_exec* functions. Now, these function returns true if the string match, otherwise it returns false.
* BUG/MINOR: http: fix typos in previous patchWilly Tarreau2014-06-171-3/+3
| | | | | | | When I renamed the modify-header action to replace-value, one of them was mistakenly set to "replace-val" instead. Additionally, differentiation of the two actions must be done on args[0][8] and not *args[8]. Thanks Thierry for spotting...
* MEDIUM: http: add actions "replace-header" and "replace-values" in http-req/respSasha Pachev2014-06-175-3/+324
| | | | | | | | | | | This patch adds two new actions to http-request and http-response rulesets : - replace-header : replace a whole header line, suited for headers which might contain commas - replace-value : replace a single header value, suited for headers defined as lists. The match consists in a regex, and the replacement string takes a log-format and supports back-references.
* MEDIUM: stats: report per-backend and per-server time stats in HTML and CSV ↵Willy Tarreau2014-06-172-5/+37
| | | | | | | | outputs The time statistics computed by previous patches are now reported in the HTML stats in the tips related to the total sessions for backend and servers, and as separate columns for the CSV stats.
* MEDIUM: session: maintain per-backend and per-server time statisticsWilly Tarreau2014-06-175-1/+66
| | | | | | Using the last rate counters, we now compute the queue, connect, response and total times per server and per backend with a 95% accuracy over the last 1024 samples. The operation is cheap so we don't need to condition it.
* MINOR: freq_ctr: introduce a new averaging methodWilly Tarreau2014-06-171-19/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While the current functions report average event counts per period, we are also interested in average values per event. For this we use a different method. The principle is to rely on a long tail which sums the new value with a fraction of the previous value, resulting in a sliding window of infinite length depending on the precision we're interested in. The idea is that we always keep (N-1)/N of the sum and add the new sampled value. The sum over N values can be computed with a simple program for a constant value 1 at each iteration : N ,--- \ N - 1 e - 1 > ( --------- )^x ~= N * ----- / N e '--- x = 1 Note: I'm not sure how to demonstrate this but at least this is easily verified with a simple program, the sum equals N * 0.632120 for any N moderately large (tens to hundreds). Inserting a constant sample value V here simply results in : sum = V * N * (e - 1) / e But we don't want to integrate over a small period, but infinitely. Let's cut the infinity in P periods of N values. Each period M is exactly the same as period M-1 with a factor of ((N-1)/N)^N applied. A test shows that given a large N : N - 1 1 ( ------- )^N ~= --- N e Our sum is now a sum of each factor times : N*P P ,--- ,--- \ N - 1 e - 1 \ 1 > v ( --------- )^x ~= VN * ----- * > --- / N e / e^x '--- '--- x = 1 x = 0 For P "large enough", in tests we get this : P ,--- \ 1 e > --- ~= ----- / e^x e - 1 '--- x = 0 This simplifies the sum above : N*P ,--- \ N - 1 > v ( --------- )^x = VN / N '--- x = 1 So basically by summing values and applying the last result an (N-1)/N factor we just get N times the values over the long term, so we can recover the constant value V by dividing by N. A value added at the entry of the sliding window of N values will thus be reduced to 1/e or 36.7% after N terms have been added. After a second batch, it will only be 1/e^2, or 13.5%, and so on. So practically speaking, each old period of N values represents only a quickly fading ratio of the global sum : period ratio 1 36.7% 2 13.5% 3 4.98% 4 1.83% 5 0.67% 6 0.25% 7 0.09% 8 0.033% 9 0.012% 10 0.0045% So after 10N samples, the initial value has already faded out by a factor of 22026, which is quite fast. If the sliding window is 1024 samples wide, it means that a sample will only count for 1/22k of its initial value after 10k samples went after it, which results in half of the value it would represent using an arithmetic mean. The benefit of this method is that it's very cheap in terms of computations when N is a power of two. This is very well suited to record response times as large values will fade out faster than with an arithmetic mean and will depend on sample count and not time. Demonstrating all the above assumptions with maths instead of a program is left as an exercise for the reader.
* MEDIUM: stats: report the last check and last agent's output on the CSV statusWilly Tarreau2014-06-162-7/+13
| | | | | | Now that we can quote unsafe string, it becomes possible to dump the health check responses on the CSV page as well. The two new fields are "last_chk" and "last_agt".
* DOC: clarify the CSV formatWilly Tarreau2014-06-161-1/+9
| | | | | | Indicate that the text cells in the CSV format may contain quotes to escape ambiguous texts. We don't have this case right now since we limit the output, but it may happen in the future.
* MINOR: tools: add new functions to quote-encode stringsWilly Tarreau2014-06-162-0/+66
| | | | | | qstr() and cstr() will be used to quote-encode strings. The first one does it unconditionally. The second one is aimed at CSV files where the quote-encoding is only needed when the field contains a quote or a comma.
* MINOR: regex: fix a little configuration memory leak.Thierry FOURNIER2014-06-161-3/+15
| | | | | The function regfree free the memory allocated to the pattern buffer by the compiling process. It is not freeing the buffer itself.
* MEDIUM: Add port_to_str helperSimon Horman2014-06-162-0/+42
| | | | | | | | | | | This helper is similar to addr_to_str but tries to convert the port rather than the address of a struct sockaddr_storage. This is in preparation for supporting an external agent check. Signed-off-by: Simon Horman <horms@verge.net.au>
* MEDIUM: connection: add support for proxy protocol v2 in accept-proxyWilly Tarreau2014-06-142-6/+60
| | | | | | | | The "accept-proxy" statement of bind lines was still limited to version 1 of the protocol, while send-proxy-v2 is now available on the server lines. This patch adds support for parsing v2 of the protocol on incoming connections. The v2 header is automatically recognized so there is no need for a new option.
* CLEANUP: connection: merge proxy proto v2 header and address blockWilly Tarreau2014-06-142-61/+70
| | | | | | | This is in order to simplify the PPv2 header parsing code to look more like the one provided as an example in the spec. No code change was performed beyond just merging the proxy_addr union into the proxy_hdr_v2 struct.
* DOC: minor updates to the proxy protocol docWilly Tarreau2014-06-141-9/+12
| | | | | Update the release data, revision history and the link to the Forwarded HTTP extension.
* DOC: proxy protocol example parser was still wrongWilly Tarreau2014-06-141-2/+3
| | | | | Now that version and cmd are in the same byte, it is not possible anymore to compare the version as a 13th byte.
* BUG/MINOR: connection: make proxy protocol v1 support the UNKNOWN protocolWilly Tarreau2014-06-141-2/+6
| | | | | | | | If haproxy receives a connection over a unix socket and forwards it to another haproxy instance using proxy protocol v1, it sends an UNKNOWN protocol, which is rejected by the other side. Make the receiver accept the UNKNOWN protocol as per the spec, and only use the local connection's address for this.
* MEDIUM: Break out check establishment into connect_chk()Simon Horman2014-06-131-84/+101
| | | | | | | This is in preparation for adding a new type of check that uses a process rather than a socket. Signed-off-by: Simon Horman <horms@verge.net.au>
* MINOR: config: warn when tcp-check rules are used without option tcp-checkWilly Tarreau2014-06-131-0/+7
| | | | Since this case means that the rules will be ignored, better emit a warning.
* MEDIUM: session: redispatch earlier when possibleWilly Tarreau2014-06-131-1/+8
| | | | | | | | | As discussed with Dmitry Sivachenko, is a server farm has more than one active server, uses a guaranteed non-determinist algorithm (round robin), and a connection was initiated from a non-persistent connection, there's no point insisting to reconnect to the same server after a connect failure, better redispatch upon the very first retry instead of insisting on the same server multiple times.
* MEDIUM: session: don't apply the retry delay when redispatchingWilly Tarreau2014-06-131-2/+7
| | | | | | | | | | | | The retry delay is only useful when sticking to a same server. During a redispatch, it's useless and counter-productive if we're sure to switch to another server, which is almost guaranteed when there's more than one server and the balancing algorithm is round robin, so better not pass via the turn-around state in this case. It could be done as well for leastconn, but there's a risk of always killing the delay after the recovery of a server in a farm where it's almost guaranteed to take most incoming traffic. So better only kill the delay when using round robin.
* MEDIUM: session: allow shorter retry delay if timeout connect is smallWilly Tarreau2014-06-131-2/+7
| | | | | | | | | | | As discussed with Dmitry Sivachenko, the default 1-second connect retry delay can be large for situations where the connect timeout is much smaller, because it means that an active connection reject will take more time to be retried than a silent drop, and that does not make sense. This patch changes this so that the retry delay is the minimum of 1 second and the connect timeout. That way people running with sub-second connect timeout will benefit from the shorter reconnect.
* MEDIUM: tcp: add a new tcp-request capture directiveWilly Tarreau2014-06-132-13/+142
| | | | | | | | | | | This new directive captures the specified fetch expression, converts it to text and puts it into the next capture slot. The capture slots are shared with header captures so that it is possible to dump all captures at once or selectively in logs and header processing. The purpose is to permit logs to contain whatever payload is found in a request, for example bytes at a fixed location or the SNI of forwarded SSL traffic.
* MINOR: tcp: prepare support for the "capture" actionWilly Tarreau2014-06-133-0/+9
| | | | | A few minor entries will be needed to capture sample fetches in requests or responses. This patch just prepares the code for this.
* MINOR: capture: extend the captures to support non-header keysWilly Tarreau2014-06-132-4/+4
| | | | | | This patch adds support for captures with no header name. The purpose is to allow extra captures to be defined and logged along with the header captures.
* MINOR: sample: improve sample_fetch_string() to report partial contentsWilly Tarreau2014-06-131-7/+16
| | | | | | | | Currently, all callers to sample_fetch_string() call it with SMP_OPT_FINAL. Now we improve it to support the case where this option is not set, and to make it return the original sample as-is. The purpose is to let the caller check the SMP_F_MAY_CHANGE flag in the result and know that it should wait to get complete contents. Currently this has no effect on existing code.
* MINOR: logs: don't limit HTTP header captures to HTTP frontendsWilly Tarreau2014-06-134-40/+23
| | | | | | | Similar to previous patches, HTTP header captures are performed when a TCP frontend switches to an HTTP backend, but are not possible to report. So let's relax the check to explicitly allow them to be present in TCP frontends.
* MINOR: log: allow the HTTP status code to be logged even in TCP frontendsWilly Tarreau2014-06-132-2/+2
| | | | | | | | | Log format is defined in the frontend, and some frontends may be chained to an HTTP backend. Sometimes it's very convenient to be able to log the HTTP status code of these HTTP backends. This status is definitely present in the internal structures, it's just that we used to limit it to be used in HTTP frontends. So let's simply relax the check to allow it to be used in TCP frontends as well.
* DOC: fix remaining occurrences of "pattern extraction"Willy Tarreau2014-06-131-8/+8
|
* MEDIUM: ssl: fix detection of ephemeral diffie-hellman key exchange by using ↵Remi Gacogne2014-06-121-8/+12
| | | | | | | | | | the cipher description. In OpenSSL, the name of a cipher using ephemeral diffie-hellman for key exchange can start with EDH, but also DHE, EXP-EDH or EXP1024-DHE. We work around this issue by using the cipher's description instead of the cipher's name. Hopefully the description is less likely to change in the future.
* MEDIUM: ssl: Add the option to use standardized DH parameters >= 1024 bitsRemi Gacogne2014-06-126-38/+380
| | | | | | | | When no static DH parameters are specified, this patch makes haproxy use standardized (rfc 2409 / rfc 3526) DH parameters with prime lenghts of 1024, 2048, 4096 or 8192 bits for DHE key exchange. The size of the temporary/ephemeral DH key is computed as the minimum of the RSA/DSA server key size and the value of a new option named tune.ssl.default-dh-param.
* BUG/MEDIUM: Fix unhandled connections problem with systemd daemon mode and ↵Simone Gotti2014-06-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SO_REUSEPORT. Using the systemd daemon mode the parent doesn't exits but waits for his childs without closing its listening sockets. As linux 3.9 introduced a SO_REUSEPORT option (always enabled in haproxy if available) this will give unhandled connections problems after an haproxy reload with open connections. The problem is that when on reload a new parent is started (-Ds $oldchildspids), in haproxy.c main there's a call to start_proxies that, without SO_REUSEPORT, should fail (as the old processes are already listening) and so a SIGTOU is sent to old processes. On this signal the old childs will call (in pause_listener) a shutdown() on the listening fd. From my tests (if I understand it correctly) this affects the in kernel file (so the listen is really disabled for all the processes, also the parent). Instead, with SO_REUSEPORT, the call to start_proxies doesn't fail and so SIGTOU is never sent. Only SIGUSR1 is sent and the listen isn't disabled for the parent but only the childs will stop listening (with a call to close()) So, with SO_REUSEPORT, the old childs will close their listening sockets but will wait for the current connections to finish or timeout, and, as their parent has its listening socket open, the kernel will schedule some connections on it. These connections will never be accepted by the parent as it's in the waitpid loop. This fix will close all the listeners on the parent before entering the waitpid loop. Signed-off-by: Simone Gotti <simone.gotti@gmail.com>
* DOC: fix proxy protocol v2 decoder exampleWilly Tarreau2014-06-111-5/+4
| | | | | | Richard Russo reported that the example code in the PP spec is wrong now that we slightly changed the format to merge <ver> and <cmd>. Also rename the field <ver_cmd> to avoid any ambiguity on the usage.
* BUG/MEDIUM: fix ignored values for half-closed timeouts (client-fin and ↵Simone Gotti2014-06-111-0/+2
| | | | | | | server-fin) in defaults section. Signed-off-by: Simone Gotti <simone.gotti@gmail.com> WT: bug introduced with the new feature in 1.5-dev25, no backport is needed.
* MINOR: checks: mysql-check: Add support for v4.1+ authenticationNenad Merdanovic2014-06-112-20/+68
| | | | | | | | | | | | MySQL will in stop supporting pre-4.1 authentication packets in the future and is already giving us a hard time regarding non-silencable warnings which are logged on each health check. Warnings look like the following: "[Warning] Client failed to provide its character set. 'latin1' will be used as client character set." This patch adds basic support for post-4.1 authentication by sending the proper authentication packet with the character set, along with the QUIT command.
* CLEANUP: http: don't clear CF_READ_NOEXP twiceWilly Tarreau2014-06-111-1/+1
| | | | | Last patch cleared the flag twice in the response, which is useless. Thanks Lukas for spotting it :-)
* BUG/MEDIUM: http: clear CF_READ_NOEXP when preparing a new transactionWilly Tarreau2014-06-111-2/+2
| | | | | | | | | | | | | | | Commit b1982e2 ("BUG/MEDIUM: http/session: disable client-side expiration only after body") was tricky and caused an issue which was fixed by commit 0943757 ("BUG/MEDIUM: session: don't clear CF_READ_NOEXP if analysers are not called"). But that's not enough, another issue was introduced and further emphasized by last fix. The issue is that the CF_READ_NOEXP flag needs to be cleared when waiting for a new request over that connection, otherwise we cannot expire anymore an idle connection waiting for a new request. This explains the neverending keepalives reported by at least 3 different persons since dev24. No backport is needed.
View Lorry Controller Status