summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* CI: github actions: add the output of $CC -dM -E-20211126-ci-gcc-output-1Willy Tarreau2021-11-262-0/+6
| | | | | | Sometimes figuring what differs between platforms is useful to fix build issues, to decide what ifdef to add for example. Let's always call $CC -dM -E- before starting make.
* BUILD: pools: only detect link-time jemalloc on ELF platformsWilly Tarreau2021-11-261-0/+2
| | | | | | | | | | | | The build broke on Windows and MacOS after commit ed232148a ("MEDIUM: pool: refactor malloc_trim/glibc and jemalloc api addition detections."), because the extern+attribute(weak) combination doesn't result in a really weak symbol and it causes an undefined symbol at link time. Let's reserve this detection to ELF platforms. The runtime detection using dladdr() remains used if defined. No backport needed, this is purely 2.6.
* BUG/MINOR: mworker: deinit of thread poller was called when not initialized20211125-revert-fd-leakWilliam Lallemand2021-11-261-4/+6
| | | | | | | | | | | | | Commit 67e371e ("BUG/MEDIUM: mworker: FD leak of the eventpoll in wait mode") introduced a regression. Upon a reload it tries to deinit the poller per thread, but no poll loop was initialized after loading the configuration. This patch fixes the issue by moving this part of the code in mworker_reload(), since this function will be called only when the poller is fully initialized. This patch must be backported in 2.5.
* MEDIUM: pool: support purging jemalloc arenas in trim_all_pools()David Carlier2021-11-251-2/+15
| | | | | In the case of Linux/glibc, falling back to malloc_trim if jemalloc had not been detected beforehand.
* MEDIUM: pool: refactor malloc_trim/glibc and jemalloc api addition detections.David Carlier2021-11-251-30/+32
| | | | | Attempt to detect jemalloc at runtime before hand whether linked or via symbols overrides, and fall back to malloc_trim/glibc for Linux otherwise.
* MINOR: quic: use more verbose QUIC traces set at compile-timeAmaury Denoyelle2021-11-251-1/+0
| | | | | | Remove the verbosity set to 0 on quic_init_stdout_traces. This will generate even more verbose traces on stdout with the default verbosity of 1 when compiling with -DENABLE_QUIC_STDOUT_TRACES.
* MINOR: quic: activate QUIC traces at compilationAmaury Denoyelle2021-11-251-0/+16
| | | | | | | | | | | Implement a function quic_init_stdout_traces called at STG_INIT. If ENABLE_QUIC_STDOUT_TRACES preprocessor define is set, the QUIC trace module will be automatically activated to emit traces on stdout on the developer level. The main purpose for now is to be able to generate traces on the haproxy docker image used for QUIC interop testing suite. This should facilitate test failure analysis.
* MINOR: qpack: support litteral field line with non-huff nameAmaury Denoyelle2021-11-251-1/+4
| | | | | | | | Support qpack header using a non-huffman encoded name in a litteral field line with name reference. This format is notably used by picoquic client and should improve haproxy interop covering.
* MEDIUM: quic: handle CIDs to rattach received packets to connectionAmaury Denoyelle2021-11-253-15/+54
| | | | | | | | | | | | Change the way the CIDs are organized to rattach received packets DCID to QUIC connection. This is necessary to be able to handle multiple DCID to one connection. For this, the quic_connection_id structure has been extended. When allocated, they are inserted in the receiver CID tree instead of the quic_conn directly. When receiving a packet, the receiver tree is inspected to retrieve the quic_connection_id. The quic_connection_id contains now contains a reference to the QUIC connection.
* CLEANUP: quic: add comments on CID codeAmaury Denoyelle2021-11-253-5/+7
| | | | | Add minor comment to explain how the CID are stored in the QUIC connection.
* REORG: quic: add comment on rare thread concurrence during CID allocAmaury Denoyelle2021-11-251-6/+10
| | | | | | The comment is here to warn about a possible thread concurrence issue when treating INITIAL packets from the same client. The macro unlikely is added to further highlight this scarce occurence.
* MINOR: quic: do not reject PADDING followed by other framesAmaury Denoyelle2021-11-251-4/+0
| | | | | | | | | | | It is valid for a QUIC packet to contain a PADDING frame followed by one or several other frames. quic_parse_padding_frame() does not require change as it detect properly the end of the frame with the first non-null byte. This allow to use quic-go implementation which uses a PADDING-CRYPTO as the first handshake packet.
* BUG/MEDIUM: mworker: FD leak of the eventpoll in wait modeWilliam Lallemand2021-11-251-7/+5
| | | | | | | | | Since 2.5, before re-executing in wait mode, the master can have a working configuration loaded, with a eventpoll fd. This case was not handled correctly and a new eventpoll FD is leaking in the master at each reload, which is inherited by the new worker. Must be backported in 2.5.
* BUG/MINOR: mworker: does not add the -sf in wait mode20211125-befab9eeWilliam Lallemand2021-11-251-16/+14
| | | | | | | | | | | | Since the wait mode is automatically executed after charging the configuration, -sf was shown in argv[] with the previous PID, which is normal, but also the current one. This is only a visual problem when listing the processes, because -sf does not do anything in wait mode. Fix the issue by removing the whole "-sf" part in wait mode, but the executed command can be seen in the argv[] of the latest worker forked. Must be backported in 2.5.
* BUG/MINOR: lua: remove loop initial declarationsBertrand Jacquin2021-11-251-1/+2
| | | | | | | | | | | | | | | | | | | HAProxy is documented to support gcc >= 3.4 as per INSTALL file, however hlua.c makes use of c11 only loop initial declarations leading to build failure when using gcc-4.9.4: x86_64-unknown-linux-gnu-gcc -Iinclude -Wchar-subscripts -Wcomment -Wformat -Winit-self -Wmain -Wmissing-braces -Wno-pragmas -Wparentheses -Wreturn-type -Wsequence-point -Wstrict-aliasing -Wswitch -Wtrigraphs -Wuninitialized -Wunknown-pragmas -Wunused-label -Wunused-variable -Wunused-value -Wpointer-sign -Wimplicit -pthread -fdiagnostics-color=auto -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -msse -mfpmath=sse -march=core2 -g -fPIC -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits -DUSE_EPOLL -DUSE_NETFILTER -DUSE_PCRE2 -DUSE_PCRE2_JIT -DUSE_POLL -DUSE_THREAD -DUSE_BACKTRACE -DUSE_TPROXY -DUSE_LINUX_TPROXY -DUSE_LINUX_SPLICE -DUSE_LIBCRYPT -DUSE_CRYPT_H -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA -DUSE_ACCEPT4 -DUSE_SLZ -DUSE_CPU_AFFINITY -DUSE_TFO -DUSE_NS -DUSE_DL -DUSE_RT -DUSE_PRCTL -DUSE_THREAD_DUMP -DUSE_PCRE2 -DPCRE2_CODE_UNIT_WIDTH=8 -I/usr/local/include -DCONFIG_HAPROXY_VERSION=\"2.5.0\" -DCONFIG_HAPROXY_DATE=\"2021/11/23\" -c -o src/connection.o src/connection.c src/hlua.c: In function 'hlua_config_prepend_path': src/hlua.c:11292:2: error: 'for' loop initial declarations are only allowed in C99 or C11 mode for (size_t i = 0; i < 2; i++) { ^ src/hlua.c:11292:2: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to compile your code This commit moves loop iterator to an explicit declaration. Must be backported to 2.5 because this issue was introduced in v2.5-dev10~69 with commit 9e5e586e35c5 ("BUG/MINOR: lua: Fix lua error handling in `hlua_config_prepend_path()`")
* MEDIUM: mworker: seamless reload use the internal sockpairsWilliam Lallemand2021-11-245-60/+63
| | | | | | | | | | | | | | | With the master worker, the seamless reload was still requiring an external stats socket to the previous process, which is a pain to configure. This patch implements a way to use the internal socketpair between the master and the workers to transfer the sockets during the reload. This way, the master will always try to transfer the socket, even without any configuration. The master will still reload with the -x argument, followed by the sockpair@ syntax. ( ex -x sockpair@4 ). Which use the FD of internal CLI to the worker.
* BUG/MINOR: lua: don't expose internal proxiesWilliam Lallemand2021-11-241-2/+4
| | | | | | | | | | Since internal proxies are now in the global proxy list, they are now reachable from core.proxies, core.backends, core.frontends. This patch fixes the issue by checking the PR_CAP_INT flag before exposing them in lua, so the user can't have access to them. This patch must be backported in 2.5.
* BUG/MINOR: httpclient: allow to replace the host headerWilliam Lallemand2021-11-243-11/+36
| | | | | | | | | | | | | This patch allows to replace the host header generated by the httpclient instead of adding a new one, resulting in the server replying an error 400. The host header is now generated from the uri only if it wasn't found in the list of headers. Also add a new request in the VTC file to test this. This patch must be backported in 2.5.
* BUG/MINOR: cache: Fix loop on cache entries in "show cache"Christopher Faulet2021-11-231-2/+1
| | | | | | | | | | | A regression was introduced in the commit da91842b6 ("BUG/MEDIUM: cache/cli: make "show cache" thread-safe"). When cli_io_handler_show_cache() is called, only one node is retrieved and is used to fill the output buffer in loop. Once set, the "node" variable is never renewed. At the end, all nodes are dumped but each one is duplicated several time into the output buffer. This patch must be backported everywhere the above commit is. It means only to 2.5 and 2.4.
* [RELEASE] Released version 2.6-dev0v2.6-dev0Willy Tarreau2021-11-236-5/+8
| | | | | Released version 2.6-dev0 with the following main changes : - MINOR: version: it's development again
* MINOR: version: it's development againWilly Tarreau2021-11-232-10/+10
| | | | This essentially reverts 9dc4057df064.
* [RELEASE] Released version 2.5.0v2.5.0Willy Tarreau2021-11-234-3/+14
| | | | | | | | | | | | | Released version 2.5.0 with the following main changes : - BUILD: SSL: add quictls build to scripts/build-ssl.sh - BUILD: SSL: add QUICTLS to build matrix - CLEANUP: sock: Wrap `accept4_broken = 1` into additional parenthesis - BUILD: cli: clear a maybe-unused warning on some older compilers - BUG/MEDIUM: cli: make sure we can report a warning from a bind keyword - BUG/MINOR: ssl: make SSL counters atomic - CLEANUP: assorted typo fixes in the code and comments - BUG/MINOR: ssl: free correctly the sni in the backend SSL cache - MINOR: version: mention that it's stable now
* MINOR: version: mention that it's stable nowWilly Tarreau2021-11-232-10/+10
| | | | | This version will be maintained up to around Q1 2023. The INSTALL file also mentions it.
* BUG/MINOR: ssl: free correctly the sni in the backend SSL cacheWilliam Lallemand2021-11-231-1/+1
| | | | | | | | | | | __ssl_sock_load_new_ckch_instance() does not free correctly the SNI in the session cache, it only frees the one in the current tid. This bug was introduced with e18d4e8 ("BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3"). This fix must be backported where the mentionned commit was backported. (all maintained versions).
* CLEANUP: assorted typo fixes in the code and commentsIlya Shipitsin2021-11-223-7/+7
| | | | This is 28th iteration of typo fixes
* BUG/MINOR: ssl: make SSL counters atomicWilly Tarreau2021-11-221-6/+6
| | | | | | | | SSL counters were added with commit d0447a7c3 ("MINOR: ssl: add counters for ssl sessions") in 2.4, but their updates were not atomic, so it's likely that under significant loads they are not correct. This needs to be backported to 2.4.
* BUG/MEDIUM: cli: make sure we can report a warning from a bind keywordWilly Tarreau2021-11-201-1/+14
| | | | | | | | | | | | | | | | | | | | Since recent 2.5 commit c8cac04bd ("MEDIUM: listener: deprecate "process" in favor of "thread" on bind lines"), the "process" bind keyword may report a warning. However some parts like the "stats socket" parser will call such bind keywords and do not expect to face warnings, so this will instantly cause a fatal error to be reported. A concrete effect is that "stats socket ... process 1" will hard-fail indicating the keyword is deprecated and will be removed in 2.7. We must relax this test, but the code isn't designed to report warnings, it uses a single string and only supports reporting an error code (-1). This patch makes a special case of the ERR_WARN code and uses ha_warning() to report it, and keeps the rest of the existing error code for other non-warning codes. Now "process" on the "stats socket" is properly reported as a warning. No backport is needed.
* BUILD: cli: clear a maybe-unused warning on some older compilersWilly Tarreau2021-11-201-2/+6
| | | | | | | | | The SHOW_TOT() and SHOW_AVG() macros used in cli_io_handler_show_activity() produce a warning on gcc 4.7 on MIPS with threads disabled because the compiler doesn't know that global.nbthread is necessarily non-null, hence that at least one iteration is performed. Let's just change the loop for a do {} while () that lets the compiler know it's always initialized. It also has the tiny benefit of making the code shorter.
* CLEANUP: sock: Wrap `accept4_broken = 1` into additional parenthesisTim Duesterhus2021-11-201-1/+1
| | | | | This makes it clear to static analysis tools that this assignment is intentional and not a mistyped comparison.
* BUILD: SSL: add QUICTLS to build matrix20211120-quictls-1Ilya Shipitsin2021-11-201-2/+3
| | | | It also enables QUIC when QUICTLS is used.
* BUILD: SSL: add quictls build to scripts/build-ssl.shIlya Shipitsin2021-11-201-0/+23
| | | | | script/build-ssl.sh is used mostly in CI, let us introduce QUIC OpenSSL fork support
* [RELEASE] Released version 2.5-dev15v2.5-dev15Willy Tarreau2021-11-194-3/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Released version 2.5-dev15 with the following main changes : - BUG/MINOR: stick-table/cli: Check for invalid ipv6 key - CLEANUP: peers: Remove useless test on peer variable in peer_trace() - DOC: log: Add comments to specify when session's listener is defined or not - BUG/MEDIUM: mux-h1: Handle delayed silent shut in h1_process() to release H1C - REGTESTS: ssl_crt-list_filters: feature cmd incorrectly set - DOC: internals: document the list API - BUG/MINOR: h3: ignore unknown frame types - MINOR: quic: redirect app_ops snd_buf through mux - MEDIUM: quic: inspect ALPN to install app_ops - MINOR: quic: support hq-interop - MEDIUM: quic: send version negotiation packet on unknown version - BUG/MEDIUM: mworker: cleanup the listeners when reexecuting - DOC: internals: document the scheduler API - BUG/MINOR: quic: fix version negotiation packet generation - CLEANUP: ssl: fix wrong #else commentary - MINOR: config: support default values for environment variables - SCRIPTS: run-regtests: reduce the number of processes needed to check options - SCRIPT: run-regtests: avoid several calls to grep to test for features - SCRIPT: run-regtests: avoid calling awk to compute the version - REGTEST: set retries count to zero for all tests that expect at 503 - REGTESTS: make tcp-check_min-recv fail fast - REGTESTS: extend the default I/O timeouts and make them overridable - BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3 - BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found - REGTESTS: ssl: test the TLS resumption - BUILD: makefile: stop opening sub-shells for each and every command - BUILD: makefile: reorder objects by build time - BUG/MEDIUM: mux-h2: always process a pending shut read - MINOR: quic_sock: missing CO_FL_ADDR_TO_SET flag - MINOR: quic: Possible wrong connection identification - MINOR: quic: Correctly pad UDP datagrams - MINOR: quic: Support transport parameters draft TLS extension - MINOR: quic: Anti-amplification implementation - MINOR: quic: Wrong Initial packet connection initialization - MINOR: quic: Wrong ACK range building - MINOR: quic: Update some QUIC protocol errors - MINOR: quic: Send CONNECTION_CLOSE frame upon TLS alert - MINOR: quic: Wrong largest acked packet number parsing - MINOR: quic: Add minimalistic support for stream flow control frames - MINOR: quic: Wrong value for version negotiation packet 'Unused' field - MINOR: quic: Support draft-29 QUIC version - BUG/MINOR: quic: fix segfault on trace for version negotiation - BUG/MINOR: hq-interop: fix potential NULL dereference - BUILD: quic: fix potential NULL dereference on xprt_quic - DOC: lua: documentation about the httpclient API - BUG/MEDIUM: cache/cli: make "show cache" thread-safe - BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found - BUG/MINOR: shctx: do not look for available blocks when the first one is enough - MINOR: shctx: add a few BUG_ON() for consistency checks
* MINOR: shctx: add a few BUG_ON() for consistency checksWilly Tarreau2021-11-191-0/+10
| | | | | | The shctx code relies on sensitive conditions that are hard to infer from the code itself, let's add some BUG_ON() to verify them. They helped spot the previous bugs.
* BUG/MINOR: shctx: do not look for available blocks when the first one is enoughWilly Tarreau2021-11-191-1/+1
| | | | | | | | | | | In shctx_row_reserve_hot() we only leave if we've found the exact requested size instead of at least as large, as is documented. This results in extra lookups and free calls in the avail loop while it is not needed, and participates to seeing a negative data_len early as spotted in previous bugs. It doesn't seem to have any other impact however, but it's better to backport it to stable branches.
* BUG/MEDIUM: shctx: leave the block allocator when enough blocks are foundWilly Tarreau2021-11-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | In shctx_row_reserve_hot(), a missing break allows the avail loop to loop for a while after having allocated the required blocks, possibly leading to the point where it could trigger the watchdog after checking up to 2 million blocks. In addition, the extra iteration may leave one block assigned with size zero at the head of the avail list, and mark it as being an isolated chain of 1 block. It's unclear whether this could have had other consequences. There is a non-negligible chance that it addreses bugs #1451 and #1284, as the pattern observed in the loop looks exactly the same as the one reported there in the crashes. It's only marked medium because it is extremely hard to trigger. Here the conditions were reproduced when starting 4k connections at once requesting objects of random sizes between 0 and 20k to store them into a small 1MB cache. However the watchdog will never trigger in such a case so one needs to instrument the functions. Thanks to Sohaib Ahmad and @g0uZ for providing useful traces. This will need to be backported to all stable branches.
* BUG/MEDIUM: cache/cli: make "show cache" thread-safeWilly Tarreau2021-11-191-1/+1
| | | | | | | | | | | | | | | The "show cache" command restarts from the previous node to look for a duplicate key, but does this after having released the lock, so under high write load, the node has many chances of having been reassigned and the dereference of the node crashes after a few iterations. Since the keys are unique anyway, there's no point looking for a dup, so let's just continue from the next value. This is only marked as medium as it seems to have been there for a while, and discovering it that late simply means that nobody uses that command, thus in practice it has a very limited impact on real users. This should be backported to all stable versions.
* DOC: lua: documentation about the httpclient APIWilliam Lallemand2021-11-191-0/+57
| | | | | The patch adds the documentation about the httpclient lua implementation.
* BUILD: quic: fix potential NULL dereference on xprt_quicAmaury Denoyelle2021-11-191-2/+2
| | | | | | | A warning is triggered by gcc9 on this code path, which is the compiler version used by ubuntu20.04 on the github CI. This is linked to github issue #1445.
* BUG/MINOR: hq-interop: fix potential NULL dereferenceAmaury Denoyelle2021-11-191-0/+3
| | | | Test return from htx_add_stline() and returns an error if NULL.
* BUG/MINOR: quic: fix segfault on trace for version negotiationAmaury Denoyelle2021-11-191-1/+1
| | | | | | When receiving Initial packets for Version Negotiation, no quic_conn is instantiated. Thus, on the final trace, the quic_conn dereferencement must be tested before using it.
* MINOR: quic: Support draft-29 QUIC versionFrédéric Lécaille2021-11-191-0/+1
| | | | This is only to support quic-tracker test suite.
* MINOR: quic: Wrong value for version negotiation packet 'Unused' fieldFrédéric Lécaille2021-11-191-1/+3
| | | | | The seven less significant bits of the first byte must be arbitrary. Without this fix, QUIC tracker "version_negotiation" test could not pass.
* MINOR: quic: Add minimalistic support for stream flow control framesFrédéric Lécaille2021-11-191-0/+9
| | | | | | | This simple patch add the parsing support for theses frames. But nothing is done at this time about the streams or flow control concerned. This is only to prevent some QUIC tracker or interop runner tests from failing for a reason independant of their tested features.
* MINOR: quic: Wrong largest acked packet number parsingFrédéric Lécaille2021-11-191-4/+4
| | | | | | When we have already received ACK frames with the same largest packet number, this is not an error at all. In this case, we must continue to parse the ACK current frame.
* MINOR: quic: Send CONNECTION_CLOSE frame upon TLS alertFrédéric Lécaille2021-11-192-13/+37
| | | | | | | | Add ->err member to quic_conn struct to store the connection errors. This is the responsability of ->send_alert callback of SSL_QUIC_METHOD struct to handle the TLS alert and consequently update ->err value. At this time, when entering qc_build_pkt() we build a CONNECTION_CLOSE frame close the connection when ->err value is not null.
* MINOR: quic: Update some QUIC protocol errorsFrédéric Lécaille2021-11-191-17/+19
| | | | | Add QC_ERR_ prefix to the macro names for the protocol errors. Add new ones which came with the last RFC drafts.
* MINOR: quic: Wrong ACK range buildingFrédéric Lécaille2021-11-191-0/+2
| | | | | | | | | When adding a range, if no "lower" range was present in the ack range root for the packet number space concerned, we did not check if the new added range could overlap the next one. This leaded haproxy to crash when encoding negative integer when building ACK frames. This bug was revealed thanks to "multi_packet_client_hello" QUIC tracker test which makes a client send two first Initial packets out of order.
* MINOR: quic: Wrong Initial packet connection initializationFrédéric Lécaille2021-11-191-1/+2
| | | | | | | | ->qc (QUIC connection) member of packet structure were badly initialized when received as second Initial packet (from picoquic -Q for instance). This leaded to corrupt the quic_conn structure with random behaviors as size effects. This bug came with this commit: "MINOR: quic: Possible wrong connection identification"
* MINOR: quic: Anti-amplification implementationFrédéric Lécaille2021-11-192-2/+19
| | | | | A QUIC server MUST not send more than three times as many bytes as received by clients before its address validation.
* MINOR: quic: Support transport parameters draft TLS extensionFrédéric Lécaille2021-11-194-4/+10
| | | | | | | | If we want to run quic-tracker against haproxy, we must at least support the draft version of the TLS extension for the QUIC transport parameters (0xffa5). quic-tracker QUIC version is draft-29 at this time. We select this depending on the QUIC version. If draft, we select the draft TLS extension.
View Lorry Controller Status