connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349

1 files changed, 15 insertions, 48 deletions

diff --git a/lib/vtls/bearssl.c b/lib/vtls/bearssl.c
index 933e3c3d0..7e3eb79ce 100644
--- a/lib/vtls/bearssl.c
+++ b/lib/vtls/bearssl.c
@@ -58,7 +58,7 @@ struct ssl_backend_data {
   unsigned char buf[BR_SSL_BUFSIZE_BIDI];
   br_x509_trust_anchor *anchors;
   size_t anchors_len;
-  const char *protocols[2];
+  const char *protocols[ALPN_ENTRIES_MAX];
   /* SSL client context is active */
   bool active;
   /* size of pending write, yet to be flushed */
@@ -691,35 +691,17 @@ static CURLcode bearssl_connect_step1(struct Curl_cfilter *cf,
     Curl_ssl_sessionid_unlock(data);
   }
 
-  if(cf->conn->bits.tls_enable_alpn) {
-    int cur = 0;
-
-    /* NOTE: when adding more protocols here, increase the size of the
-     * protocols array in `struct ssl_backend_data`.
-     */
+  if(connssl->alpn) {
+    struct alpn_proto_buf proto;
+    size_t i;
 
-    if(data->state.httpwant == CURL_HTTP_VERSION_1_0) {
-      backend->protocols[cur++] = ALPN_HTTP_1_0;
-      infof(data, VTLS_INFOF_ALPN_OFFER_1STR, ALPN_HTTP_1_0);
+    for(i = 0; i < connssl->alpn->count; ++i) {
+      backend->protocols[i] = connssl->alpn->entries[i];
     }
-    else {
-#ifdef USE_HTTP2
-      if(data->state.httpwant >= CURL_HTTP_VERSION_2
-#ifndef CURL_DISABLE_PROXY
-         && (!Curl_ssl_cf_is_proxy(cf) || !cf->conn->bits.tunnel_proxy)
-#endif
-        ) {
-        backend->protocols[cur++] = ALPN_H2;
-        infof(data, VTLS_INFOF_ALPN_OFFER_1STR, ALPN_H2);
-      }
-#endif
-
-      backend->protocols[cur++] = ALPN_HTTP_1_1;
-      infof(data, VTLS_INFOF_ALPN_OFFER_1STR, ALPN_HTTP_1_1);
-    }
-
-    br_ssl_engine_set_protocol_names(&backend->ctx.eng,
-                                     backend->protocols, cur);
+    br_ssl_engine_set_protocol_names(&backend->ctx.eng, backend->protocols,
+                                     connssl->alpn->count);
+    Curl_alpn_to_proto_str(&proto, connssl->alpn);
+    infof(data, VTLS_INFOF_ALPN_OFFER_1STR, proto.data);
   }
 
   if((1 == Curl_inet_pton(AF_INET, hostname, &addr))
@@ -868,26 +850,11 @@ static CURLcode bearssl_connect_step3(struct Curl_cfilter *cf,
   DEBUGASSERT(backend);
 
   if(cf->conn->bits.tls_enable_alpn) {
-    const char *protocol;
+    const char *proto;
 
-    protocol = br_ssl_engine_get_selected_protocol(&backend->ctx.eng);
-    if(protocol) {
-      infof(data, VTLS_INFOF_ALPN_ACCEPTED_1STR, protocol);
-
-#ifdef USE_HTTP2
-      if(!strcmp(protocol, ALPN_H2))
-        cf->conn->alpn = CURL_HTTP_VERSION_2;
-      else
-#endif
-      if(!strcmp(protocol, ALPN_HTTP_1_1))
-        cf->conn->alpn = CURL_HTTP_VERSION_1_1;
-      else
-        infof(data, "ALPN, unrecognized protocol %s", protocol);
-      Curl_multiuse_state(data, cf->conn->alpn == CURL_HTTP_VERSION_2 ?
-                          BUNDLE_MULTIPLEX : BUNDLE_NO_MULTIUSE);
-    }
-    else
-      infof(data, VTLS_INFOF_NO_ALPN);
+    proto = br_ssl_engine_get_selected_protocol(&backend->ctx.eng);
+    Curl_alpn_set_negotiated(cf, data, (const unsigned char *)proto,
+                             proto? strlen(proto) : 0);
   }
 
   if(ssl_config->primary.sessionid) {
@@ -983,7 +950,7 @@ static CURLcode bearssl_connect_common(struct Curl_cfilter *cf,
 {
   CURLcode ret;
   struct ssl_connect_data *connssl = cf->ctx;
-  curl_socket_t sockfd = cf->conn->sock[cf->sockindex];
+  curl_socket_t sockfd = Curl_conn_cf_get_socket(cf, data);
   timediff_t timeout_ms;
   int what;