summaryrefslogtreecommitdiff
path: root/lib/url.c
Commit message (Collapse)AuthorAgeFilesLines
* url: fix null dispname for --connect-to optionShohei Maeda2023-05-121-5/+5
| | | | Closes #11106
* url: fix PVS nitsDaniel Stenberg2023-04-141-22/+16
| | | | | | | | | | - expression 'hostptr' is always true - a part of conditional expression is always true: proxypasswd - expression 'proxyuser' is always true - avoid multiple Curl_now() calls in allocate_conn Ref: #10929 Closes #10959
* urlapi: cleanupsDaniel Stenberg2023-04-131-20/+4
| | | | | | | | | | | | | | | | | | | | | | | | | - move host checks together - simplify the scheme parser loop and the end of host name parser - avoid itermediate buffer storing in multiple places - reduce scope for several variables - skip the Curl_dyn_tail() call for speed - detect IPv6 earlier and skip extra checks for such hosts - normalize directly in dynbuf instead of itermediate buffer - split out the IPv6 parser into its own funciton - call the IPv6 parser directly for ipv6 addresses - remove (unused) special treatment of % in host names - junkscan() once in the beginning instead of scattered - make junkscan return error code - remove unused query management from dedotdotify() - make Curl_parse_login_details use memchr - more use of memchr() instead of strchr() and less strlen() calls - make junkscan check and return the URL length An optimized build runs one of my benchmark URL parsing programs ~41% faster using this branch. (compared against the shipped 7.88.1 library in Debian) Closes #10935
* CURLPROXY_HTTPS2: for HTTPS proxy that may speak HTTP/2Daniel Stenberg2023-04-081-12/+18
| | | | | | | Setting this proxy type allows curl to negotiate and use HTTP/2 with HTTPS proxies. Closes #10900
* url: remove call to Curl_llist_destroy in Curl_closeDaniel Stenberg2023-03-271-5/+0
| | | | | | | A list that is created with a NULL "destructor" does not need to be destroyed. Not calling it is faster than calling it. Closes #10846
* multi: remove PENDING + MSGSENT handles from the main linked listDaniel Stenberg2023-03-261-3/+1
| | | | | | | | | | | | | | | | | | | | | | As they are not driving transfers or any socket activity, the main loop does not need to iterate over these handles. A performance improvement. They are instead only held in their own separate lists. 'data->multi' is kept a pointer to the multi handle as long as the easy handle is actually part of it even when the handle is moved to the pending/msgsent lists. It needs to know which multi handle it belongs to, if for example curl_easy_cleanup() is called before the handle is removed from the multi handle. Alll 'data->multi' pointers of handles still part of the multi handle gets cleared by curl_multi_cleanup() which "orphans" all previously attached easy handles. This is take 2. The first version was reverted for the 8.0.1 release. Assisted-by: Stefan Eissing Closes #10801
* lib: skip Curl_llist_destroy callsDaniel Stenberg2023-03-151-3/+0
| | | | | | | | Linked lists themselves do not carry any allocations, so for the lists that do not have have a set destructor we can just skip the Curl_llist_destroy() call and save CPU time. Closes #10764
* ftp: add more conditions for connection reuseDaniel Stenberg2023-03-131-2/+15
| | | | | Reported-by: Harry Sintonen Closes #10730
* url: only reuse connections with same GSS delegationDaniel Stenberg2023-03-101-0/+6
| | | | | Reported-by: Harry Sintonen Closes #10731
* url: remove dummy protocol handlerDaniel Stenberg2023-03-101-32/+1
| | | | | | Just two added checks were needed saves a whole handler struct. Closes #10727
* url: fix the SSH connection reuse checkDaniel Stenberg2023-03-101-1/+1
| | | | | Reported-by: Harry Sintonen Closes #10735
* url: fix cookielist memleak when curl_easy_resetJay Satiro2023-03-101-3/+5
| | | | | | | | | | | | - Free set.cookelist in Curl_freeset instead of Curl_close. Prior to this change the cookielist linked list wasn't freed by curl_easy_reset which calls Curl_freeset to free all set. Bug: https://github.com/curl/curl/issues/10694#issuecomment-1458619157 Reported-by: Sergey Ryabinin Closes https://github.com/curl/curl/pull/10709
* url: fix logic in connection reuse to deny reuse on "unclean" connectionsStefan Eissing2023-03-071-1/+14
| | | | | | | | | | | | | | - add parameter to `conn_is_alive()` cfilter method that returns if there is input data waiting on the connection - refrain from re-using connnection from the cache that have input pending - adapt http/2 and http/3 alive checks to digest pending input to check the connection state - remove check_cxn method from openssl as that was just doing what the socket filter now does. - add tests for connection reuse with special server configs Closes #10690
* CURLOPT_PIPEWAIT: allow waited reuse also for subsequent connectionsStefan Eissing2023-02-211-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | As tested in test_02_07, when firing off 200 urls with --parallel, 199 wait for the first connection to be established. if that is multiuse, urls are added up to its capacity. The first url over capacity opens another connection. But subsequent urls found the same situation and open a connection too. They should have waited for the second connection to actually connect and make its capacity known. This change fixes that by - setting `connkeep()` early in the HTTP setup handler. as otherwise a new connection is marked as closeit by default and not considered for multiuse at all - checking the "connected" status for a candidate always and continuing to PIPEWAIT if no alternative is found. pytest: - removed "skip" from test_02_07 - added test_02_07b to check that http/1.1 continues to work as before Closes #10456
* urldata: remove `now` from struct SingleRequest - not neededStefan Eissing2023-02-171-1/+0
| | | | Closes #10549
* url: fix part of conditional expression is always truePronyushkin Petr2023-02-061-1/+1
| | | | Closes #10407
* cf-socket: improvements in socket I/O handlingStefan Eissing2023-01-311-46/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Curl_write_plain/Curl_read_plain have been eliminated. Last code use now uses Curl_conn_send/recv so that requests use conn->send/revc callbacks which defaults to cfilters use. - Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c. - USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved into cf-socket.c. The pre_recv buffer is held at the socket filter context. `postponed_data` structures have been removed from `connectdata`. - the hanger in HTTP/2 request handling was a result of read buffering on all sends and the multi handling is not prepared for this. The following happens: - multi preforms on a HTTP/2 easy handle - h2 reads and processes data - this leads to a send of h2 data - which receives and buffers before the send - h2 returns - multi selects on the socket, but no data arrives (its in the buffer already) the workaround now receives data in a loop as long as there is something in the buffer. The real fix would be for multi to change, so that `data_pending` is evaluated before deciding to wait on the socket. io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower filter have pending data. This io_buffer is only available/used when the -DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows configurations. It also maintains the original checks on protocol handler being HTTP and conn->send/recv not being replaced. The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds out that the "lower" filter chain has still pending data at the end of its IO operation. This prevents the processing from becoming stalled. Closes #10280
* vtls: fix hostname handling in filtersStefan Eissing2023-01-201-0/+14
| | | | | | | | | | | | | | | - Copy the hostname and dispname to ssl_connect_data. Use a copy instead of referencing the `connectdata` instance since this may get free'ed on connection reuse. Reported-by: Stefan Talpalaru Reported-by: sergio-nsk@users.noreply.github.com Fixes https://github.com/curl/curl/issues/10273 Fixes https://github.com/curl/curl/issues/10309 Closes https://github.com/curl/curl/pull/10310
* noproxy: support for space-separated names is deprecatedDaniel Stenberg2023-01-041-1/+5
| | | | | | | | To be removed in July 2024. Assisted-by: Michael Osipov Fixes #10209 Closes #10215
* lib: fix typos in comments which repeat a wordAndrei Rybak2023-01-041-1/+1
| | | | | | | Remove erroneously duplicated words in code comments of files `lib.connect.c` and `lib/url.c`. Closes #10220
* cf-socket: keep sockaddr local in the socket filtersStefan Eissing2023-01-031-3/+0
| | | | | | | | | | | | | - copy `struct Curl_addrinfo` on filter setup into context - remove `struct Curl_addrinfoi *` with `struct Curl_sockaddr_ex *` in connectdata that is set and NULLed by the socket filter - this means we have no reference to the resolver info in connectdata or its filters - trigger the CF_CTRL_CONN_INFO_UPDATE event when the complete filter chain reaches connected status - update easy handle connection information on CF_CTRL_DATA_SETUP event. Closes #10213
* copyright: update all copyright lines and remove year rangesDaniel Stenberg2023-01-031-1/+1
| | | | | | | | | | | | - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
* urldata: cease storing TLS auth typeDaniel Stenberg2022-12-311-3/+0
| | | | | | | | | | The only TLS auth type libcurl ever supported is SRP and that is the default type. Since nobody ever sets any other type, there is no point in wasting space to store the set type and code to check the type. If TLS auth is used, SRP is now implied. Closes #10181
* lib: connect/h2/h3 refactorStefan Eissing2022-12-301-44/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactoring of connection setup and happy eyeballing. Move nghttp2. ngtcp2, quiche and msh3 into connection filters. - eyeballing cfilter that uses sub-filters for performing parallel connects - socket cfilter for all transport types, including QUIC - QUIC implementations in cfilter, can now participate in eyeballing - connection setup is more dynamic in order to adapt to what filter did really connect. Relevant to see if a SSL filter needs to be added or if SSL has already been provided - HTTP/3 test cases similar to HTTP/2 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche - Fix for data attach/detach in VTLS filters that could lead to crashes during parallel transfers. - Eliminating setup() methods in cfilters, no longer needed. - Improving Curl_conn_is_alive() to replace Curl_connalive() and integrated ssl alive checks into cfilter. - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update connection into and persist it at the easy handle. - Several more cfilter related cleanups and moves: - stream_weigth and dependency info is now wrapped in struct Curl_data_priority - Curl_data_priority members depend is available in HTTP2|HTTP3 - Curl_data_priority members depend on NGHTTP2 support - handling init/reset/cleanup of priority part of url.c - data->state.priority same struct, but shallow copy for compares only - PROTOPT_STREAM has been removed - Curl_conn_is_mulitplex() now available to check on capability - Adding query method to connection filters. - ngtcp2+quiche: implementing query for max concurrent transfers. - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event. - setting keepalive timestamp on connect - DATA_SETUP is called after the connection has been completely setup (but may not connected yet) to allow filters to initialize data members they use. - there is no socket to be had with msh3, it is unclear how select shall work - manual test via "curl --http3 https://curl.se" fail with "empty reply from server". - Various socket/conn related cleanups: - Curl_socket is now Curl_socket_open and in cf-socket.c - Curl_closesocket is now Curl_socket_close and in cf-socket.c - Curl_ssl_use has been replaced with Cur_conn_is_ssl - Curl_conn_tcp_accepted_set has been split into Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set with a clearer purpose Closes #10141
* share: add sharing of HSTS cache among handlesDaniel Stenberg2022-12-271-1/+5
| | | | Closes #10138
* url: fix build with `--disable-cookies`Hannah Schierling2022-12-261-0/+2
| | | | | | | | | Struct `UserDefined` has no member `cookielist` if `CURL_DISABLE_COOKIES` is defined. Follow-up to af5999a Closes #10158
* urldata: remove unused struct fields, made more conditionalDaniel Stenberg2022-12-231-2/+4
| | | | | | | | | | | | | | | | | | | | | - source_quote, source_prequote and source_postquote have not been used since 5e0d9aea3; September 2006 - make several fields conditional on proxy support - make three quote struct fields conditional on FTP || SSH - make 'mime_options' depend on MIME - make trailer_* fields depend on HTTP - change 'gssapi_delegation' from long to unsigned char - make 'localportrange' unsigned short instead of int - conn->trailer now depends on HTTP Closes #10147
* urldata: make set.http200aliases conditional on HTTP being presentDaniel Stenberg2022-12-221-2/+4
| | | | | | And make a few SSH-only fields depend on SSH Closes #10140
* urldata: move the cookefilelist to the 'set' structDaniel Stenberg2022-12-221-0/+1
| | | | | | | | | The cookiefile entries are set into the handle and should remain set for the lifetime of the handle so that duplicating it also duplicates the list. Therefore, the struct field is moved from 'state' to 'set'. Fixes #10133 Closes #10134
* idn: rename the files to idn.[ch] and hold all IDN functionsDaniel Stenberg2022-12-151-124/+5
| | | | Closes #10094
* idn: remove Curl_win32_ascii_to_idnDaniel Stenberg2022-12-151-3/+1
| | | | | | It was not used. Introduce a new IDN header for the prototype(s). Closes #10094
* mime: relax easy/mime structures bindingPatrick Monnerat2022-11-281-1/+1
| | | | | | | | | | | | | | | | | | | Deprecation and removal of codeset conversion support from the library have released the strict need for an early binding of mime structures to an easy handle (https://github.com/curl/curl/commit/2610142). This constraint currently forces to create the handle before the mime structure and the latter cannot be attached to another handle once created (see https://curl.se/mail/lib-2022-08/0027.html). This commit removes the handle pointers from the mime structures allowing more flexibility on their use. When an easy handle is duplicated, bound mime structures must however still be duplicated too as their components hold send-time dynamic information. Closes #9927
* cfilter: re-add `conn` as parameter to cfilter setup methodsStefan Eissing2022-11-261-1/+1
| | | | | | | | | | | - `Curl_ssl_get_config()` now returns the first config if no SSL proxy filter is active - socket filter starts connection only on first invocation of its connect method Fixes #9982 Closes #9983
* vtls: localization of state data in filtersStefan Eissing2022-11-221-71/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - almost all backend calls pass the Curl_cfilter intance instead of connectdata+sockindex - ssl_connect_data is remove from struct connectdata and made internal to vtls - ssl_connect_data is allocated in the added filter, kept at cf->ctx - added function to let a ssl filter access its ssl_primary_config and ssl_config_data this selects the propert subfields in conn and data, for filters added as plain or proxy - adjusted all backends to use the changed api - adjusted all backends to access config data via the exposed functions, no longer using conn or data directly cfilter renames for clear purpose: - methods `Curl_conn_*(data, conn, sockindex)` work on the complete filter chain at `sockindex` and connection `conn`. - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter instance. - methods `Curl_conn_cf()` work on/with filter instances at a connection. - rebased and resolved some naming conflicts - hostname validation (und session lookup) on SECONDARY use the same name as on FIRST (again). new debug macros and removing connectdata from function signatures where not needed. adapting schannel for new Curl_read_plain paramter. Closes #9919
* url: move back the IDN conversion of proxy namesDaniel Stenberg2022-11-181-12/+15
| | | | | | | | | | | | | | | Regression: in commit 53bcf55 we moved the IDN conversion calls to happen before the HSTS checks. But the HSTS checks are only done on the server host name, not the proxy names. By moving the proxy name IDN conversions, we accidentally broke the verbose output showing the proxy name. This change moves back the IDN conversions for the proxy names to the place in the code path they were before 53bcf55. Reported-by: Andy Stamp Fixes #9937 Closes #9939
* CURLOPT_QUICK_EXIT: don't wait for DNS thread on exitAlexandre Ferrieux2022-11-181-0/+1
| | | | | Fixes #2975 Closes #9147
* url: only acknowledge fresh_reuse for non-followed transfersDaniel Stenberg2022-11-141-1/+1
| | | | | | | | | | ... to make sure NTLM auth sticks to the connection it needs, as verified by 2032. Follow-up to fa0b9227616e Assisted-by: Stefan Eissing Closes #9905
* lib: remove bad set.opt_no_body assignmentsDaniel Stenberg2022-11-111-1/+2
| | | | | | | | | | This struct field MUST remain what the application set it to, so that handle reuse and handle duplication work. Instead, the request state bit 'no_body' is introduced for code flows that need to change this in run-time. Closes #9888
* lib: connection filters (cfilter) addition to curl:Stefan Eissing2022-11-111-110/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - general construct/destroy in connectdata - default implementations of callback functions - connect: cfilters for connect and accept - socks: cfilter for socks proxying - http_proxy: cfilter for http proxy tunneling - vtls: cfilters for primary and proxy ssl - change in general handling of data/conn - Curl_cfilter_setup() sets up filter chain based on data settings, if none are installed by the protocol handler setup - Curl_cfilter_connect() boot straps filters into `connected` status, used by handlers and multi to reach further stages - Curl_cfilter_is_connected() to check if a conn is connected, e.g. all filters have done their work - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE indicators for multi select to work - Curl_cfilter_data_pending() asks filters if the have incoming data pending for recv - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks installed in conn->recv/conn->send for io handling - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters and addition/removal of a `data` from their connection - adding vtl functions to prevent use of Curl_ssl globals directly in other parts of the code. Reviewed-by: Daniel Stenberg Closes #9855
* curl.h: add CURLOPT_CA_CACHE_TIMEOUT optionMichael Drake2022-11-081-0/+2
| | | | | | | | | | Adds a new option to control the maximum time that a cached certificate store may be retained for. Currently only the OpenSSL backend implements support for caching certificate stores. Closes #9620
* strcase: use curl_str(n)equal for case insensitive matchesDaniel Stenberg2022-11-011-1/+1
| | | | | | | | | No point in having two entry points for the same functions. Also merged the *safe* function treatment into these so that they can also be used when one or both pointers are NULL. Closes #9837
* misc: typo and grammar fixesAyesh Karunaratne2022-10-271-8/+8
| | | | | | | | | | - Replace `Github` with `GitHub`. - Replace `windows` with `Windows` - Replace `advice` with `advise` where a verb is used. - A few fixes on removing repeated words. - Replace `a HTTP` with `an HTTP` Closes #9802
* cmake: really enable warnings with clangViktor Szakats2022-10-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Even though `PICKY_COMPILER=ON` is the default, warnings were not enabled when using llvm/clang, because `CMAKE_COMPILER_IS_CLANG` was always false (in my tests at least). This is the single use of this variable in curl, and in a different place we already use `CMAKE_C_COMPILER_ID MATCHES "Clang"`, which works as expected, so change the condition to use that instead. Also fix the warnings uncovered by the above: - lib: add casts to silence clang warnings - schannel: add casts to silence clang warnings in ALPN code Assuming the code is correct, solve the warnings with a cast. This particular build case isn't CI tested. There is a chance the warning is relevant for some platforms, perhaps Windows 32-bit ARM7. Closes #9783
* http_proxy: restore the protocol pointer on errorDaniel Stenberg2022-10-241-9/+0
| | | | | | Reported-by: Trail of Bits Closes #9790
* url: use IDN decoded names for HSTS checksDaniel Stenberg2022-10-241-44/+47
| | | | | | Reported-by: Hiroki Kurosawa Closes #9791
* noproxy: support proxies specified using cidr notationDaniel Stenberg2022-10-211-79/+3
| | | | | | | | | | | | | | | For both IPv4 and IPv6 addresses. Now also checks IPv6 addresses "correctly" and not with string comparisons. Split out the noproxy checks and functionality into noproxy.c Added unit test 1614 to verify checking functions. Reported-by: Mathieu Carbonneaux Fixes #9773 Fixes #5745 Closes #9775
* netrc: remove the two 'changed' argumentsDaniel Stenberg2022-10-131-3/+0
| | | | As no user of these functions used the returned content.
* netrc: use the URL-decoded userDaniel Stenberg2022-10-131-17/+17
| | | | | | | | | | | When the user name is provided in the URL it is URL encoded there, but when used for authentication the encoded version should be used. Regression introduced after 7.83.0 Reported-by: Jonas Haag Fixes #9709 Closes #9715
* strcase: add and use Curl_timestrcmpDaniel Stenberg2022-10-081-21/+13
| | | | | | | | | | | | | This is a strcmp() alternative function for comparing "secrets", designed to take the same time no matter the content to not leak match/non-match info to observers based on how fast it is. The time this function takes is only a function of the shortest input string. Reported-by: Trail of Bits Closes #9658
* url: rename function due to name-clash in Watt-32Daniel Stenberg2022-09-271-4/+4
| | | | | Follow-up to 2481dbe5f4f58 and applies the change the way it was intended.
View Lorry Controller Status