summaryrefslogtreecommitdiff
path: root/lib/multi.c
Commit message (Collapse)AuthorAgeFilesLines
* multi: enable multiplexing by default (again)Daniel Stenberg2019-06-231-0/+2
| | | | | | | It was originally made default in d7c4213bd0c (7.62.0) but mistakenly reverted in commit 2f44e94efb3d (7.65.0). Now enabled again. Closes #4051
* multi: fix the transfer hash functionDaniel Stenberg2019-06-141-1/+1
| | | | | | | | Follow-up from 8b987cc7eb Reported-by: Tom van der Woerdt Fixes #4018 Closes #4024
* multi: fix the transfer hashes in the socket hash entriesDaniel Stenberg2019-06-121-59/+20
| | | | | | | | | | | | | - The transfer hashes weren't using the correct keys so removing entries failed. - Simplified the iteration logic over transfers sharing the same socket and they now simply are set to expire and thus get handled in the "regular" timer loop instead. Reported-by: Tom van der Woerdt Fixes #4012 Closes #4014
* multi: make sure 'data' can present in several sockhash entriesDaniel Stenberg2019-06-101-65/+94
| | | | | | | | | | | | | | | | | | | Since more than one socket can be used by each transfer at a given time, each sockhash entry how has its own hash table with transfers using that socket. In addition, the sockhash entry can now be marked 'blocked = TRUE'" which then makes the delete function just set 'removed = TRUE' instead of removing it "for real", as a way to not rip out the carpet under the feet of a parent function that iterates over the transfers of that same sockhash entry. Reported-by: Tom van der Woerdt Fixes #3961 Fixes #3986 Fixes #3995 Fixes #4004 Closes #3997
* unpause: trigger a timeout for event-based transfersDaniel Stenberg2019-06-091-26/+26
| | | | | | | | | | ... so that timeouts or other state machine actions get going again after a changing pause state. For example, if the last delivery was paused there's no pending socket activity. Reported-by: sstruchtrup on github Fixes #3994 Closes #4001
* singlesocket: use separate variable for inner loopDaniel Stenberg2019-06-011-4/+4
| | | | | | | | | | An inner loop within the singlesocket() function wrongly re-used the variable for the outer loop which then could cause an infinite loop. Change to using a separate variable! Reported-by: Eric Wu Fixes #3970 Closes #3973
* multi: track users of a socket betterDaniel Stenberg2019-05-281-12/+26
| | | | | | | | | | | | | | They need to be removed from the socket hash linked list with more care. When sh_delentry() is called to remove a sockethash entry, remove all individual transfers from the list first. To enable this, each Curl_easy struct now stores a pointer to the sockethash entry to know how to remove itself. Reported-by: Tom van der Woerdt and Kunal Ekawde Fixes #3952 Fixes #3904 Closes #3953
* cleanup: remove FIXME and TODO commentsDaniel Stenberg2019-05-161-2/+0
| | | | | | | | | They serve very little purpose and mostly just add noise. Most of them have been around for a very long time. I read them all before removing or rephrasing them. Ref: #3876 Closes #3883
* http_negotiate: Move the Negotiate state out of the negotiatedata structureSteve Holme2019-05-151-2/+2
| | | | | | | Given that this member variable is not used by the SASL based protocols there is no need to have it here. Closes #3882
* http_ntlm: Move the NTLM state out of the ntlmdata structureSteve Holme2019-05-151-2/+2
| | | | | Given that this member variable is not used by the SASL based protocols there is no need to have it here.
* wildcard: disable from build when FTP isn't presentDaniel Stenberg2019-05-131-1/+6
|
* build: fix "clarify calculation precedence" warningsMarcel Raad2019-05-121-2/+2
| | | | | | | Codacy/CppCheck warns about this. Consistently use parentheses as we already do in some places to silence the warning. Closes https://github.com/curl/curl/pull/3866
* Revert "multi: support verbose conncache closure handle"Jay Satiro2019-05-101-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit b0972bc. - No longer show verbose output for the conncache closure handle. The offending commit was added so that the conncache closure handle would inherit verbose mode from the user's easy handle. (Note there is no way for the user to set options for the closure handle which is why that was necessary.) Other debug settings such as the debug function were not also inherited since we determined that could lead to crashes if the user's per-handle private data was used on an unexpected handle. The reporter here says he has a debug function to capture the verbose output, and does not expect or want any output to stderr; however because the conncache closure handle does not inherit the debug function the verbose output for that handle does go to stderr. There are other plausible scenarios as well such as the user redirects stderr on their handle, which is also not inherited since it could lead to crashes when used on an unexpected handle. Short of allowing the user to set options for the conncache closure handle I don't think there's much we can safely do except no longer inherit the verbose setting. Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html Reported-by: Kristoffer Gleditsch Ref: https://github.com/curl/curl/pull/3598 Ref: https://github.com/curl/curl/pull/3618 Closes https://github.com/curl/curl/pull/3856
* multi: provide Curl_multiuse_state to update informationDaniel Stenberg2019-05-011-0/+16
| | | | | | As soon as a TLS backend gets ALPN conformation about the specific HTTP version it can now set the multiplex situation for the "bundle" and trigger moving potentially queued up transfers to the CONNECT state.
* process_pending_handles: mark queued transfers as previously pendingDaniel Stenberg2019-05-011-0/+8
| | | | | | | | | With transfers being queued up, we only move one at a a time back to the CONNECT state but now we mark moved transfers so that when a moved transfer is confirmed "successful" (it connected) it will trigger the move of another pending transfer. Previously, it would otherwise wait until the transfer was done before doing this. This makes queued up pending transfers get processed (much) faster.
* multi: fix the statenames (follow-up fix from 2f44e94efb3df8e)Daniel Stenberg2019-04-191-2/+0
| | | | | The list of names must be in sync with the defined states in the header file!
* CURLOPT_DNS_USE_GLOBAL_CACHE: removeDaniel Stenberg2019-04-111-11/+1
| | | | | | | | Remove the code too. The functionality has been disabled in code since 7.62.0. Setting this option will from now on simply be ignored and have no function. Closes #3654
* multi_runsingle(): fix use-after-freeEven Rouault2019-04-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #3745 Closes #3746 The following snippet ``` int main() { CURL* hCurlHandle = curl_easy_init(); curl_easy_setopt(hCurlHandle, CURLOPT_URL, "http://example.com"); curl_easy_setopt(hCurlHandle, CURLOPT_PROXY, "1"); curl_easy_perform(hCurlHandle); curl_easy_cleanup(hCurlHandle); return 0; } ``` triggers the following Valgrind warning ``` ==4125== Invalid read of size 8 ==4125== at 0x4E7D1EE: Curl_llist_remove (llist.c:97) ==4125== by 0x4E7EF5C: detach_connnection (multi.c:798) ==4125== by 0x4E80545: multi_runsingle (multi.c:1451) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ==4125== Address 0x9b3d1d0 is 1,120 bytes inside a block of size 1,600 free'd ==4125== at 0x4C2ECF0: free (vg_replace_malloc.c:530) ==4125== by 0x4E62C36: conn_free (url.c:756) ==4125== by 0x4E62D34: Curl_disconnect (url.c:818) ==4125== by 0x4E48DF9: Curl_once_resolved (hostip.c:1097) ==4125== by 0x4E8052D: multi_runsingle (multi.c:1446) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ==4125== Block was alloc'd at ==4125== at 0x4C2F988: calloc (vg_replace_malloc.c:711) ==4125== by 0x4E6438E: allocate_conn (url.c:1654) ==4125== by 0x4E685B4: create_conn (url.c:3496) ==4125== by 0x4E6968F: Curl_connect (url.c:4023) ==4125== by 0x4E802E7: multi_runsingle (multi.c:1368) ==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072) ==4125== by 0x4E766A0: easy_transfer (easy.c:625) ==4125== by 0x4E76915: easy_perform (easy.c:719) ==4125== by 0x4E7697C: curl_easy_perform (easy.c:738) ==4125== by 0x4008BE: main (in /home/even/curl/test) ``` This has been bisected to commit 2f44e94 Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14109 Credit to OSS Fuzz
* pipelining: removedDaniel Stenberg2019-04-061-170/+58
| | | | | | | As previously planned and documented in DEPRECATE.md, all pipelining code is removed. Closes #3651
* multi: improved HTTP_1_1_REQUIRED handlingDaniel Stenberg2019-03-271-18/+19
| | | | | | | | | Make sure to downgrade to 1.1 even when we get this HTTP/2 stream error on first flight. Reported-by: niner on github Fixes #3696 Closes #3707
* Negotiate: fix for HTTP POST with NegotiateDominik Hölzl2019-03-141-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | * Adjusted unit tests 2056, 2057 * do not generally close connections with CURLAUTH_NEGOTIATE after every request * moved negotiatedata from UrlState to connectdata * Added stream rewind logic for CURLAUTH_NEGOTIATE * introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC * Consider authproblem state for CURLAUTH_NEGOTIATE * Consider reuse_forbid for CURLAUTH_NEGOTIATE * moved and adjusted negotiate authentication state handling from output_auth_headers into Curl_output_negotiate * Curl_output_negotiate: ensure auth done is always set * Curl_output_negotiate: Set auth done also if result code is GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may also indicate the last challenge request (only works with disabled Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1) * Consider "Persistent-Auth" header, detect if not present; Reset/Cleanup negotiate after authentication if no persistent authentication * apply changes introduced with #2546 for negotiate rewind logic Fixes #1261 Closes #1975
* multi: removed unused code for request retriesDaniel Stenberg2019-03-111-72/+0
| | | | | | | | This code was once used for the non multi-interface using code path, but ever since easy_perform was turned into a wrapper around the multi interface, this code path never runs. Closes #3666
* multi: support verbose conncache closure handleJay Satiro2019-03-051-0/+2
| | | | | | | | | | | | | | | | | - Change closure handle to receive verbose setting from the easy handle most recently added via curl_multi_add_handle. The closure handle is a special easy handle used for closing cached connections. It receives limited settings from the easy handle most recently added to the multi handle. Prior to this change that did not include verbose which was a problem because on connection shutdown verbose mode was not acknowledged. Ref: https://github.com/curl/curl/pull/3598 Co-authored-by: Daniel Stenberg Closes https://github.com/curl/curl/pull/3618
* Curl_easy: remove req.maxfd - never used!Daniel Stenberg2019-02-281-2/+0
| | | | | | | Introduced in 8b6314ccfb, but not used anymore in current code. Unclear since when. Closes #3626
* multi: call multi_done on connect timeoutsDaniel Stenberg2019-02-241-1/+2
| | | | | | | | | | Failing to do so would make the CURLINFO_TOTAL_TIME timeout to not get updated correctly and could end up getting reported to the application completely wrong (way too small). Reported-by: accountantM on github Fixes #3602 Closes #3605
* singlesocket: fix the 'sincebefore' placementDaniel Stenberg2019-02-201-2/+2
| | | | | | | | | | | | The variable wasn't properly reset within the loop and thus could remain set for sockets that hadn't been set before and miss notifying the app. This is a follow-up to 4c35574 (shipped in curl 7.64.0) Reported-by: buzo-ffm on github Detected-by: Jan Alexander Steffens Fixes #3585 Closes #3589
* multi: Dereference of null pointerDaniel Stenberg2019-02-131-14/+10
| | | | | | | | Mostly a false positive, but this makes the code easier to read anyway. Detected by scan-build. Closes #3563
* http2: multi_connchanged() moved from multi.c, only used for h2Daniel Stenberg2019-02-121-11/+0
| | | | Closes #3557
* multi: remove verbose "Expire in" ... messagesDaniel Stenberg2019-02-121-3/+0
| | | | | | Reported-by: James Brown Bug: https://curl.haxx.se/mail/archive-2019-02/0013.html Closes #3558
* multi: (void)-prefix when ignoring return valuesDaniel Stenberg2019-02-111-4/+7
| | | | | ... and added braces to two function calls which fixes warnings if they are replace by empty macros at build-time.
* cleanup: make local functions staticDaniel Stenberg2019-02-101-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | urlapi: turn three local-only functions into statics conncache: make conncache_find_first_connection static multi: make detach_connnection static connect: make getaddressinfo static curl_ntlm_core: make hmac_md5 static http2: make two functions static http: make http_setup_conn static connect: make tcpnodelay static tests: make UNITTEST a thing to mark functions with, so they can be static for normal builds and non-static for unit test builds ... and mark Curl_shuffle_addr accordingly. url: make up_free static setopt: make vsetopt static curl_endian: make write32_le static rtsp: make rtsp_connisdead static warnless: remove unused functions memdebug: remove one unused function, made another static
* multi: set the EXPIRE_*TIMEOUT timers at TIMER_STARTSINGLE timeDaniel Stenberg2019-02-011-0/+6
| | | | | | | | | To make sure Curl_timeleft() also thinks the timeout has been reached when one of the EXPIRE_*TIMEOUTs expires. Bug: https://curl.haxx.se/mail/lib-2019-01/0073.html Reported-by: Zhao Yisha Closes #3501
* multi: remove "Dead assignment"Daniel Stenberg2019-01-151-2/+0
| | | | | | Found by scan-build. Follow-up to 4c35574bb785ce. Closes #3471
* urldata: rename easy_conn to just connDaniel Stenberg2019-01-111-155/+151
| | | | | | | | | | | | | | | We use "conn" everywhere to be a pointer to the connection. Introduces two functions that "attaches" and "detaches" the connection to and from the transfer. Going forward, we should favour using "data->conn" (since a transfer always only has a single connection or none at all) to "conn->data" (since a connection can have none, one or many transfers associated with it and updating conn->data to be correct is error prone and a frequent reason for internal issues). Closes #3442
* multi: multiplexing improvementsDaniel Stenberg2019-01-101-103/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #3436 Closes #3448 Problem 1 After LOTS of scratching my head, I eventually realized that even when doing 10 uploads in parallel, sometimes the socket callback to the application that tells it what to wait for on the socket, looked like it would reflect the status of just the single transfer that just changed state. Digging into the code revealed that this was indeed the truth. When multiple transfers are using the same connection, the application did not correctly get the *combined* flags for all transfers which then could make it switch to READ (only) when in fact most transfers wanted to get told when the socket was WRITEABLE. Problem 1b A separate but related regression had also been introduced by me when I cleared connection/transfer association better a while ago, as now the logic couldn't find the connection and see if that was marked as used by more transfers and then it would also prematurely remove the socket from the socket hash table even in times other transfers were still using it! Fix 1 Make sure that each socket stored in the socket hash has a "combined" action field of what to ask the application to wait for, that is potentially the ORed action of multiple parallel transfers. And remove that socket hash entry only if there are no transfers left using it. Problem 2 The socket hash entry stored an association to a single transfer using that socket - and when curl_multi_socket_action() was called to tell libcurl about activities on that specific socket only that transfer was "handled". This was WRONG, as a single socket/connection can be used by numerous parallel transfers and not necessarily a single one. Fix 2 We now store a list of handles in the socket hashtable entry and when libcurl is told there's traffic for a particular socket, it now iterates over all known transfers using that single socket.
* curl_multi_remove_handle() don't block terminating c-ares requestsBrad Spencer2019-01-071-5/+2
| | | | | | | | | Added Curl_resolver_kill() for all three resolver modes, which only blocks when necessary, along with test 1592 to confirm curl_multi_remove_handle() doesn't block unless it must. Closes #3428 Fixes #3371
* printf: introduce CURL_FORMAT_TIMEDIFF_TRikard Falkeborn2019-01-041-7/+10
|
* disconnect: separate connections and easy handles betterDaniel Stenberg2018-12-221-3/+1
| | | | | | | | | | | | Do not assume/store assocation between a given easy handle and the connection if it can be avoided. Long-term, the 'conn->data' pointer should probably be removed as it is a little too error-prone. Still used very widely though. Reported-by: masbug on github Fixes #3391 Closes #3400
* multi: convert two timeout variables to timediff_tDaniel Stenberg2018-12-111-2/+2
| | | | | | | | The time_t type is unsigned on some systems and these variables are used to hold return values from functions that return timediff_t already. timediff_t is always a signed type. Closes #3363
* Upon HTTP_1_1_REQUIRED, retry the request with HTTP/1.1Johannes Schindelin2018-12-081-0/+20
| | | | | | | | | | | | | | | | | | | This is a companion patch to cbea2fd2c (NTLM: force the connection to HTTP/1.1, 2018-12-06): with NTLM, we can switch to HTTP/1.1 preemptively. However, with other (Negotiate) authentication it is not clear to this developer whether there is a way to make it work with HTTP/2, so let's try HTTP/2 first and fall back in case we encounter the error HTTP_1_1_REQUIRED. Note: we will still keep the NTLM workaround, as it avoids an extra round trip. Daniel Stenberg helped a lot with this patch, in particular by suggesting to introduce the Curl_h2_http_1_1_error() function. Closes #3349 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
* curl_easy_perform: fix timeout handlingDaniel Stenberg2018-11-251-2/+18
| | | | | | | | | | | | | curl_multi_wait() was erroneously used from within curl_easy_perform(). It could lead to it believing there was no socket to wait for and then instead sleep for a while instead of monitoring the socket and then miss acting on that activity as swiftly as it should (causing an up to 1000 ms delay). Reported-by: Antoni Villalonga Fixes #3305 Closes #3306 Closes #3308
* snprintf: renamed and we now only use msnprintf()Daniel Stenberg2018-11-231-7/+7
| | | | | | | | | | | The function does not return the same value as snprintf() normally does, so readers may be mislead into thinking the code works differently than it actually does. A different function name makes this easier to detect. Reported-by: Tomas Hoger Assisted-by: Daniel Gustafsson Fixes #3296 Closes #3297
* ares: remove fd from multi fd set when ares is about to close the fdRomain Fliedel2018-11-201-4/+4
| | | | | | | | | | | | | | | | | | | When using c-ares for asyn dns, the dns socket fd was silently closed by c-ares without curl being aware. curl would then 'realize' the fd has been removed at next call of Curl_resolver_getsock, and only then notify the CURLMOPT_SOCKETFUNCTION to remove fd from its poll set with CURL_POLL_REMOVE. At this point the fd is already closed. By using ares socket state callback (ARES_OPT_SOCK_STATE_CB), this patch allows curl to be notified that the fd is not longer needed for neither for write nor read. At this point by calling Curl_multi_closed we are able to notify multi with CURL_POLL_REMOVE before the fd is actually closed by ares. In asyn-ares.c Curl_resolver_duphandle we can't use ares_dup anymore since it does not allow passing a different sock_state_cb_data Closes #3238
* multi: Fix error handling in the SENDPROTOCONNECT stateMichael Kaufmann2018-10-271-1/+1
| | | | | | | If Curl_protocol_connect() returns an error code, handle the error instead of switching to the next state. Closes #3170
* multi: avoid double-freeDaniel Stenberg2018-10-191-9/+4
| | | | | | | | | | | Curl_follow() no longer frees the string. Make sure it happens in the caller function, like we normally handle allocations. This bug was introduced with the use of the URL API internally, it has never been in a release version Reported-by: Dario Weißer Closes #3149
* multi: make the closure handle "inherit" CURLOPT_NOSIGNALDaniel Stenberg2018-10-191-0/+2
| | | | | | | | Otherwise, closing that handle can still cause surprises! Reported-by: Martin Ankerl Fixes #3138 Closes #3147
* Curl_follow: Always free the passed new URLMichael Kaufmann2018-10-131-0/+1
| | | | Closes #3124
* multi: fix memory leak in content encoding related error pathDaniel Stenberg2018-09-291-3/+7
| | | | | | | | ... a missing multi_done() call. Credit to OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10728 Closes #3063
* multi: fix location URL memleak in error pathDaniel Stenberg2018-09-281-1/+4
| | | | | Follow-up to #3044 - fix a leak OSS-Fuzz detected Closes #3057
* url: use the URL API internally as wellDaniel Stenberg2018-09-221-7/+2
| | | | | | ... to make it a truly unified URL parser. Closes #3017
View Lorry Controller Status