summaryrefslogtreecommitdiff
path: root/lib/sendf.c
Commit message (Collapse)AuthorAgeFilesLines
* Websocket en-/decodingStefan Eissing2023-04-251-3/+1
| | | | | | | | | | - state is fully kept at connection, since curl_ws_send() and curl_ws_rec() have lifetime beyond usual transfers - no more limit on frame sizes Reported-by: simplerobot on github Fixes #10962 Closes #10999
* cf-socket: improvements in socket I/O handlingStefan Eissing2023-01-311-282/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Curl_write_plain/Curl_read_plain have been eliminated. Last code use now uses Curl_conn_send/recv so that requests use conn->send/revc callbacks which defaults to cfilters use. - Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c. - USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved into cf-socket.c. The pre_recv buffer is held at the socket filter context. `postponed_data` structures have been removed from `connectdata`. - the hanger in HTTP/2 request handling was a result of read buffering on all sends and the multi handling is not prepared for this. The following happens: - multi preforms on a HTTP/2 easy handle - h2 reads and processes data - this leads to a send of h2 data - which receives and buffers before the send - h2 returns - multi selects on the socket, but no data arrives (its in the buffer already) the workaround now receives data in a loop as long as there is something in the buffer. The real fix would be for multi to change, so that `data_pending` is evaluated before deciding to wait on the socket. io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower filter have pending data. This io_buffer is only available/used when the -DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows configurations. It also maintains the original checks on protocol handler being HTTP and conn->send/recv not being replaced. The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds out that the "lower" filter chain has still pending data at the end of its IO operation. This prevents the processing from becoming stalled. Closes #10280
* curl_log: for failf/infof and debug logging implementationsStefan Eissing2023-01-121-71/+0
| | | | | | | | | - new functions and macros for cfilter debugging - set CURL_DEBUG with names of cfilters where debug logging should be enabled - use GNUC __attribute__ to enable printf format checks during compile Closes #10271
* sendf: fix build for Linux TCP fastopenJay Satiro2023-01-081-1/+2
| | | | | | | | | | | | | | - Fix the remote addr struct dereference. - Include cf-socket.h in urldata.h. Follow-up to 6a8d7ef9 which changed conn->ipaddr (Curl_addrinfo* ) member to conn->remote_addr (Curl_sockaddr_ex *) several days ago. Reported-by: Stephan Guilloux Fixes https://github.com/curl/curl/issues/10249 Closes https://github.com/curl/curl/pull/10250
* cf-socket: keep sockaddr local in the socket filtersStefan Eissing2023-01-031-1/+1
| | | | | | | | | | | | | - copy `struct Curl_addrinfo` on filter setup into context - remove `struct Curl_addrinfoi *` with `struct Curl_sockaddr_ex *` in connectdata that is set and NULLed by the socket filter - this means we have no reference to the resolver info in connectdata or its filters - trigger the CF_CTRL_CONN_INFO_UPDATE event when the complete filter chain reaches connected status - update easy handle connection information on CF_CTRL_DATA_SETUP event. Closes #10213
* copyright: update all copyright lines and remove year rangesDaniel Stenberg2023-01-031-1/+1
| | | | | | | | | | | | - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
* lib: connect/h2/h3 refactorStefan Eissing2022-12-301-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactoring of connection setup and happy eyeballing. Move nghttp2. ngtcp2, quiche and msh3 into connection filters. - eyeballing cfilter that uses sub-filters for performing parallel connects - socket cfilter for all transport types, including QUIC - QUIC implementations in cfilter, can now participate in eyeballing - connection setup is more dynamic in order to adapt to what filter did really connect. Relevant to see if a SSL filter needs to be added or if SSL has already been provided - HTTP/3 test cases similar to HTTP/2 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche - Fix for data attach/detach in VTLS filters that could lead to crashes during parallel transfers. - Eliminating setup() methods in cfilters, no longer needed. - Improving Curl_conn_is_alive() to replace Curl_connalive() and integrated ssl alive checks into cfilter. - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update connection into and persist it at the easy handle. - Several more cfilter related cleanups and moves: - stream_weigth and dependency info is now wrapped in struct Curl_data_priority - Curl_data_priority members depend is available in HTTP2|HTTP3 - Curl_data_priority members depend on NGHTTP2 support - handling init/reset/cleanup of priority part of url.c - data->state.priority same struct, but shallow copy for compares only - PROTOPT_STREAM has been removed - Curl_conn_is_mulitplex() now available to check on capability - Adding query method to connection filters. - ngtcp2+quiche: implementing query for max concurrent transfers. - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event. - setting keepalive timestamp on connect - DATA_SETUP is called after the connection has been completely setup (but may not connected yet) to allow filters to initialize data members they use. - there is no socket to be had with msh3, it is unclear how select shall work - manual test via "curl --http3 https://curl.se" fail with "empty reply from server". - Various socket/conn related cleanups: - Curl_socket is now Curl_socket_open and in cf-socket.c - Curl_closesocket is now Curl_socket_close and in cf-socket.c - Curl_ssl_use has been replaced with Cur_conn_is_ssl - Curl_conn_tcp_accepted_set has been split into Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set with a clearer purpose Closes #10141
* vtls: localization of state data in filtersStefan Eissing2022-11-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - almost all backend calls pass the Curl_cfilter intance instead of connectdata+sockindex - ssl_connect_data is remove from struct connectdata and made internal to vtls - ssl_connect_data is allocated in the added filter, kept at cf->ctx - added function to let a ssl filter access its ssl_primary_config and ssl_config_data this selects the propert subfields in conn and data, for filters added as plain or proxy - adjusted all backends to use the changed api - adjusted all backends to access config data via the exposed functions, no longer using conn or data directly cfilter renames for clear purpose: - methods `Curl_conn_*(data, conn, sockindex)` work on the complete filter chain at `sockindex` and connection `conn`. - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter instance. - methods `Curl_conn_cf()` work on/with filter instances at a connection. - rebased and resolved some naming conflicts - hostname validation (und session lookup) on SECONDARY use the same name as on FIRST (again). new debug macros and removing connectdata from function signatures where not needed. adapting schannel for new Curl_read_plain paramter. Closes #9919
* sendf: change Curl_read_plain to wrap Curl_recv_plain (take 2)Jay Satiro2022-11-201-23/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change Curl_read_plain would attempt to read the socket directly. On Windows that's a problem because recv data may be cached by libcurl and that data is only drained using Curl_recv_plain. Rather than rewrite Curl_read_plain to handle cached recv data, I changed it to wrap Curl_recv_plain, in much the same way that Curl_write_plain already wraps Curl_send_plain. Curl_read_plain -> Curl_recv_plain Curl_write_plain -> Curl_send_plain This fixes a bug in the schannel backend where decryption of arbitrary TLS records fails because cached recv data is never drained. We send data (TLS records formed by Schannel) using Curl_write_plain, which calls Curl_send_plain, and that may do a recv-before-send ("pre-receive") to cache received data. The code calls Curl_read_plain to read data (TLS records from the server), which prior to this change did not call Curl_recv_plain and therefore cached recv data wasn't retrieved, resulting in malformed TLS records and decryption failure (SEC_E_DECRYPT_FAILURE). The bug has only been observed during Schannel TLS 1.3 handshakes. Refer to the issue and PR for more information. -- This is take 2 of the original fix. It preserves the original behavior of Curl_read_plain to write 0 to the bytes read parameter on error, since apparently some callers expect that (SOCKS tests were hanging). The original fix which landed in 12e1def5 and was later reverted in 18383fbf failed to work properly because it did not do that. Also, it changes Curl_write_plain the same way to complement Curl_read_plain, and it changes Curl_send_plain to return -1 instead of 0 on CURLE_AGAIN to complement Curl_recv_plain. Behavior on error with these changes: Curl_recv_plain returns -1 and *code receives error code. Curl_send_plain returns -1 and *code receives error code. Curl_read_plain returns error code and *n (bytes read) receives 0. Curl_write_plain returns error code and *written receives 0. -- Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361 Assisted-by: Joel Depooter Reported-by: Egor Pugin Fixes https://github.com/curl/curl/issues/9431 Closes https://github.com/curl/curl/pull/9949
* Revert "sendf: change Curl_read_plain to wrap Curl_recv_plain"Daniel Stenberg2022-11-181-40/+22
| | | | | | | | This reverts commit 12e1def51a75392df62e65490416007d7e68dab9. It introduced SOCKS proxy fails, like test 700 never ending. Reopens #9431
* sendf: change Curl_read_plain to wrap Curl_recv_plainJay Satiro2022-11-181-22/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this change Curl_read_plain would attempt to read the socket directly. On Windows that's a problem because recv data may be cached by libcurl and that data is only drained using Curl_recv_plain. Rather than rewrite Curl_read_plain to handle cached recv data, I changed it to wrap Curl_recv_plain, in much the same way that Curl_write_plain already wraps Curl_send_plain. Curl_read_plain -> Curl_recv_plain Curl_write_plain -> Curl_send_plain This fixes a bug in the schannel backend where decryption of arbitrary TLS records fails because cached recv data is never drained. We send data (TLS records formed by Schannel) using Curl_write_plain, which calls Curl_send_plain, and that may do a recv-before-send ("pre-receive") to cache received data. The code calls Curl_read_plain to read data (TLS records from the server), which prior to this change did not call Curl_recv_plain and therefore cached recv data wasn't retrieved, resulting in malformed TLS records and decryption failure (SEC_E_DECRYPT_FAILURE). The bug has only been observed during Schannel TLS 1.3 handshakes. Refer to the issue and PR for more information. Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361 Assisted-by: Joel Depooter Reported-by: Egor Pugin Fixes https://github.com/curl/curl/issues/9431 Closes https://github.com/curl/curl/pull/9904
* lib: connection filters (cfilter) addition to curl:Stefan Eissing2022-11-111-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - general construct/destroy in connectdata - default implementations of callback functions - connect: cfilters for connect and accept - socks: cfilter for socks proxying - http_proxy: cfilter for http proxy tunneling - vtls: cfilters for primary and proxy ssl - change in general handling of data/conn - Curl_cfilter_setup() sets up filter chain based on data settings, if none are installed by the protocol handler setup - Curl_cfilter_connect() boot straps filters into `connected` status, used by handlers and multi to reach further stages - Curl_cfilter_is_connected() to check if a conn is connected, e.g. all filters have done their work - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE indicators for multi select to work - Curl_cfilter_data_pending() asks filters if the have incoming data pending for recv - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks installed in conn->recv/conn->send for io handling - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters and addition/removal of a `data` from their connection - adding vtl functions to prevent use of Curl_ssl globals directly in other parts of the code. Reviewed-by: Daniel Stenberg Closes #9855
* sendf: remove unnecessary if conditionJoel Depooter2022-10-261-10/+8
| | | | | | | | At this point, the psnd->buffer will always exist. We have already allocated a new buffer if one did not previously exist, and returned from the function if the allocation failed. Closes #9801
* ws: minor fixes for web sockets without the CONNECT_ONLY flagPaul Seligman2022-10-091-1/+2
| | | | | | | | | | | - Fixed an issue where is_in_callback was getting cleared when using web sockets with debug logging enabled - Ensure the handle is is_in_callback when calling out to fwrite_func - Change the write vs. send_data decision to whether or not the handle is in CONNECT_ONLY mode. - Account for buflen not including the header length in curl_ws_send Closes #9665
* ws: initial websockets supportDaniel Stenberg2022-09-091-2/+14
| | | | Closes #8995
* sendf: skip storing HTTP headers if HTTP disabledDaniel Stenberg2022-07-231-0/+2
| | | | Closes #9179
* sendf: store the header type in an usigned char to avoid icc warningsDaniel Stenberg2022-07-231-6/+6
| | | | Closes #9179
* sendf: make Curl_debug a void functionDaniel Stenberg2022-07-231-5/+3
| | | | | | | | As virtually no called checked the return code, and those that did wrongly treated it as a CURLcode. Detected by the icc compiler warning: enumerated type mixed with another type Closes #9179
* sendf: fix paused header writes since after the header APIDaniel Stenberg2022-07-201-2/+7
| | | | | | | | Regression since d1e4a67 Reported-by: Sergey Ogryzkov Fixes #9180 Closes #9182
* mprintf: make dprintf_formatf never return negativeDaniel Stenberg2022-07-171-2/+2
| | | | | | | | | | | This function no longer returns a negative value if the formatting string is bad since the return value would sometimes be propagated as a return code from the mprintf* functions and they are documented to return the length of the output. Which cannot be negative. Fixes #9149 Closes #9151 Reported-by: yiyuaner on github
* copyright: make repository REUSE compliantmax.mehl2022-06-131-0/+2
| | | | | | | | | | | Add licensing and copyright information for all files in this repository. This either happens in the file itself as a comment header or in the file `.reuse/dep5`. This commit also adds a Github workflow to check pull requests and adapts copyright.pl to the changes. Closes #8869
* lib: make more protocol specific struct fields #ifdefedDaniel Stenberg2022-06-011-13/+6
| | | | | | | ... so that they don't take up space if the protocols are disabled in the build. Closes #8944
* lib: remove exclamation marksDaniel Stenberg2022-04-161-1/+1
| | | | | | ... from infof() and failf() calls. Make them less attention seeking. Closes #8713
* header api: add curl_easy_header and curl_easy_nextheaderDaniel Stenberg2022-03-221-7/+18
| | | | | | Add test 1940 to 1946 to verify. Closes #8593
* lib: remove support for CURL_DOES_CONVERSIONSDaniel Stenberg2022-02-041-57/+4
| | | | | | TPF was the only user and support for that was dropped. Closes #8378
* sendf: accept zero-length data in Curl_client_write()Patrick Monnerat2021-10-251-3/+5
| | | | | | | | | | | | | | | | | | Historically, Curl_client_write() used a length value of 0 as a marker for a null-terminated data string. This feature has been removed in commit f4b85d2. To detect leftover uses of the feature, a DEBUGASSERT statement rejecting a length with value 0 was introduced, effectively precluding use of this function with zero-length data. The current commit removes the DEBUGASSERT and makes the function to return immediately if length is 0. A direct effect is to fix trying to output a zero-length distinguished name in openldap. Another DEBUGASSERT statement is also rephrased for better readability. Closes #7898
* msnprintf: return number of printed characters excluding null byteDaniel Stenberg2021-07-081-4/+2
| | | | | | | | ... even when the output is "capped" by the maximum length argument. Clarified in the docs. Closes #7361
* infof: remove newline from format strings, always append itDaniel Stenberg2021-07-071-15/+9
| | | | | | | | | | | | | | | | - the data needs to be "line-based" anyway since it's also passed to the debug callback/application - it makes infof() work like failf() and consistency is good - there's an assert that triggers on newlines in the format string - Also removes a few instances of "..." - Removes the code that would append "..." to the end of the data *iff* it was truncated in infof() Closes #7357
* http: deal with partial CONNECT sendsDaniel Stenberg2021-05-081-0/+12
| | | | | | | | | | | Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets, which helped verifying this even more. Add test 363 to verify. Reported-by: ustcqidi on github Fixes #6950 Closes #7024
* lib: fix 0-length Curl_client_write callsJacob Hoffman-Andrews2021-04-291-1/+1
| | | | Closes #6954
* lib: remove strlen call from Curl_client_writeJacob Hoffman-Andrews2021-04-291-2/+1
| | | | | | | At all call sites with an explicit 0 len, pass an appropriate nonzero len. Closes #6954
* tidy-up: make conditional checks more consistentDaniel Stenberg2021-04-221-1/+1
| | | | | | ... remove '== NULL' and '!= 0' Closes #6912
* lib: remove redundant codeMarcel Raad2021-02-101-3/+1
| | | | Closes https://github.com/curl/curl/pull/6576
* lib: remove conn->data usesPatrick Monnerat2021-01-261-4/+5
| | | | Closes #6515
* send: assert that Curl_write_plain() has a ->conn when calledDaniel Stenberg2021-01-181-1/+3
| | | | | | To help catch bad invokes. Closes #6476
* lib: pass in 'struct Curl_easy *' to most functionsDaniel Stenberg2021-01-171-26/+39
| | | | | | | | | | | | | | | | | | | | | ... in most cases instead of 'struct connectdata *' but in some cases in addition to. - We mostly operate on transfers and not connections. - We need the transfer handle to log, store data and more. Everything in libcurl is driven by a transfer (the CURL * in the public API). - This work clarifies and separates the transfers from the connections better. - We should avoid "conn->data". Since individual connections can be used by many transfers when multiplexing, making sure that conn->data points to the current and correct transfer at all times is difficult and has been notoriously error-prone over the years. The goal is to ultimately remove the conn->data pointer for this reason. Closes #6425
* failf: remove newline from formatting stringsDaniel Stenberg2020-12-251-0/+1
| | | | | | | | | ... as failf adds one itself. Also: add an assert() to failf() that triggers on a newline in the format string! Closes #6365
* cmake: check for linux/tcp.hKlaus Crusius2020-12-011-0/+2
| | | | | | The HAVE_LINUX_TCP_H define was not set by cmake. Closes #6252
* curl.se: new homeDaniel Stenberg2020-11-041-1/+1
| | | | Closes #6172
* sendf: move the verbose-check into Curl_debugDaniel Stenberg2020-11-021-59/+58
| | | | | | Saves us from having the same check done everywhere. Closes #6159
* sendf: move Curl_sendf to dict.c and make it staticDaniel Stenberg2020-09-281-46/+0
| | | | | | | ... as the only remaining user of that function. Also fix gopher.c to instead use Curl_write() Closes #6020
* Curl_send: return error when pre_receive_plain can't mallocDaniel Stenberg2020-09-251-3/+9
| | | | | | | ... will probably trigger some false DEAD CODE positives on non-windows code analyzers for the conditional code. Closes #6011
* sendf: improve the message on client write errorsDaniel Stenberg2020-06-241-1/+1
| | | | | | | | | Replace "Failed writing body (X != Y)" with "Failure writing output to destination". Possibly slightly less cryptic. Reported-by: coinhubs on github Fixes #5594 Closes #5596
* Revert "sendf: make failf() use the mvsnprintf() return code"Daniel Stenberg2020-05-221-1/+2
| | | | | | | | | | This reverts commit 74623551f306990e70c7c5515b88972005604a74. Instead mark the function call with (void). Getting the return code and using it instead triggered Coverity warning CID 1463596 because snprintf() can return a negative value... Closes #5441
* sendf: make failf() use the mvsnprintf() return codeDaniel Stenberg2020-05-191-2/+1
| | | | | | | | ... and avoid a strlen() call. Fixes a MonocleAI warning. Reported-by: MonocleAI Fixes #5413 Closes #5420
* dynbuf: introduce internal generic dynamic buffer functionsDaniel Stenberg2020-05-041-29/+5
| | | | | | | | | | | | | A common set of functions instead of many separate implementations for creating buffers that can grow when appending data to them. Existing functionality has been ported over. In my early basic testing, the total number of allocations seem at roughly the same amount as before, possibly a few less. See docs/DYNBUF.md for a description of the API. Closes #5300
* http2: make pausing/unpausing set/clear local stream windowDaniel Stenberg2020-02-271-0/+6
| | | | | | | | | | | | | This reduces the HTTP/2 window size to 32 MB since libcurl might have to buffer up to this amount of data in memory and yet we don't want it set lower to potentially impact tranfer performance on high speed networks. Requires nghttp2 commit b3f85e2daa629 (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end up in the next release after 1.40.0. Fixes #4939 Closes #4940
* socks: make the connect phase non-blockingDaniel Stenberg2020-02-171-5/+6
| | | | | | Removes two entries from KNOWN_BUGS. Closes #4907
* global_init: assume the EINTR bit by defaultDaniel Stenberg2020-01-261-1/+1
| | | | | | | | | | | | - Removed from global_init since it isn't thread-safe. The symbol will still remain to not break compiles, it just won't have any effect going forward. - make the internals NOT loop on EINTR (the opposite from previously). It only risks returning from the select/poll/wait functions early, and that should be risk-free. Closes #4840
* build: Disable Visual Studio warning "conditional expression is constant"Jay Satiro2019-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Disable warning C4127 "conditional expression is constant" globally in curl_setup.h for when building with Microsoft's compiler. This mainly affects building with the Visual Studio project files found in the projects dir. Prior to this change the cmake and winbuild build systems already disabled 4127 globally for when building with Microsoft's compiler. Also, 4127 was already disabled for all build systems in the limited circumstance of the WHILE_FALSE macro which disabled the warning specifically for while(0). This commit removes the WHILE_FALSE macro and all other cruft in favor of disabling globally in curl_setup. Background: We have various macros that cause 0 or 1 to be evaluated, which would cause warning C4127 in Visual Studio. For example this causes it: #define Curl_resolver_asynch() 1 Full behavior is not clearly defined and inconsistent across versions. However it is documented that since VS 2015 Update 3 Microsoft has addressed this somewhat but not entirely, not warning on while(true) for example. Prior to this change some C4127 warnings occurred when I built with Visual Studio using the generated projects in the projects dir. Closes https://github.com/curl/curl/pull/4658
View Lorry Controller Status