summaryrefslogtreecommitdiff
path: root/src/tool_operate.c
Commit message (Collapse)AuthorAgeFilesLines
* curl: stop retry if Retry-After: is longer than allowedDaniel Stenberg2021-09-061-0/+16
| | | | | | | | | | | If Retry-After: specifies a period that is longer than what fits within --retry-max-time, then stop retrying immediately. Added test 366 to verify. Reported-by: Kari Pahula Fixes #7675 Closes #7676
* curl: better error message when -O fails to get a good nameDaniel Stenberg2021-08-261-1/+4
| | | | | | | | | | Due to how this currently works internally, it needs a working initial file name to store contents in, so it may still fail even with -J is used (and thus accepting a name from content-disposition:) if the file name part of the URL isn't "good enough". Fixes #7628 Closes #7635
* tool_operate: Fix --fail-early with parallel transfersJay Satiro2021-08-171-1/+37
| | | | | | | | | | | | | | | | | | | | | | - Abort via progress callback to fail early during parallel transfers. When a critical error occurs during a transfer (eg --fail-early constraint) then other running transfers will be aborted via progress callback and finish with error CURLE_ABORTED_BY_CALLBACK (42). In this case, the callback error does not become the most recent error and a custom error message is used for those transfers: curld --fail --fail-early --parallel https://httpbin.org/status/404 https://httpbin.org/delay/10 curl: (22) The requested URL returned error: 404 curl: (42) Transfer aborted due to critical error in another transfer > echo %ERRORLEVEL% 22 Fixes https://github.com/curl/curl/issues/6939 Closes https://github.com/curl/curl/pull/6984
* docs: correct spelling errors and a broken linkNyholm2021-07-181-5/+5
| | | | | | | Update grammar and spelling in docs and source code comments. Closes: #7427 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
* single_transfer: ignore blank --output-dirDaniel Stenberg2021-06-111-1/+1
| | | | | | | | | ... as otherwise it creates a rather unexpected target directory with a leading slash. Reported-by: Harry Sintonen Fixes #7218 Closes #7233
* metalink: removeDaniel Stenberg2021-06-071-150/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Warning: this will make existing curl command lines that use metalink to stop working. Reasons for removal: 1. We've found several security problems and issues involving the metalink support in curl. The issues are not detailed here. When working on those, it become apparent to the team that several of the problems are due to the system design, metalink library API and what the metalink RFC says. They are very hard to fix on the curl side only. 2. The metalink usage with curl was only very briefly documented and was not following the "normal" curl usage pattern in several ways, making it surprising and non-intuitive which could lead to further security issues. 3. The metalink library was last updated 6 years ago and wasn't so active the years before that either. An unmaintained library means there's a security problem waiting to happen. This is probably reason enough. 4. Metalink requires an XML parsing library, which is complex code (even the smaller alternatives) and to this day often gets security updates. 5. Metalink is not a widely used curl feature. In the 2020 curl user survey, only 1.4% of the responders said that they'd are using it. In 2021 that number was 1.2%. Searching the web also show very few traces of it being used, even with other tools. 6. The torrent format and associated technology clearly won for downloading large files from multiple sources in parallel. Cloes #7176
* schannel: Disable auto credentials; add an option to enable itJay Satiro2021-04-221-5/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Disable auto credentials by default. This is a breaking change for clients that are using it, wittingly or not. - New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl to automatically locate and use a client certificate for authentication, when requested by the server. - New curl tool options --ssl-auto-client-cert and --proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT. This option is only supported for Schannel (the native Windows SSL library). Prior to this change Schannel would, with no notification to the client, attempt to locate a client certificate and send it to the server, when requested by the server. Since the server can request any certificate that supports client authentication in the OS certificate store it could be a privacy violation and unexpected. Fixes https://github.com/curl/curl/issues/2262 Reported-by: Jeroen Ooms Assisted-by: Wes Hinsley Assisted-by: Rich FitzJohn Ref: https://curl.se/mail/lib-2021-02/0066.html Reported-by: Morten Minde Neergaard Closes https://github.com/curl/curl/pull/6673
* tidy-up: make conditional checks more consistentDaniel Stenberg2021-04-221-5/+5
| | | | | | ... remove '== NULL' and '!= 0' Closes #6912
* tool_operate: don't discard failed parallel transfer resultJay Satiro2021-04-201-6/+11
| | | | | | | | | | | | | | | | | | | | | | | - Save a parallel transfer's result code only when it fails and the transfer is not being retried. Prior to this change the result code was always set which meant that a failed result could be erroneously discarded if a different transfer later had a successful result (CURLE_OK). Before: > curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10 > echo %ERRORLEVEL% 0 After: > curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10 > echo %ERRORLEVEL% 22 Closes #xxxx
* tool_operate: bail if set CURLOPT_HTTP09_ALLOWED returns errorDaniel Stenberg2021-03-111-0/+4
| | | | Closes #6727
* curl: set CURLOPT_NEW_FILE_PERMS if requestedDaniel Stenberg2021-02-271-0/+3
| | | | | | | | | | The --create-file-mode code logic accepted the value but never actually passed it on to libcurl! Follow-up to a7696c73436f (shipped in 7.75.0) Reported-by: Johannes Lesr Fixes #6657 Closes #6666
* tool_operate: check argc before accessing argv[1]Daniel Stenberg2021-02-271-1/+1
| | | | | | | Follow-up to 09363500b Reported-by: Emil Engler Reviewed-by: Daniel Gustafsson Closes #6668
* cookies: Support multiple -b parametersDaniel Gustafsson2021-02-251-4/+29
| | | | | | | | | | Previously only a single -b cookie parameter was supported with the last one winning. This adds support for supplying multiple -b params to have them serialized semicolon separated. Both cookiefiles and cookies can be entered multiple times. Closes #6649 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
* doh: add options to disable ssl verificationJay Satiro2021-02-141-1/+10
| | | | | | | | | | | | | | | | | | | | | | - New libcurl options CURLOPT_DOH_SSL_VERIFYHOST, CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the same as their respective counterparts. - New curl tool options --doh-insecure and --doh-cert-status do the same as their respective counterparts. Prior to this change DOH SSL certificate verification settings for verifyhost and verifypeer were supposed to be inherited respectively from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug were not. As a result DOH verification remained at the default, ie enabled, and it was not possible to disable. This commit changes behavior so that the DOH verification settings are independent and not inherited. Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676 Fixes https://github.com/curl/curl/issues/4578 Closes https://github.com/curl/curl/pull/6597
* curl: add --fail-with-bodyDaniel Stenberg2021-02-111-8/+15
| | | | | | | | Prevent both --fail and --fail-with-body on the same command line. Verify with test 349, 360 and 361. Closes #6449
* tool_writeout: refactor write-out and write-out jsonJay Satiro2021-02-091-3/+4
| | | | | | | | | | | | | | | | | | | | | - Deduplicate the logic used by write-out and write-out json. Rather than have separate writeLong, writeString, etc, logic for each of write-out and write-out json instead have respective shared functions that can output either format and a 'use_json' parameter to indicate whether it is json that is output. This will make it easier to maintain. Rather than have to go through two sets of logic now we only have to go through one. - Support write-out %{errormsg} and %{exitcode} in json. - Clarify in the doc that %{exitcode} is the exit code of the transfer. Prior to this change it just said "The numerical exitcode" which implies it's the exit code of the tool, and it's not necessarily that. Closes https://github.com/curl/curl/pull/6544
* http: improve AWS HTTP v4 Signature authDmitry Wagin2021-01-301-1/+1
| | | | | | | | | | | | | | | | | | - Add support services without region and service prefixes in the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) by providing region and service parameters via aws-sigv4 option. - Add [:region[:service]] suffix to aws-sigv4 option; - Fix memory allocation errors. - Refactor memory management. - Use Curl_http_method instead() STRING_CUSTOMREQUEST. - Refactor canonical headers generating. - Remove repeated sha256_to_hex() usage. - Add some docs fixes. - Add some codestyle fixes. - Add overloaded strndup() for debug - curl_dbg_strndup(). - Update tests. Closes #6524
* curl: include the file name in --xattr/--remote-time error msgsDaniel Stenberg2021-01-271-2/+2
|
* curl: s/config->global/global/ in single_transfer()Daniel Stenberg2021-01-271-11/+11
|
* curl: move fprintf outputs to warnfDaniel Stenberg2021-01-271-1/+1
| | | | | | | | | For setting and getting time of the download. To make the outputs respect --silent etc. Reported-by: Viktor Szakats Fixes #6533 Closes #6535
* tool_operate: spellfix a commentDaniel Stenberg2021-01-141-1/+1
|
* tool_operate: fix the suppression logic of some error messagesJay Satiro2021-01-041-6/+8
| | | | | | | | | | | | | | | | | | | | | | - Fix the failed truncation and failed writing body error messages to not be shown unless error messages are shown. (ie the user has specified -sS, or has not specified -s). - Also prefix same error messages with "curl: ", for example: curl: (23) Failed to truncate, exiting Prior to this change the failed truncation error messages would be shown if not -s, but did not account for -sS which should show. Prior to this change the failed writing body error messages would be shown always. Ref: https://curl.se/docs/manpage.html#-S Bug: https://curl.se/mail/archive-2020-12/0017.html Reported-by: Hongyi Zhao Closes https://github.com/curl/curl/pull/6402
* language: s/behaviour/behavior/gEmil Engler2021-01-021-2/+2
| | | | | | | | We currently use both spellings the british "behaviour" and the american "behavior". However "behavior" is more used in the project so I think it's worth dropping the british name. Closes #6395
* tool_operate: avoid NULL dereference of first_argDaniel Stenberg2020-12-261-1/+1
| | | | | | | Follow-up to 6a5e020d4d2b04a Identified by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 Closes #6377
* misc: fix "warning: empty expression statement has no effect"Daniel Stenberg2020-12-261-4/+1
| | | | | | | | | | Turned several macros into do-while(0) style to allow their use to work find with semicolon. Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 Follow-up to 08e8455dddc5e4 Reported-by: Gisle Vanem Closes #6376
* curl: fix handling of -q optionCherish982020-12-231-1/+1
| | | | | | | | The match of the "-q" option (short for "--disable") should: a) allow concatenation with other single-letters; and b) be case-sensitive, lest confusing with "-Q" ("--quote") Closes #6364
* curl: add variables to --write-outDaniel Stenberg2020-12-211-1/+2
| | | | | | | | | | | | | | | | | | | | | In particular, these ones can help a user to create its own error message when one or transfers fail. writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' onerror - lets a user only show the rest on non-zero exit codes url - the input URL used for this transfer urlnum - the numerical URL counter (0 indexed) for this transfer exitcode - the numerical exit code for the transfer errormsg - obvious Reported-by: Earnestly on github Fixes #6199 Closes #6207
* tool: add AWS HTTP v4 Signature supportMatthias Gatto2020-12-211-0/+2
| | | | Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
* curl: only warn not fail, if not finding the home dirDaniel Stenberg2020-11-131-20/+15
| | | | | | | | ... as there's no good reason to error out completely. Reported-by: Andreas Fischer Fixes #6200 Closes #6201
* tool_operate: set HSTS with CURLOPT_HSTS to pass on filenameDaniel Stenberg2020-11-061-1/+1
| | | | Closes #6175
* curl.se: new homeDaniel Stenberg2020-11-041-2/+2
| | | | Closes #6172
* hsts: add support for Strict-Transport-SecurityDaniel Stenberg2020-11-031-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | - enable in the build (configure) - header parsing - host name lookup - unit tests for the above - CI build - CURL_VERSION_HSTS bit - curl_version_info support - curl -V output - curl-config --features - CURLOPT_HSTS_CTRL - man page for CURLOPT_HSTS_CTRL - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) - man page for --hsts - save cache to disk - load cache from disk - CURLOPT_HSTS - man page for CURLOPT_HSTS - added docs/HSTS.md - fixed --version docs - adjusted curl_easy_duphandle Closes #5896
* tool_operate: --retry for HTTP 408 responses tooDaniel Stenberg2020-11-021-0/+1
| | | | | | | | | | This was inadvertently dropped from the code when the parallel support was added. Regression since b88940850 (7.66.0) Reviewed-by: Jay Satiro Closes #6155
* tool_operate: bail out proper on errors for parallel setupDaniel Stenberg2020-10-291-1/+1
| | | | | | | | ... otherwise for example trying to upload a missing file just causes a loop. Reported-by: BrumBrum on hackerone Closes #6141
* tool_operate: fix compiler warning when --libcurl is disabledDaniel Stenberg2020-10-161-0/+3
| | | | Closes #6095
* curl: make sure setopt CURLOPT_IPRESOLVE passes on a longDaniel Stenberg2020-10-051-6/+1
| | | | | | | | | | Previously, it would pass on a define (int) which could make libcurl read junk as a value - which prevented the CURLOPT_IPRESOLVE option to "take". This could then make test 2100 do two DoH requests instead of one! Fixes #6042 Closes #6043
* curl: make --libcurl show binary posts correctlyDaniel Stenberg2020-10-021-1/+3
| | | | | | Reported-by: Stephan Mühlstrasser Fixes #6031 Closes #6032
* tool_writeout: add new writeout variable, %{num_headers}anio2020-09-151-1/+1
| | | | | | This variable gives the number of headers. Closes #5947
* curl: retry delays in parallel mode no longer sleeps blockingDaniel Stenberg2020-09-061-12/+36
| | | | | | | | | The previous sleep for retries would block all other concurrent transfers. Starting now, the retry will instead be properly marked to not get restarted until after the delay time but other transfers can still continue in the mean time. Closes #5917
* curl:parallel_transfers: make sure retry readds the transferDaniel Stenberg2020-09-051-3/+4
| | | | | | Reported-by: htasta on github Fixes #5905 Closes #5917
* curl: in retry output don't call all problems "transient"Daniel Stenberg2020-09-051-5/+5
| | | | | | | ... because when --retry-all-errors is used, the error isn't necessarily transient at all. Closes #5916
* tool: update --help with categoriesEmil Engler2020-09-041-1/+1
| | | | | | This commit is a part of "--help me if you can" Closes #5680
* tls: add CURLOPT_SSL_EC_CURVES and --curvesMichael Baentsch2020-08-301-0/+3
| | | | Closes #5892
* curl: add --output-dirDaniel Stenberg2020-08-241-0/+9
| | | | | | | | Works with --create-dirs and with -J Add test 3008, 3009, 3011, 3012 and 3013 to verify. Closes #5637
* curl: support XDG_CONFIG_HOME to find .curlrcDaniel Stenberg2020-08-241-1/+1
| | | | | | | | | Added test433 to verify. Updated documentation. Reviewed-by: Jay Satiro Suggested-by: Eli Schwartz Fixes #5829 Closes #5837
* etag: save and use the full received contentsDaniel Stenberg2020-08-241-1/+1
| | | | | | | | | ... which makes it support weak tags and non-standard etags too! Added test case 347 to verify blank incoming ETag: Fixes #5610 Closes #5833
* WIN32: stop forcing narrow-character APIMarcel Raad2020-07-271-1/+1
| | | | | | | | | | Except where the results are only used for character output. getenv is not touched because it's part of the public API, and having it return UTF-8 instead of ANSI would be a breaking change. Fixes https://github.com/curl/curl/issues/5658 Fixes https://github.com/curl/curl/issues/5712 Closes https://github.com/curl/curl/pull/5718
* tool_operate: Don't use Windows CA store as a fallbackJay Satiro2020-06-221-8/+0
| | | | | | | | | | | | | | | | | | | | | | | Background: 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides CURLOPT_CAINFO if both are set. The curl tool will fall back to CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set via CURLOPT_CAINFO. Problem: libcurl may be built with hardcoded paths to a certificate bundle or directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are ignored. Solution: A solution is still being discussed but since there's an impending release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. Ref: https://github.com/curl/curl/issues/5585
* all: fix codespell errorsViktor Szakats2020-05-251-1/+1
| | | | | | Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg Closes https://github.com/curl/curl/pull/5452
* curl: remove -J "informational" written on stdoutDaniel Stenberg2020-05-231-4/+0
| | | | | | | | | | | curl would previously show "curl: Saved to filename 'name from header'" if -J was used and a name was picked from the Content-Disposition header. That output could interfer with other stdout output, such as -w. This commit removes that output line. Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html Reported-by: Коваленко Анатолий Викторович Closes #5435
View Lorry Controller Status