urlapi: URL decode percent-encoded host names

The host name is stored decoded and can be encoded when used to extract
the full URL. By default when extracting the URL, the host name will not
be URL encoded to work as similar as possible as before. When not URL
encoding the host name, the '%' character will however still be encoded.

Getting the URL with the CURLU_URLENCODE flag set will percent encode
the host name part.

As a bonus, setting the host name part with curl_url_set() no longer
accepts a name that contains space, CR or LF.

Test 1560 has been extended to verify percent encodings.

Reported-by: Noam Moshe
Reported-by: Sharon Brizinov
Reported-by: Raul Onitza-Klugman
Reported-by: Kirill Efimov
Fixes #7830
Closes #7834

1 files changed, 9 insertions, 1 deletions

diff --git a/docs/libcurl/curl_url_get.3 b/docs/libcurl/curl_url_get.3
index 7fbdff0ed..51cc37cc2 100644
--- a/docs/libcurl/curl_url_get.3
+++ b/docs/libcurl/curl_url_get.3
@@ -56,7 +56,7 @@ default port for the scheme.
 .IP CURLU_URLDECODE
 Asks \fIcurl_url_get(3)\fP to URL decode the contents before returning it. It
 will not attempt to decode the scheme, the port number or the full URL.
-
+´
 The query component will also get plus-to-space conversion as a bonus when
 this bit is set.
 
@@ -66,6 +66,14 @@ encoding.
 
 If there's any byte values lower than 32 in the decoded string, the get
 operation will return an error instead.
+.IP CURLU_URLENCODE
+If set, will make \fIcurl_url_get(3)\fP URL encode the host name part when a
+full URL is retrieved. If not set (default), libcurl returns the URL with the
+host name "raw" to support IDN names to appear as-is. IDN host names are
+typically using non-ASCII bytes that otherwise will be percent-encoded.
+
+Note that even when not asking for URL encoding, the '%' (byte 37) will be URL
+encoded to make sure the host name remains valid.
 .SH PARTS
 .IP CURLUPART_URL
 When asked to return the full URL, \fIcurl_url_get(3)\fP will return a