diff options
author | Karl Williamson <khw@cpan.org> | 2016-09-19 09:52:57 -0600 |
---|---|---|
committer | Karl Williamson <khw@cpan.org> | 2016-09-25 22:24:19 -0600 |
commit | 2717076ad3197147ee82d8e263fa3cf7fc9ca19c (patch) | |
tree | fe0db5a14c1a16561655449431a351b9050030d7 | |
parent | f21517291ac6c737159b2b06bd18b58a063ddb6b (diff) | |
download | perl-2717076ad3197147ee82d8e263fa3cf7fc9ca19c.tar.gz |
perlapi: Clarifications, nits in Unicode support docs
This also does a white space change to inline.h
-rw-r--r-- | inline.h | 20 | ||||
-rw-r--r-- | utf8.h | 38 |
2 files changed, 40 insertions, 18 deletions
@@ -278,7 +278,7 @@ S_append_utf8_from_native_byte(const U8 byte, U8** dest) /* =for apidoc valid_utf8_to_uvchr -Like L</utf8_to_uvchr_buf>(), but should only be called when it is known that +Like C<L</utf8_to_uvchr_buf>>, but should only be called when it is known that the next character in the input UTF-8 string C<s> is well-formed (I<e.g.>, it passes C<L</isUTF8_CHAR>>. Surrogates, non-character code points, and non-Unicode code points are allowed. @@ -365,7 +365,8 @@ be calculated using C<strlen(s)> (which means if you use this option, that C<s> can't have embedded C<NUL> characters and has to have a terminating C<NUL> byte). Note that all characters being ASCII constitute 'a valid UTF-8 string'. -Code points above Unicode, surrogates, and non-character code points are +This function considers Perl's extended UTF-8 to be valid. That means that +code points above Unicode, surrogates, and non-character code points are considered valid by this function. See also L</is_utf8_invariant_string>(), L</is_utf8_string_loclen>(), and @@ -401,20 +402,20 @@ Implemented as a macro in utf8.h =for apidoc is_utf8_string_loc -Like L</is_utf8_string> but stores the location of the failure (in the +Like C<L</is_utf8_string>> but stores the location of the failure (in the case of "utf8ness failure") or the location C<s>+C<len> (in the case of "utf8ness success") in the C<ep> pointer. -See also L</is_utf8_string_loclen>() and L</is_utf8_string>(). +See also C<L</is_utf8_string_loclen>>. =for apidoc is_utf8_string_loclen -Like L</is_utf8_string>() but stores the location of the failure (in the +Like C<L</is_utf8_string>> but stores the location of the failure (in the case of "utf8ness failure") or the location C<s>+C<len> (in the case of "utf8ness success") in the C<ep>, and the number of UTF-8 encoded characters in the C<el> pointer. -See also L</is_utf8_string_loc>() and L</is_utf8_string>(). +See also C<L</is_utf8_string_loc>>. =cut */ @@ -528,7 +529,8 @@ failure can be signalled without having to wait for the next read. =cut */ -#define is_utf8_valid_partial_char(s, e) is_utf8_valid_partial_char_flags(s, e, 0) +#define is_utf8_valid_partial_char(s, e) \ + is_utf8_valid_partial_char_flags(s, e, 0) /* @@ -544,8 +546,8 @@ C<L</is_utf8_valid_partial_char>>. Otherwise C<flags> can be any combination of the C<UTF8_DISALLOW_I<foo>> flags accepted by C<L</utf8n_to_uvchr>>. If there is any sequence of bytes that can complete the input partial character in such a way that a non-prohibited character is formed, the function returns -TRUE; otherwise FALSE. Non characters cannot be determined based on partial -character input. But many of the other possible excluded types can be +TRUE; otherwise FALSE. Non character code points cannot be determined based on +partial character input. But many of the other possible excluded types can be determined from just the first one or two bytes. =cut @@ -964,14 +964,22 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and looking no further than S<C<e - 1>> are well-formed UTF-8, as extended by Perl, that represents some code point; otherwise it evaluates to 0. If non-zero, the value gives how many bytes starting at C<s> comprise the code point's -representation. +representation. Any bytes remaining before C<e>, but beyond the ones needed to +form the first code point in C<s>, are not examined. The code point can be any that will fit in a UV on this machine, using Perl's extension to official UTF-8 to represent those higher than the Unicode maximum of 0x10FFFF. That means that this macro is used to efficiently decide if the -next few bytes in C<s> is legal UTF-8 for a single character. Use -L</is_utf8_string>(), L</is_utf8_string_loclen>(), and -L</is_utf8_string_loc>() to check entire strings. +next few bytes in C<s> is legal UTF-8 for a single character. + +Use C<L</isSTRICT_UTF8_CHAR>> to restrict the acceptable code points to those +defined by Unicode to be fully interchangeable across applications; +C<L</isC9_STRICT_UTF8_CHAR>> to use the L<Unicode Corrigendum +#9|http://www.unicode.org/versions/corrigendum9.html> definition of allowable +code points; and C<L</isUTF8_CHAR_flags>> for a more customized definition. + +Use C<L</is_utf8_string>>, C<L</is_utf8_string_loc>>, and +C<L</is_utf8_string_loclen>> to check entire strings. Note that it is deprecated to use code points higher than what will fit in an IV. This macro does not raise any warnings for such code points, treating them @@ -1004,15 +1012,21 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and looking no further than S<C<e - 1>> are well-formed UTF-8 that represents some Unicode code point completely acceptable for open interchange between all applications; otherwise it evaluates to 0. If non-zero, the value gives how -many bytes starting at C<s> comprise the code point's representation. +many bytes starting at C<s> comprise the code point's representation. Any +bytes remaining before C<e>, but beyond the ones needed to form the first code +point in C<s>, are not examined. The largest acceptable code point is the Unicode maximum 0x10FFFF, and must not be a surrogate nor a non-character code point. Thus this excludes any code point from Perl's extended UTF-8. This is used to efficiently decide if the next few bytes in C<s> is -legal Unicode-acceptable UTF-8 for a single character. Use -C<L</isC9_STRICT_UTF8_CHAR>> to also accept non-character code points. +legal Unicode-acceptable UTF-8 for a single character. + +Use C<L</isC9_STRICT_UTF8_CHAR>> to use the L<Unicode Corrigendum +#9|http://www.unicode.org/versions/corrigendum9.html> definition of allowable +code points; C<L</isUTF8_CHAR>> to check for Perl's extended UTF-8; +and C<L</isUTF8_CHAR_flags>> for a more customized definition. =cut */ @@ -1034,7 +1048,8 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and looking no further than S<C<e - 1>> are well-formed UTF-8 that represents some Unicode non-surrogate code point; otherwise it evaluates to 0. If non-zero, the value gives how many bytes starting at C<s> comprise the code point's -representation. +representation. Any bytes remaining before C<e>, but beyond the ones needed to +form the first code point in C<s>, are not examined. The largest acceptable code point is the Unicode maximum 0x10FFFF. This differs from C<L</isSTRICT_UTF8_CHAR>> only in that it accepts non-character @@ -1044,6 +1059,9 @@ which said that non-character code points are merely discouraged rather than completely forbidden in open interchange. See L<perlunicode/Noncharacter code points>. +Use C<L</isUTF8_CHAR>> to check for Perl's extended UTF-8; and +C<L</isUTF8_CHAR_flags>> for a more customized definition. + =cut */ @@ -1064,7 +1082,9 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and looking no further than S<C<e - 1>> are well-formed UTF-8, as extended by Perl, that represents some code point, subject to the restrictions given by C<flags>; otherwise it evaluates to 0. If non-zero, the value gives how many bytes -starting at C<s> comprise the code point's representation. +starting at C<s> comprise the code point's representation. Any bytes remaining +before C<e>, but beyond the ones needed to form the first code point in C<s>, +are not examined. If C<flags> is 0, this gives the same results as C<L</isUTF8_CHAR>>; if C<flags> is C<UTF8_DISALLOW_ILLEGAL_INTERCHANGE>, this gives the same results |