summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorKarl Williamson <khw@cpan.org>2016-09-19 09:52:57 -0600
committerKarl Williamson <khw@cpan.org>2016-09-25 22:24:19 -0600
commit2717076ad3197147ee82d8e263fa3cf7fc9ca19c (patch)
treefe0db5a14c1a16561655449431a351b9050030d7
parentf21517291ac6c737159b2b06bd18b58a063ddb6b (diff)
downloadperl-2717076ad3197147ee82d8e263fa3cf7fc9ca19c.tar.gz
perlapi: Clarifications, nits in Unicode support docs
This also does a white space change to inline.h
-rw-r--r--inline.h20
-rw-r--r--utf8.h38
2 files changed, 40 insertions, 18 deletions
diff --git a/inline.h b/inline.h
index e4b857dbbc..416e162294 100644
--- a/inline.h
+++ b/inline.h
@@ -278,7 +278,7 @@ S_append_utf8_from_native_byte(const U8 byte, U8** dest)
/*
=for apidoc valid_utf8_to_uvchr
-Like L</utf8_to_uvchr_buf>(), but should only be called when it is known that
+Like C<L</utf8_to_uvchr_buf>>, but should only be called when it is known that
the next character in the input UTF-8 string C<s> is well-formed (I<e.g.>,
it passes C<L</isUTF8_CHAR>>. Surrogates, non-character code points, and
non-Unicode code points are allowed.
@@ -365,7 +365,8 @@ be calculated using C<strlen(s)> (which means if you use this option, that C<s>
can't have embedded C<NUL> characters and has to have a terminating C<NUL>
byte). Note that all characters being ASCII constitute 'a valid UTF-8 string'.
-Code points above Unicode, surrogates, and non-character code points are
+This function considers Perl's extended UTF-8 to be valid. That means that
+code points above Unicode, surrogates, and non-character code points are
considered valid by this function.
See also L</is_utf8_invariant_string>(), L</is_utf8_string_loclen>(), and
@@ -401,20 +402,20 @@ Implemented as a macro in utf8.h
=for apidoc is_utf8_string_loc
-Like L</is_utf8_string> but stores the location of the failure (in the
+Like C<L</is_utf8_string>> but stores the location of the failure (in the
case of "utf8ness failure") or the location C<s>+C<len> (in the case of
"utf8ness success") in the C<ep> pointer.
-See also L</is_utf8_string_loclen>() and L</is_utf8_string>().
+See also C<L</is_utf8_string_loclen>>.
=for apidoc is_utf8_string_loclen
-Like L</is_utf8_string>() but stores the location of the failure (in the
+Like C<L</is_utf8_string>> but stores the location of the failure (in the
case of "utf8ness failure") or the location C<s>+C<len> (in the case of
"utf8ness success") in the C<ep>, and the number of UTF-8
encoded characters in the C<el> pointer.
-See also L</is_utf8_string_loc>() and L</is_utf8_string>().
+See also C<L</is_utf8_string_loc>>.
=cut
*/
@@ -528,7 +529,8 @@ failure can be signalled without having to wait for the next read.
=cut
*/
-#define is_utf8_valid_partial_char(s, e) is_utf8_valid_partial_char_flags(s, e, 0)
+#define is_utf8_valid_partial_char(s, e) \
+ is_utf8_valid_partial_char_flags(s, e, 0)
/*
@@ -544,8 +546,8 @@ C<L</is_utf8_valid_partial_char>>. Otherwise C<flags> can be any combination
of the C<UTF8_DISALLOW_I<foo>> flags accepted by C<L</utf8n_to_uvchr>>. If
there is any sequence of bytes that can complete the input partial character in
such a way that a non-prohibited character is formed, the function returns
-TRUE; otherwise FALSE. Non characters cannot be determined based on partial
-character input. But many of the other possible excluded types can be
+TRUE; otherwise FALSE. Non character code points cannot be determined based on
+partial character input. But many of the other possible excluded types can be
determined from just the first one or two bytes.
=cut
diff --git a/utf8.h b/utf8.h
index 392a86a560..7fb63b5c4a 100644
--- a/utf8.h
+++ b/utf8.h
@@ -964,14 +964,22 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and
looking no further than S<C<e - 1>> are well-formed UTF-8, as extended by Perl,
that represents some code point; otherwise it evaluates to 0. If non-zero, the
value gives how many bytes starting at C<s> comprise the code point's
-representation.
+representation. Any bytes remaining before C<e>, but beyond the ones needed to
+form the first code point in C<s>, are not examined.
The code point can be any that will fit in a UV on this machine, using Perl's
extension to official UTF-8 to represent those higher than the Unicode maximum
of 0x10FFFF. That means that this macro is used to efficiently decide if the
-next few bytes in C<s> is legal UTF-8 for a single character. Use
-L</is_utf8_string>(), L</is_utf8_string_loclen>(), and
-L</is_utf8_string_loc>() to check entire strings.
+next few bytes in C<s> is legal UTF-8 for a single character.
+
+Use C<L</isSTRICT_UTF8_CHAR>> to restrict the acceptable code points to those
+defined by Unicode to be fully interchangeable across applications;
+C<L</isC9_STRICT_UTF8_CHAR>> to use the L<Unicode Corrigendum
+#9|http://www.unicode.org/versions/corrigendum9.html> definition of allowable
+code points; and C<L</isUTF8_CHAR_flags>> for a more customized definition.
+
+Use C<L</is_utf8_string>>, C<L</is_utf8_string_loc>>, and
+C<L</is_utf8_string_loclen>> to check entire strings.
Note that it is deprecated to use code points higher than what will fit in an
IV. This macro does not raise any warnings for such code points, treating them
@@ -1004,15 +1012,21 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and
looking no further than S<C<e - 1>> are well-formed UTF-8 that represents some
Unicode code point completely acceptable for open interchange between all
applications; otherwise it evaluates to 0. If non-zero, the value gives how
-many bytes starting at C<s> comprise the code point's representation.
+many bytes starting at C<s> comprise the code point's representation. Any
+bytes remaining before C<e>, but beyond the ones needed to form the first code
+point in C<s>, are not examined.
The largest acceptable code point is the Unicode maximum 0x10FFFF, and must not
be a surrogate nor a non-character code point. Thus this excludes any code
point from Perl's extended UTF-8.
This is used to efficiently decide if the next few bytes in C<s> is
-legal Unicode-acceptable UTF-8 for a single character. Use
-C<L</isC9_STRICT_UTF8_CHAR>> to also accept non-character code points.
+legal Unicode-acceptable UTF-8 for a single character.
+
+Use C<L</isC9_STRICT_UTF8_CHAR>> to use the L<Unicode Corrigendum
+#9|http://www.unicode.org/versions/corrigendum9.html> definition of allowable
+code points; C<L</isUTF8_CHAR>> to check for Perl's extended UTF-8;
+and C<L</isUTF8_CHAR_flags>> for a more customized definition.
=cut
*/
@@ -1034,7 +1048,8 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and
looking no further than S<C<e - 1>> are well-formed UTF-8 that represents some
Unicode non-surrogate code point; otherwise it evaluates to 0. If non-zero,
the value gives how many bytes starting at C<s> comprise the code point's
-representation.
+representation. Any bytes remaining before C<e>, but beyond the ones needed to
+form the first code point in C<s>, are not examined.
The largest acceptable code point is the Unicode maximum 0x10FFFF. This
differs from C<L</isSTRICT_UTF8_CHAR>> only in that it accepts non-character
@@ -1044,6 +1059,9 @@ which said that non-character code points are merely discouraged rather than
completely forbidden in open interchange. See
L<perlunicode/Noncharacter code points>.
+Use C<L</isUTF8_CHAR>> to check for Perl's extended UTF-8; and
+C<L</isUTF8_CHAR_flags>> for a more customized definition.
+
=cut
*/
@@ -1064,7 +1082,9 @@ Evaluates to non-zero if the first few bytes of the string starting at C<s> and
looking no further than S<C<e - 1>> are well-formed UTF-8, as extended by Perl,
that represents some code point, subject to the restrictions given by C<flags>;
otherwise it evaluates to 0. If non-zero, the value gives how many bytes
-starting at C<s> comprise the code point's representation.
+starting at C<s> comprise the code point's representation. Any bytes remaining
+before C<e>, but beyond the ones needed to form the first code point in C<s>,
+are not examined.
If C<flags> is 0, this gives the same results as C<L</isUTF8_CHAR>>;
if C<flags> is C<UTF8_DISALLOW_ILLEGAL_INTERCHANGE>, this gives the same results