summaryrefslogtreecommitdiff
path: root/embed.fnc
diff options
context:
space:
mode:
authorKarl Williamson <khw@cpan.org>2018-06-27 21:52:47 -0600
committerKarl Williamson <khw@cpan.org>2018-07-05 14:47:19 -0600
commite6a4ffc3f7aa69cbf3e5e83518e40e529a34b75b (patch)
treea861c0d604b080ed7993759dbcd53719389a608c /embed.fnc
parent5af9f82224b62a56edca6981b8638328d12ba98a (diff)
downloadperl-e6a4ffc3f7aa69cbf3e5e83518e40e529a34b75b.tar.gz
Inline dfa for translating from UTF-8
This commit inlines the simple portion of the dfa that translates from UTF-8 to code points, used in functions like utf8_to_uvchr_buf. This dfa has been changed in previous commits so that it is small, and punts on any problematic input, plus 18% of the Hangul syllable code points. (These still come out faster than blead.) The smallness allows it to be inlined, adding <2000 total bytes to the perl text space. The inlined part never calls anything that needs thread context, so that parameter can be removed. I decided to remove it also from the Perl_utf8_to_uvchr_buf() and Perl_utf8n_to_uvchr_error() functions. There is a small risk that someone is actually using those functions instead of the documented macros utf8_to_uvchr_buf() and utf8n_to_uvchr_error(). If so, this can be added back in. Perl_utf8_to_uvchr_msgs() is entirely removed, but the macro utf8_to_uvchr_msgs() which is the normal interface to it is retained unchanged, and it is marked as unstable anyway. This change decreases the number of conditional branches in the Perl statement my $a = ord("\x{foo}") where foo is a non-problematic code point by about 11%, except for ASCII characters, where it is 4%, and those Hangul syllables mentioned above, where it is 7%. Problematic code points fare much worse here than in blead. These are the surrogates, non-characters, and non-Unicode code points. We don't care very much about the speed of handling these code points, which are mostly considered illegal by Unicode anyway. The percentage decrease is higher for the just the function itself, as the measured Perl statement has unchanged overhead. Here are the annotated benchmarks: Key: Ir Instruction read Dr Data read Dw Data write COND conditional branches IND indirect branches _m branch predict miss _m1 level 1 cache miss _mm last cache (e.g. L3) miss - indeterminate percentage (e.g. 1/0) The numbers represent raw counts per loop iteration. translate_utf8_to_uv_007f my $a = ord("\x{007f}") blead dfa Ratio % ----- ----- ------- Ir 395.0 370.0 106.8 Dr 122.0 115.0 106.1 Dw 71.0 61.0 116.4 COND 49.0 47.0 104.3 IND 5.0 5.0 100.0 In all the measurements, the indirect numbers were all zeros and unchanged, and are omitted in this message. translate_utf8_to_uv_07ff my $a = ord("\x{07ff}") blead dfa Ratio % ----- ----- ------- Ir 438.0 390.0 112.3 Dr 128.0 118.0 108.5 Dw 71.0 61.0 116.4 COND 57.0 51.0 111.8 IND 5.0 5.0 100.0 translate_utf8_to_uv_cfff my $a = ord("\x{cfff}") This is the highest Hangul syllable that gets the full reduction. blead dfa Ratio % ----- ----- ------- Ir 457.0 410.0 111.5 Dr 131.0 121.0 108.3 Dw 71.0 61.0 116.4 COND 61.0 55.0 110.9 IND 5.0 5.0 100.0 translate_utf8_to_uv_d000 my $a = ord("\x{d000}") This is the lowest affected Hangul syllable blead dfa Ratio % ----- ----- ------- Ir 457.0 443.0 103.2 Dr 131.0 132.0 99.2 Dw 71.0 71.0 100.0 COND 61.0 57.0 107.0 IND 5.0 5.0 100.0 translate_utf8_to_uv_d7ff my $a = ord("\x{d7ff}") This is the highest affected Hangul syllable blead dfa Ratio % ----- ----- ------- Ir 457.0 443.0 103.2 Dr 131.0 132.0 99.2 Dw 71.0 71.0 100.0 COND 61.0 57.0 107.0 IND 5.0 5.0 100.0 translate_utf8_to_uv_d800 my $a = ord("\x{d800}") This is a surrogate, showing much worse performance, but we don't care blead dfa Ratio % ----- ----- ------- Ir 457.0 515.0 88.7 Dr 131.0 134.0 97.8 Dw 71.0 73.0 97.3 COND 61.0 75.0 81.3 IND 5.0 5.0 100.0 translate_utf8_to_uv_fdd0 my $a = ord("\x{fdd0}") This is a non-char, showing much worse performance, but we don't care blead dfa Ratio % ----- ----- ------- Ir 457.0 548.0 83.4 Dr 131.0 139.0 94.2 Dw 71.0 73.0 97.3 COND 61.0 81.0 75.3 IND 5.0 5.0 100.0 translate_utf8_to_uv_fffd my $a = ord("\x{fffd}") blead dfa Ratio % ----- ----- ------- Ir 457.0 410.0 111.5 Dr 131.0 121.0 108.3 Dw 71.0 61.0 116.4 COND 61.0 55.0 110.9 IND 5.0 5.0 100.0 translate_utf8_to_uv_ffff my $a = ord("\x{ffff}") This is another non-char, showing much worse performance, but we don't care blead dfa Ratio % ----- ----- ------- Ir 457.0 548.0 83.4 Dr 131.0 139.0 94.2 Dw 71.0 73.0 97.3 COND 61.0 81.0 75.3 IND 5.0 5.0 100.0 translate_utf8_to_uv_1fffd my $a = ord("\x{1fffd}") blead dfa Ratio % ----- ----- ------- Ir 476.0 430.0 110.7 Dr 134.0 124.0 108.1 Dw 71.0 61.0 116.4 COND 65.0 59.0 110.2 IND 5.0 5.0 100.0 translate_utf8_to_uv_10fffd my $a = ord("\x{10fffd}") blead dfa Ratio % ----- ----- ------- Ir 476.0 430.0 110.7 Dr 134.0 124.0 108.1 Dw 71.0 61.0 116.4 COND 65.0 59.0 110.2 IND 5.0 5.0 100.0 translate_utf8_to_uv_110000 my $a = ord("\x{110000}") This is a non-Unicode code point, showing much worse performance, but we don't care blead dfa Ratio % ----- ----- ------- Ir 476.0 544.0 87.5 Dr 134.0 137.0 97.8 Dw 71.0 73.0 97.3 COND 65.0 81.0 80.2 IND 5.0 5.0 100.0
Diffstat (limited to 'embed.fnc')
-rw-r--r--embed.fnc13
1 files changed, 10 insertions, 3 deletions
diff --git a/embed.fnc b/embed.fnc
index 39d99f97d5..1e1c629386 100644
--- a/embed.fnc
+++ b/embed.fnc
@@ -1842,16 +1842,23 @@ Aopd |UV |utf8_to_uvchr_buf |NN const U8 *s|NN const U8 *send|NULLOK STRLEN *ret
ApdD |UV |utf8_to_uvuni_buf |NN const U8 *s|NN const U8 *send|NULLOK STRLEN *retlen
pM |bool |check_utf8_print |NN const U8 *s|const STRLEN len
-Adop |UV |utf8n_to_uvchr |NN const U8 *s \
+Adnop |UV |utf8n_to_uvchr |NN const U8 *s \
|STRLEN curlen \
|NULLOK STRLEN *retlen \
|const U32 flags
-Adop |UV |utf8n_to_uvchr_error|NN const U8 *s \
+Adnop |UV |utf8n_to_uvchr_error|NN const U8 *s \
|STRLEN curlen \
|NULLOK STRLEN *retlen \
|const U32 flags \
|NULLOK U32 * errors
-AMdp |UV |utf8n_to_uvchr_msgs|NN const U8 *s \
+AMndi |UV |utf8n_to_uvchr_msgs|NN const U8 *s \
+ |STRLEN curlen \
+ |NULLOK STRLEN *retlen \
+ |const U32 flags \
+ |NULLOK U32 * errors \
+ |NULLOK AV ** msgs
+AMnpd |UV |_utf8n_to_uvchr_msgs_helper \
+ |NN const U8 *s \
|STRLEN curlen \
|NULLOK STRLEN *retlen \
|const U32 flags \