delta/go-git.git/src/unicode, branch dev.typeparams

unicode: correctly handle negative runes

2021-02-24T04:00:46+00:00

Is and isExcludingLatin did not handle negative runes when dispatching
to is16. TestNegativeRune covers this along with the existing uint32
casts in IsGraphic, etc. (For tests, I picked the smallest non-Latin-1
code point in each range.)

Updates #43254

Change-Id: I17261b91f0d2b5b5125d19219411b45c480df74f
Reviewed-on: https://go-review.googlesource.com/c/go/+/280493
Run-TryBot: Rob Pike 
TryBot-Result: Go Bot 
Reviewed-by: Rob Pike 
Trust: Emmanuel Odeke

unicode/utf8: document the handling of runes out of range in EncodeRune

2020-09-19T09:43:15+00:00

Document the way EncodeRune currently handles runes which are
out of range.  Also add an example showing that behaviour.

Change-Id: I0f8e7645ae053474ec319085a2bb6d7f73bc137c
Reviewed-on: https://go-review.googlesource.com/c/go/+/255998
Reviewed-by: Rob Pike 
Reviewed-by: Giovanni Bajo 
Trust: Giovanni Bajo 
Run-TryBot: Giovanni Bajo 
TryBot-Result: Go Bot

unicode/utf8: refactor benchmarks for FullRune function

2020-09-10T20:25:45+00:00

BenchmarkFullASCIIRune tests the performance of function utf8.FullRune,
which will be inlined in BenchmarkFullASCIIRune. Since the return value
of FullRune is not referenced, it will be removed as dead code.

This CL makes the FullRune functions return value referenced by a global
variable to avoid this point. In addition, this CL adds one more benchmark
to cover more code paths, and puts them together as sub benchmarks of
BenchmarkFullRune.

Change-Id: I6e79f4c087adf70e351498a4b58d7482dcd1ec4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/233979
Run-TryBot: eric fang 
TryBot-Result: Gobot Gobot 
Reviewed-by: Ian Lance Taylor

unicode: upgrade to Unicode 13.0.0

2020-08-20T13:41:13+00:00

Fixes #40755

Change-Id: I14b3977317994095db8ae1bd873c174641209356
Reviewed-on: https://go-review.googlesource.com/c/go/+/248765
Run-TryBot: Marcel van Lohuizen 
TryBot-Result: Gobot Gobot 
Reviewed-by: Ian Lance Taylor

unicode/utf8: optimize Valid and ValidString for ASCII checks

2020-04-22T14:14:34+00:00

Add a fastpath that uses 32bit loads and compares to check
8 ASCII characters per loop iteration.

This avoids the overhead of comparing and branching
for every byte individually.

Combining two 32bit loads into an uint32 allows the same
code to be used for 32bit and 64bit platforms.

amd64 (Intel i7-3520M):
name                         old time/op  new time/op  delta
ValidTenASCIIChars           15.6ns ± 4%   8.5ns ±14%  -45.27%  (p=0.000 n=10+10)
ValidTenJapaneseChars        50.0ns ± 2%  52.7ns ±15%     ~     (p=0.469 n=10+10)
ValidStringTenASCIIChars     13.5ns ± 1%   7.9ns ± 5%  -41.56%  (p=0.000 n=10+10)
ValidStringTenJapaneseChars  46.3ns ± 2%  45.8ns ± 2%     ~     (p=0.085 n=10+10)

arm (Raspberry Pi 3):
name                         old time/op  new time/op  delta
ValidTenASCIIChars           87.5ns ± 0%  58.5ns ± 0%  -33.11%  (p=0.000 n=9+10)
ValidTenJapaneseChars         359ns ± 0%   384ns ± 0%   +6.96%  (p=0.000 n=10+9)
ValidStringTenASCIIChars     87.5ns ± 0%  57.5ns ± 0%  -34.31%  (p=0.000 n=10+10)
ValidStringTenJapaneseChars   356ns ± 0%   377ns ± 0%   +5.90%  (p=0.000 n=10+10)

Change-Id: I9da942bddb250ee1f0ef7aabb4a8cb48edd9053e
Reviewed-on: https://go-review.googlesource.com/c/go/+/228823
Run-TryBot: Martin Möhrmann 
TryBot-Result: Gobot Gobot 
Reviewed-by: Keith Randall 
Reviewed-by: Brad Fitzpatrick

all: fix typo in RuneSelf, runeSelf comments

2020-01-06T02:46:02+00:00

Fixes #36396

Change-Id: I52190f450fa9ac52fbf4ecdc814e954dc29029cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/213377
Reviewed-by: Daniel Martí 
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot

unicode: upgrade to Unicode 12

2019-11-05T14:34:56+00:00

This does not include an upgrade of golang.org/x/net.
This is optional and best done as a separate CL.

Change-Id: Ifecc3fb6e3b7fe026b4ddefbe637186a3445b0bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204658
Run-TryBot: Marcel van Lohuizen 
TryBot-Result: Gobot Gobot 
Reviewed-by: Russ Cox

unicode/utf8: add link to formal UTF-8 description.

2019-10-06T00:33:51+00:00

Fixes #31590

Change-Id: I7fd6dcc5c34496776439ff0295f18b5fb5cb538a
Reviewed-on: https://go-review.googlesource.com/c/go/+/199141
Reviewed-by: Emmanuel Odeke

all: update to Unicode 11

2019-04-25T20:33:22+00:00

This does *not* update the vendored tables.
A commit updating these tables should follow soon,

Mostly generated running UNICODE_VERSION=11.0.0 in x/text.

Manually updated next.txt file.

Updates golang/go#27945.

Change-Id: I939a01e235aeca898ee9afc99a531e7ad8444e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/154420
Run-TryBot: Marcel van Lohuizen 
TryBot-Result: Gobot Gobot 
Reviewed-by: Robert Griesemer

unicode/utf8: remove some bounds checks from DecodeRune

2019-04-25T00:01:23+00:00

The compiler couldn't quite see that reading p[2] and p[3] was safe.
This change provides a few hints to help it.
First, make sz an int throughout, rather than just when checking the input length.
Second, use <= instead of == in later comparisons.

name                  old time/op  new time/op  delta
DecodeASCIIRune-8     2.62ns ± 3%  2.60ns ± 5%     ~     (p=0.126 n=18+19)
DecodeJapaneseRune-8  4.46ns ±10%  4.01ns ± 5%  -10.00%  (p=0.000 n=19+20)

Change-Id: I2f78a17e38156fbf8b0f5dd6c07c20d6a47e9209
Reviewed-on: https://go-review.googlesource.com/c/go/+/173662
Run-TryBot: Josh Bleecher Snyder 
TryBot-Result: Gobot Gobot 
Reviewed-by: Brad Fitzpatrick