delta/go-git.git/src/encoding/gob, branch dev.debug

encoding/gob: warn about decoding data from untrusted sources

2017-06-29T03:24:29+00:00

And some double space after period cleanup while I'm here.
I guess my previous regexps missed these. My next cleaner should
probably use go/ast instead of perl.

Updates #20221

Change-Id: Idb051e7ac3a7fb1fb86e015f709e32139d065d92
Reviewed-on: https://go-review.googlesource.com/47094
Reviewed-by: Ian Lance Taylor 
Reviewed-by: Rob Pike

encoding/gob: speedup decoding of maps by zeroing values

2017-05-26T21:06:24+00:00

Instead of allocating a new reflect.Value object on every loop we zero it.

DecodeComplex128Slice-8  13.1µs ± 7%  13.2µs ± 8%     ~     (p=0.347 n=18+19)
DecodeFloat64Slice-8     8.13µs ± 5%  8.00µs ± 3%     ~     (p=0.168 n=20+19)
DecodeInt32Slice-8       8.27µs ± 5%  8.08µs ± 5%   -2.27%  (p=0.001 n=19+18)
DecodeStringSlice-8      17.9µs ±12%  17.8µs ±11%     ~     (p=0.989 n=20+19)
DecodeInterfaceSlice-8    163µs ±10%   159µs ± 4%     ~     (p=0.057 n=19+19)
DecodeMap-8               220µs ± 2%   183µs ± 1%  -17.07%  (p=0.000 n=19+18)

Updates #19525

Change-Id: I27f8edd4761787f6b9928d34cefa08a34a6e25b2
Reviewed-on: https://go-review.googlesource.com/39203
Reviewed-by: Brad Fitzpatrick 
Reviewed-by: Rob Pike 
Run-TryBot: Brad Fitzpatrick 
TryBot-Result: Gobot Gobot

encoding/gob: use MakeMapWithSize when decoding map

2017-05-03T02:43:40+00:00

This allows to pre-allocate the final size of the hashmap and avoid
re-allocating as we insert entries. Furthermore for the current
implementation of the hashmap it allows avoiding several rounds of
evacuating hashmap entries after each re-allocation.

DecodeComplex128Slice-8  51.9µs ± 1%  51.9µs ± 2%     ~     (p=0.797 n=30+29)
DecodeFloat64Slice-8     31.5µs ± 2%  31.6µs ± 2%     ~     (p=0.050 n=28+28)
DecodeInt32Slice-8       32.0µs ± 2%  31.9µs ± 3%     ~     (p=0.666 n=29+28)
DecodeStringSlice-8      57.7µs ± 2%  57.8µs ± 3%     ~     (p=0.780 n=27+30)
DecodeInterfaceSlice-8    498µs ± 2%   495µs ± 2%     ~     (p=0.070 n=28+29)
DecodeMap-8               300µs ± 2%   230µs ± 5%  -23.31%  (p=0.000 n=27+27)

Updates #19525

Change-Id: Ia7233da49f05bae7a86c064d9ecebca966f5f2f7
Reviewed-on: https://go-review.googlesource.com/40113
Run-TryBot: Brad Fitzpatrick 
TryBot-Result: Gobot Gobot 
Reviewed-by: Brad Fitzpatrick

encoding/gob: replace RWMutex usage with sync.Map

2017-04-27T15:34:57+00:00

This provides a significant speedup for encoding and decoding when
using many CPU cores.

name                        old time/op  new time/op  delta
EndToEndPipe                5.26µs ± 2%  5.38µs ± 7%     ~     (p=0.121 n=8+7)
EndToEndPipe-6              1.86µs ± 5%  1.80µs ±11%     ~     (p=0.442 n=8+8)
EndToEndPipe-48             1.39µs ± 2%  1.41µs ± 4%     ~     (p=0.645 n=8+8)
EndToEndByteBuffer          1.54µs ± 5%  1.57µs ± 5%     ~     (p=0.130 n=8+8)
EndToEndByteBuffer-6         620ns ± 6%   310ns ± 8%  -50.04%  (p=0.000 n=8+8)
EndToEndByteBuffer-48        506ns ± 4%   110ns ± 3%  -78.22%  (p=0.000 n=8+8)
EndToEndSliceByteBuffer      149µs ± 3%   153µs ± 5%   +2.80%  (p=0.021 n=8+8)
EndToEndSliceByteBuffer-6    103µs ±17%    31µs ±12%  -70.06%  (p=0.000 n=8+8)
EndToEndSliceByteBuffer-48  93.2µs ± 2%  18.0µs ± 5%  -80.66%  (p=0.000 n=7+8)
EncodeComplex128Slice       20.6µs ± 5%  20.9µs ± 8%     ~     (p=0.959 n=8+8)
EncodeComplex128Slice-6     4.10µs ±10%  3.75µs ± 8%   -8.58%  (p=0.004 n=8+7)
EncodeComplex128Slice-48    1.14µs ± 2%  0.81µs ± 2%  -28.98%  (p=0.000 n=8+8)
EncodeFloat64Slice          10.2µs ± 7%  10.1µs ± 6%     ~     (p=0.694 n=7+8)
EncodeFloat64Slice-6        2.01µs ± 6%  1.80µs ±11%  -10.30%  (p=0.004 n=8+8)
EncodeFloat64Slice-48        701ns ± 3%   408ns ± 2%  -41.72%  (p=0.000 n=8+8)
EncodeInt32Slice            11.8µs ± 7%  11.7µs ± 6%     ~     (p=0.463 n=8+7)
EncodeInt32Slice-6          2.32µs ± 4%  2.06µs ± 5%  -10.89%  (p=0.000 n=8+8)
EncodeInt32Slice-48          731ns ± 2%   445ns ± 2%  -39.10%  (p=0.000 n=7+8)
EncodeStringSlice           9.13µs ± 9%  9.18µs ± 8%     ~     (p=0.798 n=8+8)
EncodeStringSlice-6         1.91µs ± 5%  1.70µs ± 5%  -11.07%  (p=0.000 n=8+8)
EncodeStringSlice-48         679ns ± 3%   397ns ± 3%  -41.50%  (p=0.000 n=8+8)
EncodeInterfaceSlice         449µs ±11%   461µs ± 9%     ~     (p=0.328 n=8+8)
EncodeInterfaceSlice-6       503µs ± 7%    88µs ± 7%  -82.51%  (p=0.000 n=7+8)
EncodeInterfaceSlice-48      335µs ± 8%    22µs ± 1%  -93.55%  (p=0.000 n=8+7)
DecodeComplex128Slice       67.2µs ± 4%  67.0µs ± 6%     ~     (p=0.721 n=8+8)
DecodeComplex128Slice-6     22.0µs ± 8%  18.9µs ± 5%  -14.44%  (p=0.000 n=8+8)
DecodeComplex128Slice-48    46.8µs ± 3%  34.9µs ± 3%  -25.48%  (p=0.000 n=8+8)
DecodeFloat64Slice          39.4µs ± 4%  40.3µs ± 3%     ~     (p=0.105 n=8+8)
DecodeFloat64Slice-6        16.1µs ± 2%  11.2µs ± 7%  -30.64%  (p=0.001 n=6+7)
DecodeFloat64Slice-48       38.1µs ± 3%  24.0µs ± 7%  -37.10%  (p=0.000 n=8+8)
DecodeInt32Slice            39.1µs ± 4%  40.1µs ± 5%     ~     (p=0.083 n=8+8)
DecodeInt32Slice-6          16.3µs ±21%  10.6µs ± 1%  -35.17%  (p=0.000 n=8+7)
DecodeInt32Slice-48         36.5µs ± 6%  21.9µs ± 9%  -39.89%  (p=0.000 n=8+8)
DecodeStringSlice           82.9µs ± 6%  85.5µs ± 5%     ~     (p=0.121 n=8+7)
DecodeStringSlice-6         32.4µs ±11%  26.8µs ±16%  -17.37%  (p=0.000 n=8+8)
DecodeStringSlice-48        76.0µs ± 2%  57.0µs ± 5%  -25.02%  (p=0.000 n=8+8)
DecodeInterfaceSlice         718µs ± 4%   752µs ± 5%   +4.83%  (p=0.038 n=8+8)
DecodeInterfaceSlice-6       500µs ± 6%   165µs ± 7%  -66.95%  (p=0.000 n=7+8)
DecodeInterfaceSlice-48      470µs ± 5%   120µs ± 6%  -74.55%  (p=0.000 n=8+7)
DecodeMap                   3.29ms ± 5%  3.34ms ± 5%     ~     (p=0.279 n=8+8)
DecodeMap-6                 7.73ms ± 8%  7.53ms ±18%     ~     (p=0.779 n=7+8)
DecodeMap-48                7.46ms ± 6%  7.71ms ± 3%     ~     (p=0.161 n=8+8)

https://perf.golang.org/search?q=upload:20170426.4

Change-Id: I335874028ef8d7c991051004f8caadd16c92d5cc
Reviewed-on: https://go-review.googlesource.com/41872
Run-TryBot: Bryan Mills 
TryBot-Result: Gobot Gobot 
Reviewed-by: Ian Lance Taylor

encoding/gob: parallelize Encode/Decode benchmarks

2017-04-26T19:04:18+00:00

Results remain comparable with the non-parallel version with -cpu=1:

benchmark                              old ns/op     new ns/op     delta
BenchmarkEndToEndPipe                  6200          6171          -0.47%
BenchmarkEndToEndPipe-6                1073          1024          -4.57%
BenchmarkEndToEndByteBuffer            2925          2664          -8.92%
BenchmarkEndToEndByteBuffer-6          516           560           +8.53%
BenchmarkEndToEndSliceByteBuffer       231683        237450        +2.49%
BenchmarkEndToEndSliceByteBuffer-6     59080         59452         +0.63%
BenchmarkEncodeComplex128Slice         67541         66003         -2.28%
BenchmarkEncodeComplex128Slice-6       72740         11316         -84.44%
BenchmarkEncodeFloat64Slice            25769         27899         +8.27%
BenchmarkEncodeFloat64Slice-6          26655         4557          -82.90%
BenchmarkEncodeInt32Slice              18685         18845         +0.86%
BenchmarkEncodeInt32Slice-6            18389         3462          -81.17%
BenchmarkEncodeStringSlice             19089         19354         +1.39%
BenchmarkEncodeStringSlice-6           20155         3237          -83.94%
BenchmarkEncodeInterfaceSlice          659601        677129        +2.66%
BenchmarkEncodeInterfaceSlice-6        640974        251621        -60.74%
BenchmarkDecodeComplex128Slice         117130        129955        +10.95%
BenchmarkDecodeComplex128Slice-6       155447        24924         -83.97%
BenchmarkDecodeFloat64Slice            67695         68776         +1.60%
BenchmarkDecodeFloat64Slice-6          82966         15225         -81.65%
BenchmarkDecodeInt32Slice              63102         62733         -0.58%
BenchmarkDecodeInt32Slice-6            77857         13003         -83.30%
BenchmarkDecodeStringSlice             130240        129562        -0.52%
BenchmarkDecodeStringSlice-6           165500        31507         -80.96%
BenchmarkDecodeInterfaceSlice          937637        1060835       +13.14%
BenchmarkDecodeInterfaceSlice-6        973495        270613        -72.20%

updates #18177

Change-Id: Ib3579010faa70827d5cbd02a826dbbb66ca13eb7
Reviewed-on: https://go-review.googlesource.com/36722
Run-TryBot: Bryan Mills 
TryBot-Result: Gobot Gobot 
Reviewed-by: Brad Fitzpatrick

all: remove some unused parameters in test code

2017-04-25T14:38:10+00:00

Mostly unnecessary *testing.T arguments.

Found with github.com/mvdan/unparam.

Change-Id: Ifb955cb88f2ce8784ee4172f4f94d860fa36ae9a
Reviewed-on: https://go-review.googlesource.com/41691
Run-TryBot: Daniel Martí 
TryBot-Result: Gobot Gobot 
Reviewed-by: Ian Lance Taylor

all: remove redundant returns

2017-04-19T20:03:51+00:00

Returns at the end of func bodies where the funcs have no return values
are pointless.

Change-Id: I0da5ea78671503e41a9f56dd770df8c919310ce5
Reviewed-on: https://go-review.googlesource.com/41093
Run-TryBot: Daniel Martí 
Reviewed-by: Brad Fitzpatrick 
TryBot-Result: Gobot Gobot

encoding/gob: Speedup map decoding by reducing the allocations.

2017-03-24T19:36:14+00:00

The improvementis achieved in encoding/gob/decode.go decodeMap by
allocate keyInstr and elemInstr only once and pass it to
decodeIntoValue, instead of allocating a new instance on every loop
cycle.

name                     old time/op  new time/op  delta
DecodeComplex128Slice-8  64.2µs ±10%  62.2µs ± 8%     ~     (p=0.686 n=4+4)
DecodeFloat64Slice-8     37.1µs ± 3%  36.5µs ± 5%     ~     (p=0.343 n=4+4)
DecodeInt32Slice-8       33.7µs ± 3%  32.7µs ± 4%     ~     (p=0.200 n=4+4)
DecodeStringSlice-8      59.7µs ± 5%  57.3µs ± 1%     ~     (p=0.114 n=4+4)
DecodeInterfaceSlice-8    543µs ± 7%   497µs ± 3%     ~     (p=0.057 n=4+4)
DecodeMap-8              3.78ms ± 8%  2.66ms ± 2%  -29.69%  (p=0.029 n=4+4)

Updates #19525

Change-Id: Iec5fa4530de76f0a70da5de8a129a567b4aa096e
Reviewed-on: https://go-review.googlesource.com/38317
Reviewed-by: Brad Fitzpatrick 
Reviewed-by: Rob Pike 
Run-TryBot: Brad Fitzpatrick 
TryBot-Result: Gobot Gobot

encoding/gob: document the extra byte after a singleton

2017-03-21T04:07:46+00:00

This paragraph has been added, as the notion was missing from the
documentation.

If a value is passed to Encode and the type is not a struct (or pointer to struct,
etc.), for simplicity of processing it is represented as a struct of one field.
The only visible effect of this is to encode a zero byte after the value, just as
after the last field of an encoded struct, so that the decode algorithm knows when
the top-level value is complete.

Fixes #16978

Change-Id: I5f008e792d1b6fe80d2e026a7ff716608889db32
Reviewed-on: https://go-review.googlesource.com/38414
Reviewed-by: Ian Lance Taylor

encoding/gob: speedup floats encoding and decoding

2017-03-18T18:30:41+00:00

By replacing bytes-reversing routines with bits.ReverseBytes64 calls.

name                     old time/op  new time/op  delta
EncodeComplex128Slice-4  35.1µs ± 1%  23.2µs ± 2%  -33.94%  (p=0.000 n=20+20)
EncodeFloat64Slice-4     17.9µs ± 1%  11.0µs ± 1%  -38.36%  (p=0.000 n=17+18)

name                     old time/op  new time/op  delta
DecodeComplex128Slice-4  79.7µs ± 0%  69.9µs ± 1%  -12.31%  (p=0.000 n=20+20)
DecodeFloat64Slice-4     47.3µs ± 1%  42.2µs ± 1%  -10.65%  (p=0.000 n=17+17)

Change-Id: I91a6401c6009b5712fca6258dd1e57c6fe68ea64
Reviewed-on: https://go-review.googlesource.com/38352
Run-TryBot: Alberto Donizetti 
TryBot-Result: Gobot Gobot 
Reviewed-by: Brad Fitzpatrick