delta/go-git.git/src/archive, branch dev.typeparams

archive/zip: only preallocate File slice if reasonably sized

2021-05-25T23:41:42+00:00

Since the number of files in the EOCD record isn't validated, it isn't
safe to preallocate Reader.Files using that field. A malformed archive
can indicate it contains up to 1 << 128 - 1 files. We can still safely
preallocate the slice by checking if the specified number of files in
the archive is reasonable, given the size of the archive.

Thanks to the OSS-Fuzz project for discovering this issue and to
Emmanuel Odeke for reporting it.

Fixes #46242
Fixes CVE-2021-33196

Change-Id: I3c76d8eec178468b380d87fdb4a3f2cb06f0ee76
Reviewed-on: https://go-review.googlesource.com/c/go/+/318909
Trust: Roland Shoemaker 
Trust: Katie Hockman 
Trust: Joe Tsai 
Run-TryBot: Roland Shoemaker 
TryBot-Result: Go Bot 
Reviewed-by: Katie Hockman 
Reviewed-by: Joe Tsai

archive/zip: add File.OpenRaw, Writer.CreateRaw, Writer.Copy

2021-05-03T21:11:47+00:00

These new methods provide support for cases where performance is a
primary concern. For example, copying files from an existing zip to a
new zip without incurring the decompression and compression overhead.
Using an optimized, external compression method and writing the output
to a zip archive. And compressing file contents in parallel and then
sequentially writing the compressed bytes to a zip archive.

TestWriterCopy is copied verbatim from https://github.com/rsc/zipmerge

Fixes #34974

Change-Id: Iade5bc245ba34cdbb86364bf59f79f38bb9e2eb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/312310
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Go Bot 
Reviewed-by: Ian Lance Taylor 
Trust: Carlos Amedee

archive/zip: only return directory once via io/fs.FS

2021-04-29T19:04:31+00:00

While we're here fix the ModTime value for directories.

Fixes #43872
Fixes #45345

Change-Id: I155e6517713ef6a9482b9431f1167a44337c6ad2
Reviewed-on: https://go-review.googlesource.com/c/go/+/311530
Trust: Ian Lance Taylor 
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Go Bot 
Reviewed-by: Jeremy Faller

archive/zip: fix imports block of biggestZipBytes generator

2021-04-18T17:40:12+00:00

Fixes #45529

Change-Id: I4d64c40aa6733b783dc4066e222f17abeb7ad413
Reviewed-on: https://go-review.googlesource.com/c/go/+/309357
Reviewed-by: Joe Tsai 
Trust: Joe Tsai 
Trust: Ian Lance Taylor 
Run-TryBot: Joe Tsai 
TryBot-Result: Go Bot

archive/tar: replace os.MkdirTemp with T.TempDir

2021-04-13T21:06:12+00:00

Updates #45402

Change-Id: I296f8c676c68ed1e10b6ad1a17b5b23d2c395252
Reviewed-on: https://go-review.googlesource.com/c/go/+/309355
Run-TryBot: Ian Lance Taylor 
Reviewed-by: Joe Tsai 
Reviewed-by: Ian Lance Taylor 
Trust: Joe Tsai 
TryBot-Result: Go Bot

archive/zip: fix character device handling in fileModeToUnixMode

2021-04-03T20:16:00+00:00

The switch case for fs.ModeDevice can only be reached for block devices
while character devices match fs.ModeDevice | fs.ModeCharDevice. This
would cause character devices to wrongly be reported as regular files.

This bug has existed since the switch was first introduced in CL 5624048.

Change-Id: Icdbedb015e5376b385b3115d2e4574daa052f796
Reviewed-on: https://go-review.googlesource.com/c/go/+/300891
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Go Bot 
Reviewed-by: Ian Lance Taylor 
Trust: Emmanuel Odeke

archive/zip: fix panic in Reader.Open

2021-03-10T18:18:28+00:00

When operating on a Zip file that contains a file prefixed with "../",
Open(...) would cause a panic in toValidName when attempting to strip
the prefixed path components.

Fixes CVE-2021-27919
Fixes #44916

Change-Id: Ic755d8126cb0897e2cbbdacf572439c38dde7b35
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004761
Reviewed-by: Filippo Valsorda 
Reviewed-by: Russ Cox 
Reviewed-by: Katie Hockman 
Reviewed-on: https://go-review.googlesource.com/c/go/+/300489
Trust: Katie Hockman 
Run-TryBot: Katie Hockman 
TryBot-Result: Go Bot 
Reviewed-by: Alexander Rakoczy 
Reviewed-by: Filippo Valsorda

all: go fmt std cmd (but revert vendor)

2021-02-20T03:54:50+00:00

Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).

Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild

Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/294430
Trust: Russ Cox 
Run-TryBot: Russ Cox 
TryBot-Result: Go Bot 
Reviewed-by: Jason A. Donenfeld 
Reviewed-by: Ian Lance Taylor

archive/tar: detect out of bounds accesses in PAX records resulting from padded lengths

2021-02-09T05:28:50+00:00

Handles the case in which padding of a PAX record's length field
violates invariants about the formatting of record, whereby it no
longer matches the prescribed format:

    "%d %s=%s\n", , , 

as per:

    https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_03

0-padding, and paddings of other sorts weren't handled and we assumed
that only non-padded decimal lengths would be passed in.
Added test cases to ensure that the parsing still proceeds as expected.

The prior crashing repro:

    0000000000000000000000000000000030 mtime=1432668921.098285006\n30 ctime=2147483649.15163319

exposed the fallacy in the code, that assumed that the length would ALWAYS be a
non-padded decimal length string.

This bug has existed since Go1.1 as per CL 6700047.

Thanks to Josh Bleecher Snyder for fuzzing this package, and thanks to Tom
Thorogood for advocacy, raising parity with GNU Tar, but for providing more test cases.

Fixes #40196

Change-Id: I32e0af4887bc9221481bd9e8a5120a79f177f08c
Reviewed-on: https://go-review.googlesource.com/c/go/+/289629
Trust: Emmanuel Odeke 
Trust: Joe Tsai 
Run-TryBot: Emmanuel Odeke 
TryBot-Result: Go Bot 
Reviewed-by: Joe Tsai

archive/tar: fix typo in comment

2021-01-08T02:03:24+00:00

Change-Id: Ifcc565b34b3c3bb7ee62bb0525648a5d2895bf0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/282013
Reviewed-by: Ian Lance Taylor 
Trust: Alberto Donizetti