delta/go-git.git/src/runtime/trace/trace_stack_test.go, branch dev.go2go

os, internal/poll: loop on EINTR for all file syscalls

2020-08-19T21:49:56+00:00

When using a FUSE file system, any system call that touches the file
system can return EINTR.

Fixes #40846

Change-Id: I25d32da22cec08dea81ab297291a85ad72db2df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/249178
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Gobot Gobot 
Reviewed-by: Bryan C. Mills

internal/poll, os: loop on EINTR

2020-05-11T22:38:32+00:00

Historically we've assumed that we can install all signal handlers
with the SA_RESTART flag set, and let the system restart slow functions
if a signal is received. Therefore, we don't have to worry about EINTR.

This is only partially true, and we've added EINTR checks already for
connect, and open/read on Darwin, and sendfile on Solaris.

Other cases have turned up in #36644, #38033, and #38836.

Also, #20400 points out that when Go code is included in a C program,
the C program may install its own signal handlers without SA_RESTART.
In that case, Go code will see EINTR no matter what it does.

So, go ahead and check for EINTR. We don't check in the syscall package;
people using syscalls directly may want to check for EINTR themselves.
But we do check for EINTR in the higher level APIs in os and net,
and retry the system call if we see it.

This change looks safe, but of course we may be missing some cases
where we need to check for EINTR. As such cases turn up, we can add
tests to runtime/testdata/testprogcgo/eintr.go, and fix the code.
If there are any such cases, their handling after this change will be
no worse than it is today.

For #22838
Fixes #20400
Fixes #36644
Fixes #38033
Fixes #38836

Change-Id: I7e46ca8cafed0429c7a2386cc9edc9d9d47a6896
Reviewed-on: https://go-review.googlesource.com/c/go/+/232862
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Gobot Gobot 
Reviewed-by: Bryan C. Mills

runtime: don't hold worldsema across mark phase

2020-03-18T19:13:50+00:00

This change makes it so that worldsema isn't held across the mark phase.
This means that various operations like ReadMemStats may now stop the
world during the mark phase, reducing latency on such operations.

Only three such operations are still no longer allowed to occur during
marking: GOMAXPROCS, StartTrace, and StopTrace.

For the former it's because any change to GOMAXPROCS impacts GC mark
background worker scheduling and the details there are tricky.

For the latter two it's because tracing needs to observe consistent GC
start and GC end events, and if StartTrace or StopTrace may stop the
world during marking, then it's possible for it to see a GC end event
without a start or GC start event without an end, respectively.

To ensure that GOMAXPROCS and StartTrace/StopTrace cannot proceed until
marking is complete, the runtime now holds a new semaphore, gcsema,
across the mark phase just like it used to with worldsema.

This change is being landed once more after being reverted in the Go
1.14 release cycle, since CL 215157 allows it to have a positive
effect on system performance.

For the benchmark BenchmarkReadMemStatsLatency in the runtime, which
measures ReadMemStats latencies while the GC is exercised, the tail of
these latencies reduced dramatically on an 8-core machine:

name                   old 50%tile-ns  new 50%tile-ns  delta
ReadMemStatsLatency-8      4.40M ±74%      0.12M ± 2%  -97.35%  (p=0.008 n=5+5)

name                   old 90%tile-ns  new 90%tile-ns  delta
ReadMemStatsLatency-8       102M ± 6%         0M ±14%  -99.79%  (p=0.008 n=5+5)

name                   old 99%tile-ns  new 99%tile-ns  delta
ReadMemStatsLatency-8       147M ±18%         4M ±57%  -97.43%  (p=0.008 n=5+5)

Fixes #19812.

Change-Id: If66c3c97d171524ae29f0e7af4bd33509d9fd0bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/216557
Run-TryBot: Michael Knyszek 
TryBot-Result: Gobot Gobot 
Reviewed-by: Cherry Zhang

Revert "runtime: don't hold worldsema across mark phase"

2020-01-24T23:27:33+00:00

This reverts commit 7b294cdd8df0a9523010f6ffc80c59e64578f34b, CL 182657.

Reason for revert: This change may be causing latency problems
for applications which call ReadMemStats, because it may cause
all goroutines to stop until the GC completes.

https://golang.org/cl/215157 fixes this problem, but it's too
late in the cycle to land that.

Updates #19812.

Change-Id: Iaa26f4dec9b06b9db2a771a44e45f58d0aa8f26d
Reviewed-on: https://go-review.googlesource.com/c/go/+/216358
Run-TryBot: Michael Knyszek 
TryBot-Result: Gobot Gobot 
Reviewed-by: Cherry Zhang

runtime: don't hold worldsema across mark phase

2019-09-04T15:53:59+00:00

This change makes it so that worldsema isn't held across the mark phase.
This means that various operations like ReadMemStats may now stop the
world during the mark phase, reducing latency on such operations.

Only three such operations are still no longer allowed to occur during
marking: GOMAXPROCS, StartTrace, and StopTrace.

For the former it's because any change to GOMAXPROCS impacts GC mark
background worker scheduling and the details there are tricky.

For the latter two it's because tracing needs to observe consistent GC
start and GC end events, and if StartTrace or StopTrace may stop the
world during marking, then it's possible for it to see a GC end event
without a start or GC start event without an end, respectively.

To ensure that GOMAXPROCS and StartTrace/StopTrace cannot proceed until
marking is complete, the runtime now holds a new semaphore, gcsema,
across the mark phase just like it used to with worldsema.

Fixes #19812.

Change-Id: I15d43ed184f711b3d104e8f267fb86e335f86bf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/182657
Run-TryBot: Michael Knyszek 
TryBot-Result: Gobot Gobot 
Reviewed-by: Keith Randall 
Reviewed-by: Cherry Zhang

runtime/trace: add simple benchmarks for user annotation

2018-04-24T17:43:19+00:00

Also, avoid Region creation when tracing is disabled.
Unfortunate side-effect of this change is that we no longer trace
pre-existing regions in tracing, but we can add the feature in
the future when we find it useful and justifiable. Until then,
let's avoid the overhead from this low-level api use as much as
possible.

goos: linux
goarch: amd64
pkg: runtime/trace

// Trace disabled
BenchmarkStartRegion-12 2000000000	         0.66 ns/op	       0 B/op	       0 allocs/op
BenchmarkNewTask-12    	30000000	        40.4 ns/op	      56 B/op	       2 allocs/op

// Trace enabled, -trace=/dev/null
BenchmarkStartRegion-12  5000000	       287 ns/op	      32 B/op	       1 allocs/op
BenchmarkNewTask-12    	 5000000	       283 ns/op	      56 B/op	       2 allocs/op

Also, skip other tests if tracing is already enabled.

Change-Id: Id3028d60b5642fcab4b09a74fd7d79361a3861e5
Reviewed-on: https://go-review.googlesource.com/109115
Reviewed-by: Peter Weinberger

runtime/trace: fix TestTraceSymbolize when GOMAXPROCS=1

2018-02-13T22:55:49+00:00

CL 92916 added the GOMAXPROCS test in TestTraceSymbolize.

This test only succeeds when the value of GOMAXPROCS changes.

Since the test calls runtime.GOMAXPROCS(1), it will fails
on machines where GOMAXPROCS=1.

This change fixes the test by calling runtime.GOMAXPROCS(oldGoMaxProcs+1).

Fixes #23816.

Change-Id: I1183dbbd7db6077cbd7fa0754032ff32793b2195
Reviewed-on: https://go-review.googlesource.com/93735
Run-TryBot: David du Colombier <0intro@gmail.com>
Reviewed-by: Hyang-Ah Hana Kim 
TryBot-Result: Gobot Gobot

runtime/trace: add stack tests for GOMAXPROCS

2018-02-13T18:45:32+00:00

and reorganize test log messages for stack dumps
for easier debugging.

The error log will be formatted like the following:

	trace_stack_test.go:282: Did not match event GoCreate with stack
		 runtime/trace_test.TestTraceSymbolize	 :39
		 testing.tRunner			 :0

		Seen 30 events of the type
		Offset 1890
		 runtime/trace_test.TestTraceSymbolize	/go/src/runtime/trace/trace_stack_test.go:30
		 testing.tRunner			/go/src/testing/testing.go:777
		Offset 1899
		 runtime/trace_test.TestTraceSymbolize	/go/src/runtime/trace/trace_stack_test.go:30
		 testing.tRunner			/go/src/testing/testing.go:777
		 ...

Change-Id: I0468de04507d6ae38ba84d99d13f7bf592e8d115
Reviewed-on: https://go-review.googlesource.com/92916
Reviewed-by: Heschi Kreinick 
Run-TryBot: Hyang-Ah Hana Kim

runtime/trace: better output when test fails

2017-04-14T12:21:02+00:00

Change-Id: I108d15eb4cd25904bb76de4ed7548c039c69d1a3
Reviewed-on: https://go-review.googlesource.com/40590
Run-TryBot: David Lazar 
TryBot-Result: Gobot Gobot 
Reviewed-by: Austin Clements

os: use poller for file I/O

2017-02-15T19:31:55+00:00

This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.

For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.

Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.

This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.

Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.

Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.

Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Gobot Gobot 
Reviewed-by: Russ Cox