delta/go-git.git/src/runtime, branch master

runtime/cgo: store M for C-created thread in pthread key

2023-05-17T21:53:11+00:00

This reapplies CL 485500, with a fix drafted in CL 492987 incorporated.

CL 485500 is reverted due to #60004 and #60007. #60004 is fixed in
CL 492743. #60007 is fixed in CL 492987 (incorporated in this CL).

[Original CL 485500 description]

This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and
CL 485316 incorporated.

CL 481061, by doujiang24 , speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 482975 is a followup fix to a C declaration in testprogcgo.
CL 485315 is a followup fix for x_cgo_getstackbound on Illumos.
CL 485316 is a followup cleanup for ppc64 assembly.

CL 479915 passed the G to _cgo_getstackbound for direct updates to
gp.stack.lo. A G can be reused on a new thread after the previous thread
exited. This could trigger the C TSAN race detector because it couldn't
see the synchronization in Go (lockextra) preventing the same G from
being used on multiple threads at the same time.

We work around this by passing the address of a stack variable to
_cgo_getstackbound rather than the G. The stack is generally unique per
thread, so TSAN won't see the same address from multiple threads. Even
if stacks are reused across threads by pthread, C TSAN should see the
synchonization in the stack allocator.

A regression test is added to misc/cgo/testsanitizer.

[Original CL 481061 description]

This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 , speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

[CL 492987 description]

On the first call into Go from a C thread, currently we set the g0
stack's high bound imprecisely based on the SP. With CL 485500, we
keep the M and don't recompute the stack bounds when it calls into
Go again. If the first call is made when the C thread uses some
deep stack, but a subsequent call is made with a shallower stack,
the SP may be above g0.stack.hi.

This is usually okay as we don't check usually stack.hi. One place
where we do check for stack.hi is in the signal handler, in
adjustSignalStack. In particular, C TSAN delivers signals on the
g0 stack (instead of the usual signal stack). If the SP is above
g0.stack.hi, we don't see it is on the g0 stack, and throws.

This CL makes it get an accurate stack upper bound with the
pthread API (on the platforms where it is available).

Also add some debug print for the "handler not on signal stack"
throw.

Fixes #51676.
Fixes #59294.
Fixes #59678.
Fixes #60007.

Change-Id: Ie51c8e81ade34ec81d69fd7bce1fe0039a470776
Reviewed-on: https://go-review.googlesource.com/c/go/+/495855
Run-TryBot: Cherry Mui
TryBot-Result: Gopher Robot
Reviewed-by: Michael Pratt

runtime: publish netpoll info after incrementing fdseq

2023-05-17T20:12:00+00:00

I think there is a theoretical possibility of a mistake before this CL.
pollCache.free would increment fdseq, but would not update atomicInfo.
The epoll code could compare to fdseq before the increment, but suspend
before calling setEventErr. The pollCache could get reallocated,
and pollOpen could clear eventErr. Then the setEventErr could continue
and set it again. Then pollOpen could call publishInfo.

Avoid this rather remote possibility by calling publishInfo after
incrementing fdseq. That ensures that delayed setEventErr will not
modify the eventErr flag.

Fixes #60133

Change-Id: I69e336535312544690821c9fd53f3023ff15b80c
Reviewed-on: https://go-review.googlesource.com/c/go/+/495297
Run-TryBot: Ian Lance Taylor 
TryBot-Result: Gopher Robot 
Reviewed-by: Ian Lance Taylor 
Run-TryBot: Ian Lance Taylor 
Reviewed-by: Bryan Mills 
Auto-Submit: Ian Lance Taylor

runtime: replace sysBlockTraced with tracedSyscallEnter

2023-05-17T15:50:17+00:00

sysBlockTraced is a subtle and confusing flag.

Currently, it's only used in one place: a condition around whether to
traceGoSysExit when a goroutine is about to start running. That condition
looks like "gp.syscallsp != 0 && gp.trace.sysBlockTraced".

In every case but one, "gp.syscallsp != 0" is equivalent to
"gp.trace.sysBlockTraced."

That one case is where a goroutine is running without a P and racing
with trace start (world is stopped). It switches itself back to
_Grunnable from _Gsyscall before the trace start goroutine notices, such
that the trace start goroutine fails to emit a EvGoInSyscall event for
it (EvGoInSyscall or EvGoSysBlock must precede any EvGoSysExit event).
sysBlockTraced is set unconditionally on every syscall entry and the
trace start goroutine clears it if there was no EvGoInSyscall event
emitted (i.e. did not observe _Gsyscall on the goroutine). That way when
the goroutine-without-a-P wakes up and gets scheduled, it only emits
EvGoSysExit if the flag is set, i.e. trace start didn't _clear_ the
flag.

What makes this confusing is the fact that the flag is set
unconditionally and the code relies on it being *cleared*. Really, all
it's trying to communicate is whether the tracer is aware of a
goroutine's syscall at the point where a goroutine that lost its P
during a syscall is trying to run again.

Therefore, we can replace this flag with a less subtle one:
tracedSyscallEnter. It is set when GoSysCall is traced, indicating on
the goroutine that the tracer is aware of the syscall. Later, if
traceGoSysExit is called, the tracer knows its safe to emit an event
because the tracer is aware of the syscall.

This flag is then also set at trace start, when it emits EvGoInSyscall,
which again, lets the goroutine know the tracer is aware of its syscall.

The flag is cleared by GoSysExit to indicate that the tracer is no
longer aware of any syscalls on the goroutine. It's also cleared by
trace start. This is necessary because a syscall may have been started
while a trace was stopping. If the GoSysExit isn't emitted (because it
races with the trace end STW) then the flag will be left set at the
start of the next trace period, which will result in an erroneous
GoSysExit. Instead, the flag is cleared in the same way sysBlockTraced
is today: if the tracer doesn't notice the goroutine is in a syscall, it
makes that explicit to the goroutine.

A more direct flag to use here would be one that explicitly indicates
whether EvGoInSyscall or EvGoSysBlock specifically were already emitted
for a goroutine. The reason why we don't just do this is because setting
the flag when EvGoSysBlock is emitted would be racy: EvGoSysBlock is
emitted by whatever thread is stealing the P out from under the
syscalling goroutine, so it would need to synchronize with the goroutine
its emitting the event for.

The end result of all this is that the new flag can be managed entirely
within trace.go, hiding another implementation detail about the tracer.

Tested with `stress ./trace.test -test.run="TestTraceStressStartStop"`
which was occasionally failing before the CL in which sysBlockTraced was
added (CL 9132). I also confirmed also that this test is still sensitive
to `EvGoSysExit` by removing the one use of sysBlockTraced. The result
is about a 5% error rate. If there is something very subtly wrong about
how this CL emits `EvGoSysExit`, I would expect to see it as a test
failure. Instead:

    53m55s: 200434 runs so far, 0 failures

Change-Id: If1d24ee6b6926eec7e90cdb66039a5abac819d9b
Reviewed-on: https://go-review.googlesource.com/c/go/+/494715
TryBot-Result: Gopher Robot 
Reviewed-by: Michael Pratt 
Run-TryBot: Michael Knyszek 
Auto-Submit: Michael Knyszek

runtime: hide sysExitTicks a little better

2023-05-17T14:53:01+00:00

Just another step to hiding implementation details.

Change-Id: I71b7cc522d18c23f03a9bf32e428279e62b39a89
Reviewed-on: https://go-review.googlesource.com/c/go/+/494192
Run-TryBot: Michael Knyszek 
Auto-Submit: Michael Knyszek 
TryBot-Result: Gopher Robot 
Reviewed-by: Michael Pratt

runtime: capture per-p trace state in a type

2023-05-17T14:46:57+00:00

More tightening up of the tracer's interface.

Change-Id: I992141c7f30e5c2d5d77d1fcd6817d35bc6e5f6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/494191
Auto-Submit: Michael Knyszek 
Reviewed-by: Michael Pratt 
Run-TryBot: Michael Knyszek 
TryBot-Result: Gopher Robot

runtime: capture per-m trace state in a type

2023-05-17T14:45:42+00:00

More tightening up of the tracer's interface.

While we're here, clarify why waittraceskip isn't included by explaining
what the wait* fields in the M are really for.

Change-Id: I0e7b4cac79fb77a7a0b3ca6b6cc267668e3610bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/494190
Auto-Submit: Michael Knyszek 
Run-TryBot: Michael Knyszek 
Reviewed-by: Michael Pratt 
TryBot-Result: Gopher Robot

runtime: capture per-g trace state in a type

2023-05-17T14:45:40+00:00

More tightening up of the tracer's interface.

This increases the size of each G very slightly, which isn't great, but
we stay within the same size class, so actually memory use will be
unchanged.

Change-Id: I7d1f5798edcf437c212beb1e1a2619eab833aafb
Reviewed-on: https://go-review.googlesource.com/c/go/+/494188
TryBot-Result: Gopher Robot 
Auto-Submit: Michael Knyszek 
Reviewed-by: Michael Pratt 
Run-TryBot: Michael Knyszek

hash/maphash: weaken avalanche test a bit

2023-05-17T14:45:20+00:00

Give the test a bit more wiggle room.

Previously the allowed range was about 46.5% to 53.5%. Now it is about 43% TO 57%.

Fixes #60170

Change-Id: Ieda471e0986c52edb9f6d31beb8e41917876d6c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/495415
TryBot-Result: Gopher Robot 
Reviewed-by: Heschi Kreinick 
Reviewed-by: Cuong Manh Le 
Reviewed-by: Keith Randall 
Auto-Submit: Keith Randall 
Run-TryBot: Keith Randall

runtime: factor our oneNewExtraM trace code

2023-05-17T14:43:25+00:00

In the interest of further cleaning up the trace.go API, move the trace
logic in oneNewExtraM into its own function.

Change-Id: I5cf478cb8cd0d301ee3b068347ed48ce768b8882
Reviewed-on: https://go-review.googlesource.com/c/go/+/494186
Reviewed-by: Michael Pratt 
Run-TryBot: Michael Knyszek 
Auto-Submit: Michael Knyszek 
TryBot-Result: Gopher Robot

runtime: consistently define fcntl

2023-05-17T04:58:07+00:00

Clean up and consolidate on a single consistent definition of fcntl,
which takes three int32 arguments and returns either a positive result
or a negative errno value.

Change-Id: Id9505492712db4b0aab469c6bd15e4fce3c9ff6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/495075
Run-TryBot: Ian Lance Taylor 
Reviewed-by: Ian Lance Taylor 
TryBot-Result: Gopher Robot 
Auto-Submit: Ian Lance Taylor 
Run-TryBot: Ian Lance Taylor 
Reviewed-by: Tobias Klauser 
Reviewed-by: Michael Pratt