delta/go-git.git/src/runtime, branch dev.unified

runtime: clean up panic and deadlock lock ranks

2022-08-04T15:31:49+00:00

I'm not entirely sure why these locks are currently ranked "deadlock <
panic" since we drop panic before acquiring deadlock, and we actually
want deadlock to be below panic because panic is implicitly below
everything else and we want deadlock to be, too. My best guess is that
we had this edge because we intentionally acquire deadlock twice to
deadlock, and that causes the lock rank checking to panic on the
second acquire.

Fix this in a more sensible way by capturing that deadlock can be
acquired in a self-cycle and flipping the rank to "panic < deadlock"
to express that deadlock needs to be under all other locks, just like
panic.

For #53789.

Change-Id: I8809e5d102ce473bd3ace0ba07bf2200ef60263f
Reviewed-on: https://go-review.googlesource.com/c/go/+/418719
TryBot-Result: Gopher Robot 
Reviewed-by: Michael Pratt 
Run-TryBot: Austin Clements

runtime: make the lock rank DAG make more sense

2022-08-04T15:31:48+00:00

This groups, comments, and generally reorganizes the lock rank graph
description by subsystem. It also introduces several pseudo-nodes that
more cleanly describe the inherent layering of lock ranks by
subsystem.

I believe this doesn't actually change the graph, but haven't verified
this.

For #53789.

Change-Id: I72f332f5a23b8217c7dc1b21411631ad48cee4b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/418718
TryBot-Result: Gopher Robot 
Run-TryBot: Austin Clements 
Reviewed-by: Michael Pratt

runtime: add mayAcquire annotation for finlock

2022-08-04T15:31:46+00:00

We're missing lock edges to finlock that happen only rarely. Anything
that calls mallocgc can potentially trigger sweeping, which can
potentially queue a finalizer, which acquires finlock. While this can
happen on any malloc, it happens relatively rarely, so we simply
haven't seen some of the lock edges that could happen.

Add a mayAcquire annotation to mallocgc to capture the possibility of
acquiring finlock.

With this change, we add "fin" to the set of "malloc" locks. Several
of these edges were already there, but not quite all of them.

This was found by inspecting the rank graph for things that didn't
make sense.

For #53789.

Change-Id: Idc10ce6f250596b0c07ba07ac93f2198fb38c22b
Reviewed-on: https://go-review.googlesource.com/c/go/+/418717
Run-TryBot: Austin Clements 
Reviewed-by: Michael Pratt 
TryBot-Result: Gopher Robot

runtime: add missing trace lock edges

2022-08-04T15:31:45+00:00

We're missing lock edges to trace.lock that happen only rarely. Any
trace event can potentially fill up a trace buffer and acquire
trace.lock in order to flush the buffer, but this happens relatively
rarely, so we simply haven't seen some of these lock edges that could
happen.

With this change, we promote "fin, notifyList < traceStackTab" to
"fin, notifyList < trace" and now everything that emits trace events
with a P enters the tracer lock ranks via "trace", rather than some
things entering at "trace" and others at "traceStackTab".

This was found by inspecting the rank graph for things that didn't
make sense.

Ideally we would add a mayAcquire annotation that any trace event can
potentially acquire trace.lock, but there are actually cases that
violate this ranking right now. This is #53979. The chance of a lock
cycle is extremely low given the number of conditions that have to
happen simultaneously.

For #53789.

Change-Id: Ic65947d27dee88d2daf639b21b2c9d37552f0ac0
Reviewed-on: https://go-review.googlesource.com/c/go/+/418716
Reviewed-by: Michael Pratt 
Run-TryBot: Austin Clements 
TryBot-Result: Gopher Robot

runtime: generate the lock ranking from a DAG description

2022-08-04T15:31:44+00:00

Currently, the runtime lock rank graph is maintained manually in a
large set of arrays that give the partial order and a manual
topological sort of this partial order. Any changes to the rank graph
are difficult to reason about and hard to review, as well as likely to
cause merge conflicts. Furthermore, because the partial order is
manually maintained, it's not actually transitively closed (though
it's close), meaning there are many cases where rank a can be acquired
before b and b before c, but a cannot be acquired before c. While this
isn't technically wrong, it's very strange in the context of lock
ordering.

Replace all of this with a much more compact, readable, and
maintainable description of the rank graph written in the internal/dag
graph language. We statically generate the runtime structures from
this description, which has the advantage that the parser doesn't have
to run during runtime initialization and the structures can live in
static data where they can be accessed from any point during runtime
init.

The current description was automatically generated from the existing
partial order, combined with a transitive reduction. This ensures it's
correct, but it could use some manual messaging to call out the
logical layers and add some structure.

We do lose the ad hoc string names of the lock ranks in this
translation, which could mostly be derived from the rank constant
names, but not always. I may bring those back but in a more uniform
way.

We no longer need the tests in lockrank_test.go because they were
checking that we manually maintained the structures correctly.

Fixes #53789.

Change-Id: I54451d561b22e61150aff7e9b8602ba9737e1b9b
Reviewed-on: https://go-review.googlesource.com/c/go/+/418715
Run-TryBot: Austin Clements 
Reviewed-by: Michael Pratt 
TryBot-Result: Gopher Robot

runtime: delete unused lock ranks

2022-08-04T15:31:43+00:00

For #53789.

Change-Id: Ic7379afcfdcc47b541bac9b44b5bc6b43604fc0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/418714
Reviewed-by: Michael Pratt 
TryBot-Result: Gopher Robot 
Run-TryBot: Austin Clements

runtime: support riscv64 SV57 mode

2022-08-04T05:33:40+00:00

riscv64 has SV57 mode when user-space VA is 56 bits.
Linux kernel recently got support for this mode and Go binaries started crashing as:

runtime: lfstack.push invalid packing: node=0xffffff5908a940 cnt=0x1
packed=0xffff5908a9400001 -> node=0xffff5908a940

Adjust lfstack code to use only 8 top bits of pointers on riscv64.

For context see:
https://groups.google.com/g/syzkaller-bugs/c/lU0GQTZoNQQ/m/O_c3vmE3AAAJ

Update #54104

Change-Id: Ib5d3d6a79c0c6eddf11618d73fcc8bc1832a9c25
Reviewed-on: https://go-review.googlesource.com/c/go/+/409055
Reviewed-by: Joel Sing 
Reviewed-by: Meng Zhuo 
Reviewed-by: Michael Knyszek 
Reviewed-by: Cherry Mui

runtime: trivial replacements of g in remaining files

2022-08-02T18:52:33+00:00

Rename g variables to gp for consistency.

Change-Id: I09ecdc7e8439637bc0e32f9c5f96f515e6436362
Reviewed-on: https://go-review.googlesource.com/c/go/+/418591
Reviewed-by: Austin Clements 
Run-TryBot: Michael Pratt

runtime: trivial replacements of g in signal_unix.go

2022-08-02T18:52:18+00:00

Change-Id: I0d8d50c8b7ae6cd01594091392dc726cf149e34a
Reviewed-on: https://go-review.googlesource.com/c/go/+/418590
TryBot-Result: Gopher Robot 
Reviewed-by: Austin Clements 
Run-TryBot: Michael Pratt

runtime: tricky replacements of g in traceback.go

2022-08-02T18:52:13+00:00

Change-Id: I80e847451b697de049a01df051a73b5976b39d4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/418589
Reviewed-by: Austin Clements 
Run-TryBot: Michael Pratt 
TryBot-Result: Gopher Robot