delta/go-git.git/src/runtime/cgo, branch dev.link github.com: golang/go runtime,cmd/cgo: simplify C -> Go call path 2020-10-26T14:50:32+00:00 Austin Clements austin@google.com 2020-10-01T21:22:38+00:00 30c18878730434027dbefd343aad74963a1fdc48 This redesigns the way calls work from C to exported Go functions. It removes several steps from the call path, makes cmd/cgo no longer sensitive to the Go calling convention, and eliminates the use of reflectcall from cgo. In order to avoid generating a large amount of FFI glue between the C and Go ABIs, the cgo tool has long depended on generating a C function that marshals the arguments into a struct, and then the actual ABI switch happens in functions with fixed signatures that simply take a pointer to this struct. In a way, this CL simply pushes this idea further. Currently, the cgo tool generates this argument struct in the exact layout of the Go stack frame and depends on reflectcall to unpack it into the appropriate Go call (even though it's actually reflectcall'ing a function generated by cgo). In this CL, we decouple this struct from the Go stack layout. Instead, cgo generates a Go function that takes the struct, unpacks it, and calls the exported function. Since this generated function has a generic signature (like the rest of the call path), we don't need reflectcall and can instead depend on the Go compiler itself to implement the call to the exported Go function. One complication is that syscall.NewCallback on Windows, which converts a Go function into a C function pointer, depends on cgocallback's current dynamic calling approach since the signatures of the callbacks aren't known statically. For this specific case, we continue to depend on reflectcall. Really, the current approach makes some overly simplistic assumptions about translating the C ABI to the Go ABI. Now we're at least in a much better position to do a proper ABI translation. For comparison, the current cgo call path looks like: GoF (generated C function) -> crosscall2 (in cgo/asm_*.s) -> _cgoexp_GoF (generated Go function) -> cgocallback (in asm_*.s) -> cgocallback_gofunc (in asm_*.s) -> cgocallbackg (in cgocall.go) -> cgocallbackg1 (in cgocall.go) -> reflectcall (in asm_*.s) -> _cgoexpwrap_GoF (generated Go function) -> p.GoF Now the call path looks like: GoF (generated C function) -> crosscall2 (in cgo/asm_*.s) -> cgocallback (in asm_*.s) -> cgocallbackg (in cgocall.go) -> cgocallbackg1 (in cgocall.go) -> _cgoexp_GoF (generated Go function) -> p.GoF Notably: 1. We combine _cgoexp_GoF and _cgoexpwrap_GoF and move the combined operation to the end of the sequence. This combined function also handles reflectcall's previous role. 2. We combined cgocallback and cgocallback_gofunc since the only purpose of having both was to convert a raw PC into a Go function value. We instead construct the Go function value in cgocallbackg1. 3. cgocallbackg1 no longer reaches backwards through the stack to get the arguments to cgocallback_gofunc. Instead, we just pass the arguments down. 4. Currently, we need an explicit msanwrite to mark the results struct as written because reflectcall doesn't do this. Now, the results are written by regular Go assignments, so the Go compiler generates the necessary MSAN annotations. This also means we no longer need to track the size of the arguments frame. Updates #40724, since now we don't need to teach cgo about the register ABI or change how it uses reflectcall. Change-Id: I7840489a2597962aeb670e0c1798a16a7359c94f Reviewed-on: https://go-review.googlesource.com/c/go/+/258938 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> 
This redesigns the way calls work from C to exported Go functions. It
removes several steps from the call path, makes cmd/cgo no longer
sensitive to the Go calling convention, and eliminates the use of
reflectcall from cgo.

In order to avoid generating a large amount of FFI glue between the C
and Go ABIs, the cgo tool has long depended on generating a C function
that marshals the arguments into a struct, and then the actual ABI
switch happens in functions with fixed signatures that simply take a
pointer to this struct. In a way, this CL simply pushes this idea
further.

Currently, the cgo tool generates this argument struct in the exact
layout of the Go stack frame and depends on reflectcall to unpack it
into the appropriate Go call (even though it's actually
reflectcall'ing a function generated by cgo).

In this CL, we decouple this struct from the Go stack layout. Instead,
cgo generates a Go function that takes the struct, unpacks it, and
calls the exported function. Since this generated function has a
generic signature (like the rest of the call path), we don't need
reflectcall and can instead depend on the Go compiler itself to
implement the call to the exported Go function.

One complication is that syscall.NewCallback on Windows, which
converts a Go function into a C function pointer, depends on
cgocallback's current dynamic calling approach since the signatures of
the callbacks aren't known statically. For this specific case, we
continue to depend on reflectcall. Really, the current approach makes
some overly simplistic assumptions about translating the C ABI to the
Go ABI. Now we're at least in a much better position to do a proper
ABI translation.

For comparison, the current cgo call path looks like:

    GoF (generated C function) ->
    crosscall2 (in cgo/asm_*.s) ->
    _cgoexp_GoF (generated Go function) ->
    cgocallback (in asm_*.s) ->
    cgocallback_gofunc (in asm_*.s) ->
    cgocallbackg (in cgocall.go) ->
    cgocallbackg1 (in cgocall.go) ->
    reflectcall (in asm_*.s) ->
    _cgoexpwrap_GoF (generated Go function) ->
    p.GoF

Now the call path looks like:

    GoF (generated C function) ->
    crosscall2 (in cgo/asm_*.s) ->
    cgocallback (in asm_*.s) ->
    cgocallbackg (in cgocall.go) ->
    cgocallbackg1 (in cgocall.go) ->
    _cgoexp_GoF (generated Go function) ->
    p.GoF

Notably:

1. We combine _cgoexp_GoF and _cgoexpwrap_GoF and move the combined
operation to the end of the sequence. This combined function also
handles reflectcall's previous role.

2. We combined cgocallback and cgocallback_gofunc since the only
purpose of having both was to convert a raw PC into a Go function
value. We instead construct the Go function value in cgocallbackg1.

3. cgocallbackg1 no longer reaches backwards through the stack to get
the arguments to cgocallback_gofunc. Instead, we just pass the
arguments down.

4. Currently, we need an explicit msanwrite to mark the results struct
as written because reflectcall doesn't do this. Now, the results are
written by regular Go assignments, so the Go compiler generates the
necessary MSAN annotations. This also means we no longer need to track
the size of the arguments frame.

Updates #40724, since now we don't need to teach cgo about the
register ABI or change how it uses reflectcall.

Change-Id: I7840489a2597962aeb670e0c1798a16a7359c94f
Reviewed-on: https://go-review.googlesource.com/c/go/+/258938
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
syscall: support POSIX semantics for Linux syscalls 2020-10-23T20:53:14+00:00 Andrew G. Morgan agm@google.com 2019-12-10T05:50:16+00:00 d1b1145cace8b968307f9311ff611e4bb810710c This change adds two new methods for invoking system calls under Linux: syscall.AllThreadsSyscall() and syscall.AllThreadsSyscall6(). These system call wrappers ensure that all OSThreads mirror a common system call. The wrappers serialize execution of the runtime to ensure no race conditions where any Go code observes a non-atomic OS state change. As such, the syscalls have higher runtime overhead than regular system calls, and only need to be used where such thread (or 'm' in the parlance of the runtime sources) consistency is required. The new support is used to enable these functions under Linux: syscall.Setegid(), syscall.Seteuid(), syscall.Setgroups(), syscall.Setgid(), syscall.Setregid(), syscall.Setreuid(), syscall.Setresgid(), syscall.Setresuid() and syscall.Setuid(). They work identically to their glibc counterparts. Extensive discussion of the background issue addressed in this patch can be found here: https://github.com/golang/go/issues/1435 In the case where cgo is used, the C runtime can launch pthreads that are not managed by the Go runtime. As such, the added syscall.AllThreadsSyscall*() return ENOTSUP when cgo is enabled. However, for the 9 syscall.Set*() functions listed above, when cgo is active, these functions redirect to invoke their C.set*() equivalents in glibc, which wraps the raw system calls with a nptl:setxid fixup mechanism. This achieves POSIX semantics for these functions in the combined Go and C runtime. As a side note, the glibc/nptl:setxid support (2019-11-30) does not extend to all security related system calls under Linux so using native Go (CGO_ENABLED=0) and these AllThreadsSyscall*()s, where needed, will yield more well defined/consistent behavior over all threads of a Go program. That is, using the syscall.AllThreadsSyscall*() wrappers for things like setting state through SYS_PRCTL and SYS_CAPSET etc. Fixes #1435 Change-Id: Ib1a3e16b9180f64223196a32fc0f9dce14d9105c Reviewed-on: https://go-review.googlesource.com/c/go/+/210639 Trust: Emmanuel Odeke <emm.odeke@gmail.com> Trust: Ian Lance Taylor <iant@golang.org> Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Austin Clements <austin@google.com> 
This change adds two new methods for invoking system calls
under Linux: syscall.AllThreadsSyscall() and
syscall.AllThreadsSyscall6().

These system call wrappers ensure that all OSThreads mirror
a common system call. The wrappers serialize execution of the
runtime to ensure no race conditions where any Go code observes
a non-atomic OS state change. As such, the syscalls have
higher runtime overhead than regular system calls, and only
need to be used where such thread (or 'm' in the parlance
of the runtime sources) consistency is required.

The new support is used to enable these functions under Linux:

  syscall.Setegid(), syscall.Seteuid(), syscall.Setgroups(),
  syscall.Setgid(), syscall.Setregid(), syscall.Setreuid(),
  syscall.Setresgid(), syscall.Setresuid() and syscall.Setuid().

They work identically to their glibc counterparts.

Extensive discussion of the background issue addressed in this
patch can be found here:

   https://github.com/golang/go/issues/1435

In the case where cgo is used, the C runtime can launch pthreads that
are not managed by the Go runtime. As such, the added
syscall.AllThreadsSyscall*() return ENOTSUP when cgo is enabled.
However, for the 9 syscall.Set*() functions listed above, when cgo is
active, these functions redirect to invoke their C.set*() equivalents
in glibc, which wraps the raw system calls with a nptl:setxid fixup
mechanism. This achieves POSIX semantics for these functions in the
combined Go and C runtime.

As a side note, the glibc/nptl:setxid support (2019-11-30) does not
extend to all security related system calls under Linux so using
native Go (CGO_ENABLED=0) and these AllThreadsSyscall*()s, where
needed, will yield more well defined/consistent behavior over all
threads of a Go program. That is, using the
syscall.AllThreadsSyscall*() wrappers for things like setting state
through SYS_PRCTL and SYS_CAPSET etc.

Fixes #1435

Change-Id: Ib1a3e16b9180f64223196a32fc0f9dce14d9105c
Reviewed-on: https://go-review.googlesource.com/c/go/+/210639
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
runtime/cgo: fix build tag placement vet warning 2020-10-17T00:01:16+00:00 Cherry Zhang cherryyz@google.com 2020-10-16T23:11:42+00:00 689a7a13780dc7a5138215aa4d369bdcf789fee8 Change-Id: Ie6583b46213caae897fc2189d4973c88759f5f4b Reviewed-on: https://go-review.googlesource.com/c/go/+/263258 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> 
Change-Id: Ie6583b46213caae897fc2189d4973c88759f5f4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/263258
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime/cgo: build iOS-specific code only on iOS 2020-10-17T00:01:00+00:00 Cherry Zhang cherryyz@google.com 2020-10-14T23:26:20+00:00 f1e3c8f14232cde8da8666ad68df493563287634 Don't build them on macOS/ARM64. Updates #38485. Change-Id: I9fbea838fdce52db22742487926879761dea0d6a Reviewed-on: https://go-review.googlesource.com/c/go/+/262559 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> 
Don't build them on macOS/ARM64.

Updates #38485.

Change-Id: I9fbea838fdce52db22742487926879761dea0d6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/262559
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
internal/poll, net, syscall: use accept4 on illumos 2020-10-16T19:53:34+00:00 Tobias Klauser tklauser@distanz.ch 2020-10-16T17:35:09+00:00 9cec50f50c29f5ef7264bf06ee7ac0991b4b36d6 Illumos supports the accept4 syscall, use it in internal/poll.accept like on other platforms. Add Accept4 to package syscall despite the package being frozen. The other option would have been to add this to internal/syscall/unix, but adding it to syscall avoids duplicating a lot of code in internal/poll and net/internal/socktest. Also, all other platforms supporting the accept4 syscall already export Accept4. Follow CL 97196, CL 40895 and CL 94295 Change-Id: I13b32f0163a683840c02b16722730d9dfdb98f56 Reviewed-on: https://go-review.googlesource.com/c/go/+/256101 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> 
Illumos supports the accept4 syscall, use it in internal/poll.accept
like on other platforms.

Add Accept4 to package syscall despite the package being frozen. The
other option would have been to add this to internal/syscall/unix, but
adding it to syscall avoids duplicating a lot of code in internal/poll
and net/internal/socktest. Also, all other platforms supporting the
accept4 syscall already export Accept4.

Follow CL 97196, CL 40895 and CL 94295

Change-Id: I13b32f0163a683840c02b16722730d9dfdb98f56
Reviewed-on: https://go-review.googlesource.com/c/go/+/256101
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime/cgo: only build xx_cgo_panicmem on iOS 2020-10-06T22:54:58+00:00 Cherry Zhang cherryyz@google.com 2020-10-03T20:18:43+00:00 3923460dda205721d9bee2714a7f0dd403082a90 On iOS, when running under lldb, we install xx_cgo_panicmem as EXC_BAD_ACCESS handler so we can get a proper Go panic for SIGSEGV. Only build it on iOS. Updates #38485. Change-Id: I801c477439e05920a4bb8fdf5eae6f4923ab8274 Reviewed-on: https://go-review.googlesource.com/c/go/+/259440 Trust: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> 
On iOS, when running under lldb, we install xx_cgo_panicmem as
EXC_BAD_ACCESS handler so we can get a proper Go panic for
SIGSEGV. Only build it on iOS.

Updates #38485.

Change-Id: I801c477439e05920a4bb8fdf5eae6f4923ab8274
Reviewed-on: https://go-review.googlesource.com/c/go/+/259440
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
misc/ios: add support for running programs on the iOS simulator 2020-10-03T17:02:58+00:00 Elias Naur mail@eliasnaur.com 2020-09-16T13:23:58+00:00 869c02ce1f635960bfc2f06bb52e2b4e17eaa199 Update the README to mention the emulator. Remove reference to gomobile while here; there are multiple ways to develop for iOS today, including using the c-archive buildmode directly. Updates #38485 Change-Id: Iccef75e646ea8e1b9bc3fc37419cc2d6bf3dfdf4 Reviewed-on: https://go-review.googlesource.com/c/go/+/255257 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Elias Naur <mail@eliasnaur.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> 
Update the README to mention the emulator. Remove reference to gomobile
while here; there are multiple ways to develop for iOS today, including
using the c-archive buildmode directly.

Updates #38485

Change-Id: Iccef75e646ea8e1b9bc3fc37419cc2d6bf3dfdf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/255257
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
runtime: netbsd-arm64 fixes 2020-09-28T06:01:37+00:00 Benny Siegert bsiegert@gmail.com 2020-09-26T19:40:17+00:00 8ab020adb27089fa207d015f2f69600ef3d1d307 Add missing declaration of crosscall1. Fix stack alignment for pipe2 return value. Work around kernel clobbering of r28 on aarch64 by reloading from ucontext. https://nxr.netbsd.org/xref/src/sys/arch/aarch64/aarch64/sig_machdep.c#104 Update #30824 Change-Id: I7f9472939f4c02953f8c207308610118f5d3c54c Reviewed-on: https://go-review.googlesource.com/c/go/+/257645 Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Benny Siegert <bsiegert@gmail.com> 
Add missing declaration of crosscall1.

Fix stack alignment for pipe2 return value.

Work around kernel clobbering of r28 on aarch64 by reloading from ucontext.
https://nxr.netbsd.org/xref/src/sys/arch/aarch64/aarch64/sig_machdep.c#104

Update #30824

Change-Id: I7f9472939f4c02953f8c207308610118f5d3c54c
Reviewed-on: https://go-review.googlesource.com/c/go/+/257645
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Benny Siegert <bsiegert@gmail.com>
runtime: fix dead link in gcc_androd.c file 2020-05-09T23:17:17+00:00 Alberto Donizetti alb.donizetti@gmail.com 2020-05-09T15:10:32+00:00 c9442dc8506e52c2846e436adb6be3831c37beb6 Old url 404s because the file no longer exists on master; change it to point to the android 10 release branch. Change-Id: If0f8b645f2c746f9fc8bbd68f4d1fe41868493ba Reviewed-on: https://go-review.googlesource.com/c/go/+/232809 Reviewed-by: Ian Lance Taylor <iant@golang.org> 
Old url 404s because the file no longer exists on master; change it to
point to the android 10 release branch.

Change-Id: If0f8b645f2c746f9fc8bbd68f4d1fe41868493ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/232809
Reviewed-by: Ian Lance Taylor <iant@golang.org>
all: remove darwin/386 build-tags and files 2020-04-08T18:37:36+00:00 Austin Clements austin@google.com 2020-04-06T17:43:17+00:00 8b4cbcc32098733832db0ba0cd8a8569315861c2 This removes all files that are only used on darwin/386 and cleans up build tags in files that are still used on other platforms. Updates #37610. Change-Id: If246642476c12d15f59a474e2b91a29c0c02fe75 Reviewed-on: https://go-review.googlesource.com/c/go/+/227581 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> 
This removes all files that are only used on darwin/386 and cleans up
build tags in files that are still used on other platforms.

Updates #37610.

Change-Id: If246642476c12d15f59a474e2b91a29c0c02fe75
Reviewed-on: https://go-review.googlesource.com/c/go/+/227581
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>