delta/glibc.git/nptl_db, branch azanella/generic-strings

nptl_db: Remove stale `match_pid' parameter from `iterate_thread_list'

2018-03-01T16:10:05+00:00

Complement commit c579f48edba8 ("Remove cached PID/TID in clone") and
remove the `match_pid' parameter not used by `iterate_thread_list' any
longer.  Update call sites accordingly.

	* nptl_db/td_ta_thr_iter.c (iterate_thread_list): Remove
	`match_pid' parameter.
	(td_ta_thr_iter): Update accordingly.

Update copyright dates with scripts/update-copyrights.

2018-01-01T00:32:25+00:00

	* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.

Move all old ChangeLogs to a top-level ChangeLog.old directory.

2017-09-01T13:31:43+00:00

Narrowing the visibility of libc-internal.h even further.

2017-03-02T01:33:46+00:00

posix/wordexp-test.c used libc-internal.h for PTR_ALIGN_DOWN; similar
to what was done with libc-diag.h, I have split the definitions of
cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and PTR_ALIGN_DOWN
to a new header, libc-pointer-arith.h.

It then occurred to me that the remaining declarations in libc-internal.h
are mostly to do with early initialization, and probably most of the
files including it, even in the core code, don't need it anymore.  Indeed,
only 19 files actually need what remains of libc-internal.h.  23 others
need libc-diag.h instead, and 12 need libc-pointer-arith.h instead.
No file needs more than one of them, and 16 don't need any of them!

So, with this patch, libc-internal.h stops including libc-diag.h as
well as losing the pointer arithmetic macros, and all including files
are adjusted.

        * include/libc-pointer-arith.h: New file.  Define
	cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and
        PTR_ALIGN_DOWN here.
        * include/libc-internal.h: Definitions of above macros
	moved from here.  Don't include libc-diag.h anymore either.
	* posix/wordexp-test.c: Include stdint.h and libc-pointer-arith.h.
        Don't include libc-internal.h.

	* debug/pcprofile.c, elf/dl-tunables.c, elf/soinit.c, io/openat.c
	* io/openat64.c, misc/ptrace.c, nptl/pthread_clock_gettime.c
	* nptl/pthread_clock_settime.c, nptl/pthread_cond_common.c
	* string/strcoll_l.c, sysdeps/nacl/brk.c
	* sysdeps/unix/clock_settime.c
	* sysdeps/unix/sysv/linux/i386/get_clockfreq.c
	* sysdeps/unix/sysv/linux/ia64/get_clockfreq.c
	* sysdeps/unix/sysv/linux/powerpc/get_clockfreq.c
	* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c:
	Don't include libc-internal.h.

	* elf/get-dynamic-info.h, iconv/loop.c
	* iconvdata/iso-2022-cn-ext.c, locale/weight.h, locale/weightwc.h
	* misc/reboot.c, nis/nis_table.c, nptl_db/thread_dbP.h
	* nscd/connections.c, resolv/res_send.c, soft-fp/fmadf4.c
	* soft-fp/fmasf4.c, soft-fp/fmatf4.c, stdio-common/vfscanf.c
	* sysdeps/ieee754/dbl-64/e_lgamma_r.c
	* sysdeps/ieee754/dbl-64/k_rem_pio2.c
	* sysdeps/ieee754/flt-32/e_lgammaf_r.c
	* sysdeps/ieee754/flt-32/k_rem_pio2f.c
	* sysdeps/ieee754/ldbl-128/k_tanl.c
	* sysdeps/ieee754/ldbl-128ibm/k_tanl.c
	* sysdeps/ieee754/ldbl-96/e_lgammal_r.c
	* sysdeps/ieee754/ldbl-96/k_tanl.c, sysdeps/nptl/futex-internal.h:
	Include libc-diag.h instead of libc-internal.h.

        * elf/dl-load.c, elf/dl-reloc.c, locale/programs/locarchive.c
        * nptl/nptl-init.c, string/strcspn.c, string/strspn.c
	* malloc/malloc.c, sysdeps/i386/nptl/tls.h
	* sysdeps/nacl/dl-map-segments.h, sysdeps/x86_64/atomic-machine.h
	* sysdeps/unix/sysv/linux/spawni.c
        * sysdeps/x86_64/nptl/tls.h:
        Include libc-pointer-arith.h instead of libc-internal.h.

	* elf/get-dynamic-info.h, sysdeps/nacl/dl-map-segments.h
	* sysdeps/x86_64/atomic-machine.h:
        Add multiple include guard.

Update copyright dates with scripts/update-copyrights.

2017-01-01T00:14:16+00:00

Remove cached PID/TID in clone

2016-11-24T21:38:51+00:00

This patch remove the PID cache and usage in current GLIBC code.  Current
usage is mainly used a performance optimization to avoid the syscall,
however it adds some issues:

  - The exposed clone syscall will try to set pid/tid to make the new
    thread somewhat compatible with current GLIBC assumptions.  This cause
    a set of issue with new workloads and usecases (such as BZ#17214 and
    [1]) as well for new internal usage of clone to optimize other algorithms
    (such as clone plus CLONE_VM for posix_spawn, BZ#19957).

  - The caching complexity also added some bugs in the past [2] [3] and
    requires more effort of each port to handle such requirements (for
    both clone and vfork implementation).

  - Caching performance gain in mainly on getpid and some specific
    code paths.  The getpid performance leverage is questionable [4],
    either by the idea of getpid being a hotspot as for the getpid
    implementation itself (if it is indeed a justifiable hotspot a
    vDSO symbol could let to a much more simpler solution).

    Other usage is mainly for non usual code paths, such as pthread
    cancellation signal and handling.

For thread creation (on stack allocation) the code simplification in fact
adds some performance gain due the no need of transverse the stack cache
and invalidate each element pid.

Other thread usages will require a direct getpid syscall, such as
cancellation/setxid signal, thread cancellation, thread fail path (at
create_thread), and thread signal (pthread_kill and pthread_sigqueue).
However these are hardly usual hotspots and I think adding a syscall is
justifiable.

It also simplifies both the clone and vfork arch-specific implementation.
And by review each fork implementation there are some discrepancies that
this patch also solves:

  - microblaze clone/vfork does not set/reset the pid/tid field
  - hppa uses the default vfork implementation that fallback to fork.
    Since vfork is deprecated I do not think we should bother with it.

The patch also removes the TID caching in clone. My understanding for
such semantic is try provide some pthread usage after a user program
issue clone directly (as done by thread creation with CLONE_PARENT_SETTID
and pthread tid member).  However, as stated before in multiple discussions
threads, GLIBC provides clone syscalls without further supporting all this
semantics.

I ran a full make check on x86_64, x32, i686, armhf, aarch64, and powerpc64le.
For sparc32, sparc64, and mips I ran the basic fork and vfork tests from
posix/ folder (on a qemu system).  So it would require further testing
on alpha, hppa, ia64, m68k, nios2, s390, sh, and tile (I excluded microblaze
because it is already implementing the patch semantic regarding clone/vfork).

[1] https://codereview.chromium.org/800183004/
[2] https://sourceware.org/ml/libc-alpha/2006-07/msg00123.html
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=15368
[4] http://yarchive.net/comp/linux/getpid_caching.html

	* sysdeps/nptl/fork.c (__libc_fork): Remove pid cache setting.
	* nptl/allocatestack.c (allocate_stack): Likewise.
	(__reclaim_stacks): Likewise.
	(setxid_signal_thread): Obtain pid through syscall.
	* nptl/nptl-init.c (sigcancel_handler): Likewise.
	(sighandle_setxid): Likewise.
	* nptl/pthread_cancel.c (pthread_cancel): Likewise.
	* sysdeps/unix/sysv/linux/pthread_kill.c (__pthread_kill): Likewise.
	* sysdeps/unix/sysv/linux/pthread_sigqueue.c (pthread_sigqueue):
	Likewise.
	* sysdeps/unix/sysv/linux/createthread.c (create_thread): Likewise.
	* sysdeps/unix/sysv/linux/getpid.c: Remove file.
	* nptl/descr.h (struct pthread): Change comment about pid value.
	* nptl/pthread_getattr_np.c (pthread_getattr_np): Remove thread
	pid assert.
	* sysdeps/unix/sysv/linux/pthread-pids.h (__pthread_initialize_pids):
	Do not set pid value.
	* nptl_db/td_ta_thr_iter.c (iterate_thread_list): Remove thread
	pid cache check.
	* nptl_db/td_thr_validate.c (td_thr_validate): Likewise.
	* sysdeps/aarch64/nptl/tcb-offsets.sym: Remove pid offset.
	* sysdeps/alpha/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/arm/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/hppa/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/i386/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/ia64/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/m68k/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/microblaze/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/mips/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/nios2/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/powerpc/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/s390/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/sh/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/sparc/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/tile/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/x86_64/nptl/tcb-offsets.sym: Likewise.
	* sysdeps/unix/sysv/linux/aarch64/clone.S: Remove pid and tid caching.
	* sysdeps/unix/sysv/linux/alpha/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/arm/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/hppa/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/i386/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/ia64/clone2.S: Likewise.
	* sysdeps/unix/sysv/linux/mips/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/nios2/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/sh/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/tile/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/aarch64/vfork.S: Remove pid set and reset.
	* sysdeps/unix/sysv/linux/alpha/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/arm/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/i386/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/ia64/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/m68k/clone.S: Likewise.
	* sysdeps/unix/sysv/linux/m68k/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/mips/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/nios2/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/sh/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/tile/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/vfork.S: Likewise.
	* sysdeps/unix/sysv/linux/tst-clone2.c (f): Remove direct pthread
	struct access.
	(clone_test): Remove function.
	(do_test): Rewrite to take in consideration pid is not cached anymore.

Bug 20729: Include libc-internal.h where required.

2016-10-31T20:46:57+00:00

The original fix for bug 20729 failed to include
libc-internal.h in the files that needed them and
this caused build failures on machines that don't
implicitly include this header. This commit fixes
that by following the consensus rule that a header,
if needed, should always be directly included.

Bug 20729: Fix building with -Os.

2016-10-30T03:50:56+00:00

This commit adds a new DIAG_IGNORE_Os_NEEDS_COMMENT which is only
enabled when compiling with -Os. This allows developers working on
-Os enabled builds to mark false-positive warnings without impacting the
warnings emitted at -O2.

Then using the new DIAG_IGNORE_Os_NEEDS_COMMENT we fix 6 warnings
generated with GCC 5 to get -Os builds working again.

Installed-header hygiene (BZ#20366): obsolete BSD u_* types.

2016-09-23T12:43:56+00:00

The types u_char, u_short, u_int, u_long, ushort, uint, ulong, u_int8_t,
u_int16_t, u_int32_t, u_int64_t, quad_t, and u_quad_t are BSDisms that
have never been standardized.  While glibc should continue to *provide*
these types for compatibility's sake, its public headers should not
use them.

The meat of this change was mechanically generated by the following
shell command:

    perl -pi~ -e '
        s/\b(__)?u_char\b/unsigned char/g;
        s/\b(__)?u_?short\b/unsigned short/g;
        s/\b(__)?u_?int\b/unsigned int/g;
        s/\b(__)?u_?long\b/unsigned long/g;
        s/\b(__)?u_int8_t\b/uint8_t/g;
        s/\b(__)?u_int16_t\b/uint16_t/g;
        s/\b(__)?u_int32_t\b/uint32_t/g;
        s/\b(__)?u_int64_t\b/uint64_t/g;
        s/\b(__)?u_quad_t\b/uint64_t/g;
        s/\b(__)?quad_t\b/uint64_t/g;
    ' $(grep -lE -e '\<((__)?(quad_t|u(short|int|long|_(char|short|int([0-9]+_t)?|long|quad_t))))\>' \
        $(grep -LE '\<(_(SYS|BITS)_TYPES_H|rpc/(rpc|rpc_msg|types|xdr)\.h)\>' \
          $(find . \( -false $(sed 's/^/-o -name /' all-installed-headers) \
                   \) -printf '%P\n' | sort -u)))

where 'all-installed-headers' was a list of the basenames of all installed
header files, manually extracted from the Makefiles.  Non-installed
wrapper headers in include/ are also adjusted, for consistency.
I then manually fixed up indentation and line-wrapping.

sys/types.h and bits/types.h are excluded because they must continue
to define the u_* types (under __USE_MISC) for compatibility with
applications.  They do not use these types themselves.

All headers that (transitively) include rpc/types.h are also excluded,
for three reasons.  First, the u_* types are defined by rpc/types.h,
unconditionally (not just under __USE_MISC) so they are logically part
of the SunRPC API.  Second, many of those headers appear to be
machine-generated.  Third, it's my understanding that we are getting
rid of as much of SunRPC as possible in the near future.

(The one file under sunrpc/ that's touched, sunrpc/rpc/rpc_des.h, does
*not* include rpc/types.h.  This may itself be a bug.)

After changing from u_intNN_t to uintNN_t, a number of headers now
need to include stdint.h to pick up those types.  It might be more
hygenic, namespace-wise, to use __uintNN_t instead, but none of these
headers are bound by ISO or POSIX to do so, and it's unlikely that
anyone using them will be bothered.  (The two files that were using
__-prefixed versions of the u_types, sysdeps/mach/hurd/net/route.h and
sysdeps/unix/sysv/linux/net/route.h, both already also contained uses of
the unprefixed versions.)

Some of these files directly included features.h and/or sys/cdefs.h,
which I removed, as the style generally seems to be to let sys/types.h
do that for us.  (This does not change the set of definitions exposed
by any header; sys/types.h unconditionally includes both features.h
and sys/cdefs.h.)

One file included asm/types.h unnecessarily.

	* bits/in.h, gmon/sys/gmon.h, inet/netinet/igmp.h
	* inet/protocols/routed.h, inet/protocols/talkd.h
	* inet/protocols/timed.h, io/fts.h, nptl_db/thread_db.h
	* resolv/arpa/nameser.h, resolv/resolv.h, sunrpc/rpc/rpc_des.h
	* sysdeps/generic/netinet/if_ether.h
	* sysdeps/generic/netinet/in_systm.h
	* sysdeps/generic/netinet/ip.h, sysdeps/generic/netinet/tcp.h
	* sysdeps/gnu/netinet/ip_icmp.h, sysdeps/gnu/netinet/tcp.h
	* sysdeps/gnu/netinet/udp.h, sysdeps/mach/hurd/net/ethernet.h
	* sysdeps/mach/hurd/net/if_arp.h
	* sysdeps/mach/hurd/net/if_ppp.h
	* sysdeps/mach/hurd/net/route.h, sysdeps/mach/sys/reboot.h
	* sysdeps/unix/sysv/linux/bits/in.h
	* sysdeps/unix/sysv/linux/net/ethernet.h
	* sysdeps/unix/sysv/linux/net/if_arp.h
	* sysdeps/unix/sysv/linux/net/if_ppp.h
	* sysdeps/unix/sysv/linux/net/if_shaper.h
	* sysdeps/unix/sysv/linux/net/route.h
	* sysdeps/unix/sysv/linux/netinet/if_ether.h
	* sysdeps/unix/sysv/linux/netinet/if_fddi.h
	* sysdeps/unix/sysv/linux/netinet/if_tr.h
	* sysdeps/unix/sysv/linux/netipx/ipx.h
	* sysdeps/unix/sysv/linux/sys/acct.h
	* include/arpa/nameser.h, include/resolv.h:
	Change all uses of u_char to unsigned char,
	u_short and ushort to unsigned short, u_int and uint to unsigned int,
	u_long and ulong to unsigned long, u_int8_t to uint8_t,
        u_int16_t to uint16_t, u_int32_t to uint32_t, quad_t to int64_t,
	and u_int64_t and u_quad_t to uint64_t.

	* mach/sys/reboot.h: Remove two casts of integer literals
	to the types they already have.

	* bits/in.h: Correct error in description of IP_MULTICAST_LOOP.
	* sysdeps/unix/sysv/linux/bits/in.h: Likewise.
	* sysdeps/unix/sysv/linux/netinet/if_ether.h: Change a comment
	from referring to 'unsigned char' to 'uint8_t' for consistency with
	the macro definition below.

	* gmon/sys/gmon.h, inet/netinet/igmp.h, inet/protocols/talkd.h
	* io/fts.h, resolv/arpa/nameser.h, resolv/resolv.h
	* sunrpc/rpc/rpc_des.h, sysdeps/generic/netinet/ip.h
	* sysdeps/gnu/netinet/tcp.h, sysdeps/gnu/netinet/udp.h
	* sysdeps/mach/hurd/net/if_ppp.h, sysdeps/unix/sysv/linux/net/if_ppp.h
	* sysdeps/unix/sysv/linux/sys/acct.h
	* include/arpa/nameser.h, include/resolv.h:
	Fix indentation disrupted by mechanical edits.

	* inet/protocols/talkd.h, resolv/arpa/nameser.h
	* sysdeps/generic/netinet/in_systm.h
	* sysdeps/gnu/netinet/ip_icmp.h, sysdeps/gnu/netinet/tcp.h
	* sysdeps/gnu/netinet/udp.h
	* sysdeps/unix/sysv/linux/net/ethernet.h
	* sysdeps/unix/sysv/linux/net/if_arp.h
	* sysdeps/unix/sysv/linux/net/if_ppp.h
	* sysdeps/unix/sysv/linux/net/if_shaper.h
	* sysdeps/unix/sysv/linux/netinet/if_fddi.h
	* sysdeps/unix/sysv/linux/netinet/if_tr.h
	* sysdeps/unix/sysv/linux/netipx/ipx.h
	* sysdeps/unix/sysv/linux/sys/acct.h
	Include stdint.h for uintNN_t definitions.
	Don't include sys/cdefs.h, features.h, or asm/types.h directly.

Update and install proc_service.h [BZ #20311]

2016-08-03T14:26:32+00:00

This adds an include guard and __BEGIN/__END_DECLS to proc_service.h,
removes some extraneous "const"s, and then arranges to install the
header.  The idea here is to make it more convenient to implement the
proc_service.h API.