* sysdeps/generic/bits/libc-tsd.h [USE___THREAD]: Conditional

	changed from [USE_TLS && HAVE___THREAD].

	* sysdeps/i386/dl-machine.h (elf_machine_type_class, elf_machine_rel):
	Disable TLS relocs if [RTLD_BOOTSTRAP && !USE___THREAD].
	* sysdeps/x86_64/dl-machine.h
	(elf_machine_type_class, elf_machine_rela): Likewise.
	* sysdeps/sh/dl-machine.h (elf_machine_type_class, elf_machine_rela):
	Likewise.

	* include/link.h (struct link_map): Remove member l_tls_tp_initialized.
	* elf/rtld.c (_dl_start_final, dl_main): Don't use it.
	(_dl_start): Conditionalize PT_TLS check on [USE___THREAD].

	* sysdeps/i386/dl-tls.h (__TLS_GET_ADDR): Use ___tls_get_addr_internal
	instead of ___tls_get_addr.
	(___tls_get_addr_internal): Add attribute_hidden to decl.

	* sysdeps/generic/ldsodefs.h (struct rtld_global): New variable
	_dl_error_catch_tsd.
	* elf/rtld.c (startup_error_tsd): New function.
	(dl_main): Point _dl_error_catch_tsd at that.
	* elf/dl-error.c: Don't use libc-tsd.h for DL_ERROR,
	use new function pointer instead.
	* elf/dl-tsd.c: New file.
	* elf/Makefile (routines): Add it.

2002-10-07  Roland McGrath  <roland@redhat.com>

	* elf/dl-misc.c (_dl_debug_vdprintf): Use INTERNAL_SYSCALL macro for
	writev if it's available.  Otherwise if [RTLD_PRIVATE_ERRNO] then
	take _dl_load_lock around calling __writev.

	* sysdeps/unix/sysv/linux/i386/sysdep.h (INTERNAL_SYSCALL): New macro.
	(INLINE_SYSCALL): Use that.

	* sysdeps/generic/dl-sysdep.h: New file.
	* sysdeps/mach/hurd/dl-sysdep.h: New file.
	* sysdeps/generic/ldsodefs.h: Include <dl-sysdep.h>.
	* include/errno.h [IS_IN_rtld]: Include <dl-sysdep.h> to define ...
	[RTLD_PRIVATE_ERRNO]: Use a hidden global variable for errno and
	access it directly.
	* elf/dl-minimal.c (__errno_location): Removed.
	* sysdeps/unix/i386/sysdep.S (__syscall_errno) [RTLD_PRIVATE_ERRNO]:
	Use GOTOFF access for errno.
	* sysdeps/unix/sysv/linux/i386/sysdep.h
	[RTLD_PRIVATE_ERRNO] (SYSCALL_ERROR_HANDLER): Likewise.

	* sysdeps/unix/x86_64/sysdep.S (__syscall_errno) [RTLD_PRIVATE_ERRNO]:
	Use PC-relative access for errno.
	* sysdeps/unix/sysv/linux/x86_64/sysdep.h
	[RTLD_PRIVATE_ERRNO] (SYSCALL_ERROR_HANDLER): Likewise.

	* include/tls.h: New file.
	(USE___THREAD): New macro.
	Define to 1 under [USE_TLS && HAVE___THREAD] and only when compiling
	libc or libpthread.
	* sysdeps/unix/sysv/linux/i386/sysdep.h [USE___THREAD]: Conditional
	changed from [USE_TLS && HAVE___THREAD].
	* sysdeps/unix/sysv/linux/x86_64/sysdep.h: Likewise.
	* sysdeps/unix/i386/sysdep.S: Likewise.
	* sysdeps/unix/x86_64/sysdep.S: Likewise.
	* include/errno.h: Likewise.
	* include/netdb.h: Likewise.
	* include/resolv.h: Likewise.

	* sysdeps/generic/errno.c: New file.
	* csu/Makefile (aux): New variable, list errno.
	* sysdeps/unix/sysv/linux/i386/sysdep.S (errno, _errno): Remove defns.
	* sysdeps/unix/sysv/linux/m68k/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/arm/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/cris/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/hppa/sysdep.c: Likewise.
	* sysdeps/unix/sysv/linux/ia64/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/sysdep.c: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sysdep.S: Likewise.
	* sysdeps/unix/sysv/linux/sh/sysdep.S: Likewise.
	* sysdeps/unix/alpha/sysdep.S: Likewise.
	* sysdeps/generic/start.c: Likewise.
	* sysdeps/unix/start.c: Likewise.
	* sysdeps/unix/arm/start.c: Likewise.
	* sysdeps/unix/bsd/ultrix4/mips/start.S: Likewise.
	* sysdeps/unix/sparc/start.c: Likewise.
	* sysdeps/unix/sysv/irix4/start.c: Likewise.
	* sysdeps/unix/sysv/linux/mips/sysdep.S: File removed.

	* manual/search.texi (Tree Search Function, Hash Search Function):
	Mention search.h clearly.

2002-10-05  Roland McGrath  <roland@redhat.com>

	* elf/dl-fxstat64.c: File removed.
	* elf/dl-xstat64.c: File removed.
	* elf/Makefile (rtld-routines): Remove them.
	* sysdeps/unix/sysv/linux/xstat64.c: Remove RTLD_STAT64 conditionals.
	Instead, use strong_alias instead of versioned_symbol in the
	!SHLIB_COMPAT case.
	* sysdeps/unix/sysv/linux/fxstat64.c: Likewise.
	* sysdeps/unix/sysv/linux/lxstat64.c: Likewise.

	* include/shlib-compat.h
	(SHLIB_COMPAT): Require that IS_IN_##lib be defined nonzero.
	[! NOT_IN_libc] (IS_IN_libc): Define it.
	* cppflags-iterator.mk (CPPFLAGS-$(cpp-src)): Use -Dx=1 not just -Dx.
	* elf/Makefile (CPPFLAGS-.os): Likewise.

	* sunrpc/rpc_main.c (main): Don't declare with noreturn attribute.
	Return the status instead of calling exit.

	* Makeconfig (CFLAGS): Prepend -std=gnu99.
	* Makerules (+make-deps): Use $(CFLAGS) only for .c sources.
	Remove superfluous rm command, whose @ plus make bugs hid
	all these commands from the make output.

	* include/stubs-prologue.h: New file.  Give #error under #ifdef _LIBC.
	* Makefile ($(inst_includedir)/gnu/stubs.h): Depend on it.
	Use that file's contents instead of literal echo's for the prologue.
	* include/features.h: Include <gnu/stubs.h> unconditionally.
	* include/gnu/stubs.h: New file.

2002-09-30  Roland McGrath  <roland@redhat.com>

	* elf/rtld-Rules: New file.
	* elf/Makefile ($(objpfx)librtld.map, $(objpfx)librtld.mk,
	$(objpfx)rtld-libc.a): New targets.
	(generated): Add them.
	(reloc-link): Remove -o $@ from the variable.
	($(objpfx)dl-allobjs.os): Add -o $@ after $(reloc-link).
	(distribute): Add rtld-Rules.
	(CPPFLAGS-.os): Define this instead of CFLAGS-.os.
	* Makerules ($(+sysdir_pfx)sysd-rules): Emit rules for rtld-% targets.
	(common-mostlyclean, common-clean): Clean up rtld-* files.
	* sysdeps/unix/make-syscalls.sh: Add rtld-*.os target name to rules.

36 files changed, 164 insertions, 266 deletions

diff --git a/sysdeps/generic/bits/libc-tsd.h b/sysdeps/generic/bits/libc-tsd.h
index 1954b20334..cc2c2c1219 100644
--- a/sysdeps/generic/bits/libc-tsd.h
+++ b/sysdeps/generic/bits/libc-tsd.h
@@ -51,7 +51,7 @@
    We don't define an enum for the possible key values, because the KEYs
    translate directly into variables by macro magic.  */
 
-#if USE_TLS && HAVE___THREAD
+#if USE___THREAD
 # define __libc_tsd_define(CLASS, KEY)	CLASS __thread void *__libc_tsd_##KEY;
 
 # define __libc_tsd_address(KEY)	(&__libc_tsd_##KEY)
diff --git a/sysdeps/generic/dl-sysdep.h b/sysdeps/generic/dl-sysdep.h
new file mode 100644
index 0000000000..b39103cbc2
--- /dev/null
+++ b/sysdeps/generic/dl-sysdep.h
@@ -0,0 +1,28 @@
+/* System-specific settings for dynamic linker code.  Generic version.
+   Copyright (C) 2002 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, write to the Free
+   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+   02111-1307 USA.  */
+
+/* This macro must be defined to either 0 or 1.
+
+   If 1, then an errno global variable hidden in ld.so will work right with
+   all the errno-using libc code compiled for ld.so, and there is never a
+   need to share the errno location with libc.  This is appropriate only if
+   all the libc functions that ld.so uses are called without PLT and always
+   get the versions linked into ld.so rather than the libc ones.  */
+
+#define RTLD_PRIVATE_ERRNO 1
diff --git a/sysdeps/unix/sysv/linux/mips/sysdep.S b/sysdeps/generic/errno.c
index 2584982bb0..cba5ab9c3b 100644
--- a/sysdeps/unix/sysv/linux/mips/sysdep.S
+++ b/sysdeps/generic/errno.c
@@ -1,4 +1,5 @@
-/* Copyright (C) 1998, 2002 Free Software Foundation, Inc.
+/* Definition of `errno' variable.  Canonical version.
+   Copyright (C) 2002 Free Software Foundation, Inc.
    This file is part of the GNU C Library.
 
    The GNU C Library is free software; you can redistribute it and/or
@@ -16,20 +17,16 @@
    Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
    02111-1307 USA.  */
 
-#include <sysdep.h>
+#include <errno.h>
+#include <tls.h>
+#undef errno
 
-/* The Linux version is in fact MIPS/ELF and the start.? file for this
-   system (sysdeps/mips/elf/start.S) is also used by The Hurd.  This file
-   must not contain the definition of the `errno' variable, we have to
-   define it somewhere else.
-
-   ...and this place is here.  */
-	.bss
-	.globl  errno
-	.type   errno,@object
-	.size   errno,4
-errno:	.word   4
-	.text
-weak_alias(errno, _errno)
-
-#include <sysdeps/unix/mips/sysdep.S>
+#if USE___THREAD
+__thread int errno;
+#else
+/* This differs from plain `int errno;' in that it doesn't create
+   a common definition, but a plain symbol that resides in .bss,
+   which can have an alias.  */
+int errno __attribute__ ((section (".bss")));
+strong_alias (errno, _errno)
+#endif
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index c149656446..36582a7463 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -34,6 +34,7 @@
 #include <sys/mman.h>
 #include <link.h>
 #include <dl-lookupcfg.h>
+#include <dl-sysdep.h>
 #include <bits/libc-lock.h>
 #include <hp-timing.h>
 #include <tls.h>
@@ -372,6 +373,10 @@ struct rtld_global
 
   /* Structure describing the dynamic linker itself.  */
   EXTERN struct link_map _dl_rtld_map;
+
+#ifdef _LIBC_REENTRANT
+  EXTERN void **(*_dl_error_catch_tsd) (void) __attribute__ ((const));
+#endif
 #ifdef SHARED
 };
 # define __rtld_global_attribute__
diff --git a/sysdeps/generic/start.c b/sysdeps/generic/start.c
index cf7efc931a..08f985c482 100644
--- a/sysdeps/generic/start.c
+++ b/sysdeps/generic/start.c
@@ -11,6 +11,3 @@ int __data_start = 0;
 #ifdef HAVE_WEAK_SYMBOLS
 weak_alias (__data_start, data_start)
 #endif
-
-volatile int __errno;
-strong_alias (__errno, errno)
diff --git a/sysdeps/i386/dl-machine.h b/sysdeps/i386/dl-machine.h
index 1784a3a737..6f60cf3af0 100644
--- a/sysdeps/i386/dl-machine.h
+++ b/sysdeps/i386/dl-machine.h
@@ -299,7 +299,7 @@ _dl_start_user:\n\
    define the value.
    ELF_RTYPE_CLASS_NOCOPY iff TYPE should not be allowed to resolve to one
    of the main executable's symbols, as for a COPY reloc.  */
-#ifdef USE_TLS
+#if defined USE_TLS && (!defined RTLD_BOOTSTRAP || USE___THREAD)
 # define elf_machine_type_class(type) \
   ((((type) == R_386_JMP_SLOT || (type) == R_386_TLS_DTPMOD32		      \
      || (type) == R_386_TLS_DTPOFF32 || (type) == R_386_TLS_TPOFF32	      \
@@ -411,7 +411,7 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc,
 	  *reloc_addr = value;
 	  break;
 
-#ifdef USE_TLS
+#if defined USE_TLS && (!defined RTLD_BOOTSTRAP || USE___THREAD)
 	case R_386_TLS_DTPMOD32:
 # ifdef RTLD_BOOTSTRAP
 	  /* During startup the dynamic linker is always the module
diff --git a/sysdeps/i386/dl-tls.h b/sysdeps/i386/dl-tls.h
index 5066b8dcd6..b34bbe2c9e 100644
--- a/sysdeps/i386/dl-tls.h
+++ b/sysdeps/i386/dl-tls.h
@@ -31,7 +31,7 @@ typedef struct
 extern void *___tls_get_addr (tls_index *ti)
      __attribute__ ((__regparm__ (1)));
 extern void *___tls_get_addr_internal (tls_index *ti)
-     __attribute__ ((__regparm__ (1)));
+     __attribute__ ((__regparm__ (1))) attribute_hidden;
 
 /* The special thing about the x86 TLS ABI is that we have two
    variants of the __tls_get_addr function with different calling
@@ -50,5 +50,5 @@ __tls_get_addr (tls_index *ti)
    version of this file.  */
 # define __tls_get_addr __attribute__ ((__regparm__ (1))) ___tls_get_addr
 strong_alias (___tls_get_addr, ___tls_get_addr_internal)
-# define __TLS_GET_ADDR ___tls_get_addr
+# define __TLS_GET_ADDR ___tls_get_addr_internal
 #endif
diff --git a/sysdeps/mach/hurd/dl-sysdep.h b/sysdeps/mach/hurd/dl-sysdep.h
new file mode 100644
index 0000000000..2dc9b0a910
--- /dev/null
+++ b/sysdeps/mach/hurd/dl-sysdep.h
@@ -0,0 +1,25 @@
+/* System-specific settings for dynamic linker code.  Hurd version.
+   Copyright (C) 2002 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, write to the Free
+   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
+   02111-1307 USA.  */
+
+/* The private errno doesn't make sense on the Hurd.  errno is always the
+   thread-local slot shared with libc, and it matters to share the cell
+   with libc because after startup we use libc functions that set errno
+   (open, mmap, etc).  */
+
+#define RTLD_PRIVATE_ERRNO 0
diff --git a/sysdeps/sh/dl-machine.h b/sysdeps/sh/dl-machine.h
index 350ac5297a..5e264d3cdd 100644
--- a/sysdeps/sh/dl-machine.h
+++ b/sysdeps/sh/dl-machine.h
@@ -401,7 +401,7 @@ _dl_start_user:\n\
    define the value.
    ELF_RTYPE_CLASS_NOCOPY iff TYPE should not be allowed to resolve to one
    of the main executable's symbols, as for a COPY reloc.  */
-#ifdef USE_TLS
+#if defined USE_TLS && (!defined RTLD_BOOTSTRAP || USE___THREAD)
 # define elf_machine_type_class(type) \
   ((((type) == R_SH_JMP_SLOT || (type) == R_SH_TLS_DTPMOD32		      \
      || (type) == R_SH_TLS_DTPOFF32 || (type) == R_SH_TLS_TPOFF32)	      \
@@ -546,7 +546,7 @@ elf_machine_rela (struct link_map *map, const Elf32_Rela *reloc,
 	  /* These addresses are always aligned.  */
 	  *reloc_addr = value;
 	  break;
-#ifdef USE_TLS
+#if defined USE_TLS && (!defined RTLD_BOOTSTRAP || USE___THREAD)
 	  /* XXX Remove TLS relocations which are not needed.  */
 	case R_SH_TLS_DTPMOD32:
 # ifdef RTLD_BOOTSTRAP
diff --git a/sysdeps/unix/alpha/sysdep.S b/sysdeps/unix/alpha/sysdep.S
index 5279b86076..05c00918e6 100644
--- a/sysdeps/unix/alpha/sysdep.S
+++ b/sysdeps/unix/alpha/sysdep.S
@@ -20,19 +20,6 @@
 #include <sysdep.h>
 #include <features.h>
 
-	.section .bss
-	.globl errno
-	.align 2
-errno:	.space 4
-#ifdef __ELF__
-	.type errno, @object
-	.size errno, 4
-#endif
-	.globl __errno
-__errno = errno
-	.globl _errno
-_errno = errno
-
 	.text
 	.align 2
 
diff --git a/sysdeps/unix/arm/start.c b/sysdeps/unix/arm/start.c
index a8fea14b9f..6088faaed1 100644
--- a/sysdeps/unix/arm/start.c
+++ b/sysdeps/unix/arm/start.c
@@ -28,11 +28,6 @@ int __data_start = 0;
 weak_alias (__data_start, data_start)
 #endif
 
-#ifndef errno
-volatile int __errno;
-strong_alias (__errno, errno)
-#endif
-
 extern void __libc_init __P ((int argc, char **argv, char **envp));
 extern int main __P ((int argc, char **argv, char **envp));
 
diff --git a/sysdeps/unix/bsd/ultrix4/mips/start.S b/sysdeps/unix/bsd/ultrix4/mips/start.S
index df86229ea5..c2fc4882d8 100644
--- a/sysdeps/unix/bsd/ultrix4/mips/start.S
+++ b/sysdeps/unix/bsd/ultrix4/mips/start.S
@@ -19,9 +19,6 @@
 
 #include <sysdep.h>
 
-__errno:
-.comm errno,		4
-
 ENTRY(__start)
   .set noreorder
 
diff --git a/sysdeps/unix/i386/sysdep.S b/sysdeps/unix/i386/sysdep.S
index 294865877f..dceb6815f4 100644
--- a/sysdeps/unix/i386/sysdep.S
+++ b/sysdeps/unix/i386/sysdep.S
@@ -23,6 +23,10 @@
 #include <bp-asm.h>
 #include <bp-sym.h>
 
+#ifdef IS_IN_rtld
+# include <dl-sysdep.h>		/* Defines RTLD_PRIVATE_ERRNO.  */
+#endif
+
 .globl C_SYMBOL_NAME(errno)
 .globl syscall_error
 
@@ -42,7 +46,7 @@ syscall_error:
 notb:
 #endif
 #ifndef	PIC
-# if USE_TLS && HAVE___THREAD
+# if USE___THREAD
 	movl %eax, %gs:C_SYMBOL_NAME(errno@NTPOFF)
 # elif !defined _LIBC_REENTRANT
 	movl %eax, C_SYMBOL_NAME(errno)
@@ -57,12 +61,17 @@ notb:
 #else
 	/* The caller has pushed %ebx and then set it up to
 	   point to the GOT before calling us through the PLT.  */
-# if USE_TLS && HAVE___THREAD
+# if USE___THREAD
 	movl C_SYMBOL_NAME(errno@GOTNTPOFF)(%ebx), %ecx
 
 	/* Pop %ebx value saved before jumping here.  */
 	popl %ebx
 	movl %eax, %gs:0(%ecx)
+# elif RTLD_PRIVATE_ERRNO
+	movl %eax, C_SYMBOL_NAME(errno@GOTOFF)(%ebx)
+
+	/* Pop %ebx value saved before jumping here.  */
+	popl %ebx
 # elif !defined _LIBC_REENTRANT
 	movl C_SYMBOL_NAME(errno@GOT)(%ebx), %ecx
 
diff --git a/sysdeps/unix/make-syscalls.sh b/sysdeps/unix/make-syscalls.sh
index 0a7b5fba56..e85a21dd9b 100644
--- a/sysdeps/unix/make-syscalls.sh
+++ b/sysdeps/unix/make-syscalls.sh
@@ -128,7 +128,8 @@ shared-only-routines += $file
     ;;
   *)
     echo "\
-\$(foreach o,\$(object-suffixes),\$(objpfx)$file\$o): \\"
+\$(foreach o,\$(object-suffixes),\$(objpfx)$file\$o) \
+\$(objpfx)rtld-$file.os: \\"
     ;;
   esac
 
diff --git a/sysdeps/unix/sparc/start.c b/sysdeps/unix/sparc/start.c
index 3191753505..664cb9dbaf 100644
--- a/sysdeps/unix/sparc/start.c
+++ b/sysdeps/unix/sparc/start.c
@@ -39,10 +39,6 @@ int __data_start = 0;
 weak_alias (__data_start, data_start)
 #endif
 
-VOLATILE int __errno;
-strong_alias (__errno, errno)
-
-
 extern void __libc_init __P ((int argc, char **argv, char **envp));
 extern int main __P ((int argc, char **argv, char **envp));
 
diff --git a/sysdeps/unix/start.c b/sysdeps/unix/start.c
index 49c96403fe..af524751b4 100644
--- a/sysdeps/unix/start.c
+++ b/sysdeps/unix/start.c
@@ -35,13 +35,6 @@ weak_alias (__data_start, data_start)
 #define	DECL_DUMMIES
 #endif
 
-#ifndef errno
-/* __errno must be initialized since otherwise one cannot create an
-   alias (at least on some platforms).  */
-volatile int __errno = 0;
-strong_alias (__errno, errno)
-#endif
-
 extern void __libc_init (int argc, char **argv, char **envp);
 extern int main (int argc, char **argv, char **envp);
 
diff --git a/sysdeps/unix/sysv/irix4/start.c b/sysdeps/unix/sysv/irix4/start.c
index 719d017993..a88d0d4d1b 100644
--- a/sysdeps/unix/sysv/irix4/start.c
+++ b/sysdeps/unix/sysv/irix4/start.c
@@ -27,9 +27,6 @@
 /* The first piece of initialized data.  */
 int __data_start = 0;
 
-__volatile int __errno = 0;
-strong_alias (__errno, errno)
-
 extern void __libc_init __P ((int argc, char **argv, char **envp));
 extern int main __P ((int argc, char **argv, char **envp));
 
diff --git a/sysdeps/unix/sysv/linux/arm/sysdep.S b/sysdeps/unix/sysv/linux/arm/sysdep.S
index 48dcffae13..1a4de2adb3 100644
--- a/sysdeps/unix/sysv/linux/arm/sysdep.S
+++ b/sysdeps/unix/sysv/linux/arm/sysdep.S
@@ -18,13 +18,6 @@
 
 #include <sysdep.h>
 
-/* We define errno here, to be consistent with Linux/i386.  */
-	.bss
-	.globl C_SYMBOL_NAME(errno)
-	.type C_SYMBOL_NAME(errno),%object
-	.size C_SYMBOL_NAME(errno),4
-C_SYMBOL_NAME(errno):	.zero 4
-weak_alias (errno, _errno)
 	.text
 
 /* The syscall stubs jump here when they detect an error.
diff --git a/sysdeps/unix/sysv/linux/cris/sysdep.S b/sysdeps/unix/sysv/linux/cris/sysdep.S
index 64fb8505af..e53e7bbfb9 100644
--- a/sysdeps/unix/sysv/linux/cris/sysdep.S
+++ b/sysdeps/unix/sysv/linux/cris/sysdep.S
@@ -18,14 +18,6 @@
 
 #include <sysdep.h>
 
-/* Make space for the errno variable.  */
-
-	.globl	C_SYMBOL_NAME(errno)
-	.type	C_SYMBOL_NAME(errno),@object
-	.lcomm	C_SYMBOL_NAME(errno),4
-
-weak_alias (errno, _errno)
-
 /* The syscall stubs jump here when they detect an error, bot for PIC and
    non-PIC.  */
 
diff --git a/sysdeps/unix/sysv/linux/fxstat64.c b/sysdeps/unix/sysv/linux/fxstat64.c
index e64d8dfacb..f5e16050b6 100644
--- a/sysdeps/unix/sysv/linux/fxstat64.c
+++ b/sysdeps/unix/sysv/linux/fxstat64.c
@@ -84,17 +84,14 @@ ___fxstat64 (int vers, int fd, struct stat64 *buf)
 #endif
 }
 
-#ifndef RTLD_STAT64
-# include <shlib-compat.h>
+#include <shlib-compat.h>
 
+#if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
 versioned_symbol (libc, ___fxstat64, __fxstat64, GLIBC_2_2);
-
-# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
 strong_alias (___fxstat64, __old__fxstat64)
 compat_symbol (libc, __old__fxstat64, __fxstat64, GLIBC_2_1);
-# endif
-
+hidden_ver (___fxstat64, __fxstat64)
 #else
-strong_alias (___fxstat64, __fxstat64);
+strong_alias (___fxstat64, __fxstat64)
+hidden_def (__fxstat64)
 #endif
-hidden_ver (___fxstat64, __fxstat64)
diff --git a/sysdeps/unix/sysv/linux/hppa/sysdep.c b/sysdeps/unix/sysv/linux/hppa/sysdep.c
index b333b7013d..bf4d4af5c7 100644
--- a/sysdeps/unix/sysv/linux/hppa/sysdep.c
+++ b/sysdeps/unix/sysv/linux/hppa/sysdep.c
@@ -28,11 +28,6 @@ __syscall_error (int err_no)
   return -1;
 }
 
-/* We also have to have a 'real' definition of errno.  */
-#undef errno
-int errno = 0;
-weak_alias (errno, _errno)
-
 
 /* HPPA implements syscall() in 'C'; the assembler version would
    typically be in syscall.S.  */
diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.S b/sysdeps/unix/sysv/linux/i386/sysdep.S
index 9546bb3c64..b8df43cd7c 100644
--- a/sysdeps/unix/sysv/linux/i386/sysdep.S
+++ b/sysdeps/unix/sysv/linux/i386/sysdep.S
@@ -17,29 +17,6 @@
    02111-1307 USA.  */
 
 #include <sysdep.h>
-#include <tls.h>
-
-/* The Linux version is in fact i386/ELF and the start.? file for this
-   system (sysdeps/i386/elf/start.S) is also used by The Hurd.  This file
-   must not contain the definition of the `errno' variable, we have to
-   define it somewhere else.
-
-   ...and this place is here.  */
-#if USE_TLS && HAVE___THREAD
-	.section .tbss
-#else
-	.bss
-#endif
-	.globl	errno
-	.type errno,@object
-	.size errno,4
-	.globl	_errno
-	.type _errno,@object
-	.size _errno,4
-	.align 4
-errno:
-_errno:
-	.space	4
 
 /* The following code is only used in the shared library when we
    compile the reentrant version.  Otherwise each system call defines
diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h
index ed0bdf6b7d..87dfe9042c 100644
--- a/sysdeps/unix/sysv/linux/i386/sysdep.h
+++ b/sysdeps/unix/sysv/linux/i386/sysdep.h
@@ -26,6 +26,12 @@
 #include <bp-asm.h>
 #include <tls.h>
 
+
+#ifdef IS_IN_rtld
+# include <dl-sysdep.h>		/* Defines RTLD_PRIVATE_ERRNO.  */
+#endif
+
+
 /* For Linux we can use the system call table in the header file
 	/usr/include/asm/unistd.h
    of the kernel.  But these symbols do not follow the SYS_* syntax
@@ -92,10 +98,19 @@ __i686.get_pc_thunk.reg:						      \
   call __i686.get_pc_thunk.reg
 # endif
 
-/* Store (- %eax) into errno through the GOT.  */
-# ifdef _LIBC_REENTRANT
+# if RTLD_PRIVATE_ERRNO
+#  define SYSCALL_ERROR_HANDLER						      \
+0:SETUP_PIC_REG(cx);							      \
+  addl $_GLOBAL_OFFSET_TABLE_, %ecx;					      \
+  xorl %edx, %edx;							      \
+  subl %eax, %edx;							      \
+  movl %edx, errno@GOTOFF(%ecx);					      \
+  orl $-1, %eax;							      \
+  jmp L(pseudo_end);
 
-#  if USE_TLS && HAVE___THREAD
+# elif defined _LIBC_REENTRANT
+
+#  if USE___THREAD
 #   define SYSCALL_ERROR_HANDLER					      \
 0:SETUP_PIC_REG (cx);							      \
   addl $_GLOBAL_OFFSET_TABLE_, %ecx;					      \
@@ -125,6 +140,7 @@ __i686.get_pc_thunk.reg:						      \
    not modify the stack!  */
 #  endif
 # else
+/* Store (- %eax) into errno through the GOT.  */
 #  define SYSCALL_ERROR_HANDLER						      \
 0:SETUP_PIC_REG(cx);						      \
   addl $_GLOBAL_OFFSET_TABLE_, %ecx;					      \
@@ -274,6 +290,21 @@ asm (".L__X'%ebx = 1\n\t"
 #undef INLINE_SYSCALL
 #define INLINE_SYSCALL(name, nr, args...) \
   ({									      \
+    unsigned int resultvar = INTERNAL_SYSCALL(name, nr, args);		      \
+    if (resultvar >= 0xfffff001)					      \
+      {									      \
+	__set_errno (-resultvar);					      \
+	resultvar = 0xffffffff;						      \
+      }									      \
+    (int) resultvar; })
+
+/* Define a macro which expands inline into the wrapper code for a system
+   call.  This use is for internal calls that do not need to handle errors
+   normally.  It will never touch errno.  This returns just what the kernel
+   gave back.  */
+#undef INTERNAL_SYSCALL
+#define INTERNAL_SYSCALL(name, nr, args...) \
+  ({									      \
     unsigned int resultvar;						      \
     asm volatile (							      \
     LOADARGS_##nr							      \
@@ -282,11 +313,6 @@ asm (".L__X'%ebx = 1\n\t"
     RESTOREARGS_##nr							      \
     : "=a" (resultvar)							      \
     : "i" (__NR_##name) ASMFMT_##nr(args) : "memory", "cc");		      \
-    if (resultvar >= 0xfffff001)					      \
-      {									      \
-	__set_errno (-resultvar);					      \
-	resultvar = 0xffffffff;						      \
-      }									      \
     (int) resultvar; })
 
 #define LOADARGS_0
diff --git a/sysdeps/unix/sysv/linux/ia64/sysdep.S b/sysdeps/unix/sysv/linux/ia64/sysdep.S
index 2008b84e88..31b5f9bde9 100644
--- a/sysdeps/unix/sysv/linux/ia64/sysdep.S
+++ b/sysdeps/unix/sysv/linux/ia64/sysdep.S
@@ -20,17 +20,6 @@
 #include <sysdep.h>
 #include <features.h>
 
-	.global errno
-	.common errno,4,4
-	.type errno, @object
-	.size errno, 4
-
-	.global __errno
-__errno = errno
-
-	.global _errno
-_errno = errno
-
 ENTRY(__syscall_error)
 #ifdef _LIBC_REENTRANT
 	.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(0)
diff --git a/sysdeps/unix/sysv/linux/lxstat64.c b/sysdeps/unix/sysv/linux/lxstat64.c
index 0a29200014..e7f488848c 100644
--- a/sysdeps/unix/sysv/linux/lxstat64.c
+++ b/sysdeps/unix/sysv/linux/lxstat64.c
@@ -85,17 +85,14 @@ ___lxstat64 (int vers, const char *name, struct stat64 *buf)
 #endif
 }
 
-#ifndef RTLD_STAT64
-# include <shlib-compat.h>
+#include <shlib-compat.h>
 
+#if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
 versioned_symbol (libc, ___lxstat64, __lxstat64, GLIBC_2_2);
-
-# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
 strong_alias (___lxstat64, __old__lxstat64)
 compat_symbol (libc, __old__lxstat64, __lxstat64, GLIBC_2_1);
-# endif
-
+hidden_ver (___lxstat64, __lxstat64)
 #else
 strong_alias (___lxstat64, __lxstat64);
+hidden_def (__lxstat64)
 #endif
-hidden_ver (___lxstat64, __lxstat64)
diff --git a/sysdeps/unix/sysv/linux/m68k/sysdep.S b/sysdeps/unix/sysv/linux/m68k/sysdep.S
index 628335b8d0..e4ec92d836 100644
--- a/sysdeps/unix/sysv/linux/m68k/sysdep.S
+++ b/sysdeps/unix/sysv/linux/m68k/sysdep.S
@@ -18,19 +18,6 @@
 
 #include <sysdep.h>
 
-/* The Linux version is in fact m68k/ELF and the start.? file for this
-   system (sysdeps/m68k/elf/start.S) is also used by The Hurd.  This file
-   must not contain the definition of the `errno' variable, we have to
-   define it somewhere else.
-
-   ...and this place is here.  */
-	.bss
-	.globl errno
-	.type errno,@object
-errno:	.space 4
-	.size errno,4
-weak_alias (errno, _errno)
-	.text
 
 /* The following code is only used in the shared library when we
    compile the reentrant version.  Otherwise each system call defines
diff --git a/sysdeps/unix/sysv/linux/powerpc/sysdep.c b/sysdeps/unix/sysv/linux/powerpc/sysdep.c
index 8703c03e25..f16c4c938f 100644
--- a/sysdeps/unix/sysv/linux/powerpc/sysdep.c
+++ b/sysdeps/unix/sysv/linux/powerpc/sysdep.c
@@ -27,8 +27,3 @@ __syscall_error (int err_no)
   __set_errno (err_no);
   return -1;
 }
-
-/* We also have to have a 'real' definition of errno.  */
-#undef errno
-int errno = 0;
-weak_alias (errno, _errno)
diff --git a/sysdeps/unix/sysv/linux/s390/s390-32/sysdep.S b/sysdeps/unix/sysv/linux/s390/s390-32/sysdep.S
index 5b17f6ba4b..98b00722d8 100644
--- a/sysdeps/unix/sysv/linux/s390/s390-32/sysdep.S
+++ b/sysdeps/unix/sysv/linux/s390/s390-32/sysdep.S
@@ -19,20 +19,6 @@
 
 #include <sysdep.h>
 
-/* The Linux version is in fact S390-32/ELF and the start.? file for this
-   system (sysdeps/s390/s390-32/elf/start.S) is also used by The Hurd.
-   This file must not contain the definition of the `errno' variable,
-   we have to define it somewhere else.
-
-    ...and this place is here.  */
-	.bss
-	.globl	errno
-	.type errno,@object
-	.size errno,4
-errno:
-	.space	4
-weak_alias (errno, _errno)
-
 /* The following code is only used in the shared library when we
    compile the reentrant version.  Otherwise each system call defines
    each own version.  */
diff --git a/sysdeps/unix/sysv/linux/s390/s390-64/sysdep.S b/sysdeps/unix/sysv/linux/s390/s390-64/sysdep.S
index caa50ccc3d..a29b0b983a 100644
--- a/sysdeps/unix/sysv/linux/s390/s390-64/sysdep.S
+++ b/sysdeps/unix/sysv/linux/s390/s390-64/sysdep.S
@@ -19,19 +19,6 @@
 
 #include <sysdep.h>
 
-/* The Linux version is in fact S390-64/ELF and the start.? file for this
-   system (sysdeps/s390/s390-64/elf/start.S) is also used by The Hurd.
-   This file must not contain the definition of the `errno' variable,
-   we have to define it somewhere else.
-
-    ...and this place is here.  */
-	.bss
-	.globl	errno
-	.type errno,@object
-	.size errno,4
-errno:
-	.space	4
-weak_alias (errno, _errno)
 
 /* The following code is only used in the shared library when we
    compile the reentrant version.  Otherwise each system call defines
diff --git a/sysdeps/unix/sysv/linux/sh/sysdep.S b/sysdeps/unix/sysv/linux/sh/sysdep.S
index 3f4b0dd0a2..33fdc748b5 100644
--- a/sysdeps/unix/sysv/linux/sh/sysdep.S
+++ b/sysdeps/unix/sysv/linux/sh/sysdep.S
@@ -18,18 +18,6 @@
 
 #include <sysdep.h>
 
-/* We define errno here, to be consistent with Linux/i386.  */
-
-	.section .bss
-	.align 2
-	.globl C_SYMBOL_NAME(errno)
-	.type C_SYMBOL_NAME(errno), @object
-	.size C_SYMBOL_NAME(errno), 4
-C_SYMBOL_NAME(errno):
-	.space	4
-weak_alias (errno, _errno)
-	.text
-
 /* The syscall stubs jump here when they detect an error.
    The code for Linux is almost identical to the canonical Unix
    code, except that the error number in R0 is negated.  */
diff --git a/sysdeps/unix/sysv/linux/sparc/sysdep.S b/sysdeps/unix/sysv/linux/sparc/sysdep.S
index 437060e866..b031238ceb 100644
--- a/sysdeps/unix/sysv/linux/sparc/sysdep.S
+++ b/sysdeps/unix/sysv/linux/sparc/sysdep.S
@@ -1,30 +1 @@
-/* Copyright (C) 1997, 1998, 1999 Free Software Foundation, Inc.
-   This file is part of the GNU C Library.
-   Contributed by Richard Henderson <richard@gnu.ai.mit.edu>, 1997.
-
-   The GNU C Library is free software; you can redistribute it and/or
-   modify it under the terms of the GNU Lesser General Public
-   License as published by the Free Software Foundation; either
-   version 2.1 of the License, or (at your option) any later version.
-
-   The GNU C Library is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Lesser General Public License for more details.
-
-   You should have received a copy of the GNU Lesser General Public
-   License along with the GNU C Library; if not, write to the Free
-   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
-   02111-1307 USA.  */
-
-
-/* Define errno */
-
-	.section .bss
-	.globl errno
-	.align 4
-errno:	.space 4
-	.type errno, @object
-	.size errno, 4
-
-weak_alias (errno, _errno)
+/* Nothing to do here.  */
diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.S b/sysdeps/unix/sysv/linux/x86_64/sysdep.S
index 00086ca1e4..5b72cef205 100644
--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.S
+++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.S
@@ -17,30 +17,6 @@
    02111-1307 USA.  */
 
 #include <sysdep.h>
-#include <tls.h>
-
-/* The Linux version is in fact x86-64/ELF and the start.? file for this
-   system (sysdeps/x86_64/elf/start.S) is also used by The Hurd.  This file
-   must not contain the definition of the `errno' variable, we have to
-   define it somewhere else.
-
-   ...and this place is here.  */
-#if USE_TLS && HAVE___THREAD
-	.section .tbss
-#else
-	.bss
-#endif
-	.globl	errno
-	.type errno,@object
-	.size errno,4
-	.globl	_errno
-	.type _errno,@object
-	.size _errno,4
-	.align 4
-errno:
-_errno:
-	.space	4
-
 
 /* The following code is only used in the shared library when we
    compile the reentrant version.  Otherwise each system call defines
diff --git a/sysdeps/unix/sysv/linux/x86_64/sysdep.h b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
index 5588b4606e..b0f51b4072 100644
--- a/sysdeps/unix/sysv/linux/x86_64/sysdep.h
+++ b/sysdeps/unix/sysv/linux/x86_64/sysdep.h
@@ -24,6 +24,10 @@
 #include <bp-sym.h>
 #include <bp-asm.h>
 
+#ifdef IS_IN_rtld
+# include <dl-sysdep.h>		/* Defines RTLD_PRIVATE_ERRNO.  */
+#endif
+
 /* For Linux we can use the system call table in the header file
 	/usr/include/asm/unistd.h
    of the kernel.  But these symbols do not follow the SYS_* syntax
@@ -78,12 +82,18 @@
 
 #ifndef PIC
 #define SYSCALL_ERROR_HANDLER	/* Nothing here; code in sysdep.S is used.  */
-#elif USE_TLS && HAVE___THREAD
+#elif USE___THREAD
 # define SYSCALL_ERROR_HANDLER			\
   movq errno@GOTTPOFF(%rip), %rcx;		\
   xorq %rdx, %rdx;				\
   subq %rax, %rdx;				\
   movl %eax, %fs:0(%rcx)
+#elif RTLD_PRIVATE_ERRNO
+# define SYSCALL_ERROR_HANDLER			\
+  leaq errno(%rip), %rcx;			\
+  xorq %rdx, %rdx;				\
+  subq %rax, %rdx;				\
+  movl %eax, (%rcx)
 #elif defined _LIBC_REENTRANT
 /* Store (- %rax) into errno through the GOT.
    Note that errno occupies only 4 bytes.  */
diff --git a/sysdeps/unix/sysv/linux/xstat64.c b/sysdeps/unix/sysv/linux/xstat64.c
index 388ff8fa68..7835fc2050 100644
--- a/sysdeps/unix/sysv/linux/xstat64.c
+++ b/sysdeps/unix/sysv/linux/xstat64.c
@@ -88,17 +88,15 @@ ___xstat64 (int vers, const char *name, struct stat64 *buf)
 #endif
 }
 
-#ifndef RTLD_STAT64
-# include <shlib-compat.h>
 
-versioned_symbol (libc, ___xstat64, __xstat64, GLIBC_2_2);
+#include <shlib-compat.h>
 
-# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
+#if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
+versioned_symbol (libc, ___xstat64, __xstat64, GLIBC_2_2);
 strong_alias (___xstat64, __old__xstat64)
 compat_symbol (libc, __old__xstat64, __xstat64, GLIBC_2_1);
-# endif
-
+hidden_ver (___xstat64, __xstat64)
 #else
-strong_alias (___xstat64, __xstat64);
+strong_alias (___xstat64, __xstat64)
+hidden_def (__xstat64)
 #endif
-hidden_ver (___xstat64, __xstat64)
diff --git a/sysdeps/unix/x86_64/sysdep.S b/sysdeps/unix/x86_64/sysdep.S
index b0580a39ae..dfa92dcd2c 100644
--- a/sysdeps/unix/x86_64/sysdep.S
+++ b/sysdeps/unix/x86_64/sysdep.S
@@ -23,6 +23,10 @@
 #include <bp-sym.h>
 #include <tls.h>
 
+#ifdef IS_IN_rtld
+# include <dl-sysdep.h>		/* Defines RTLD_PRIVATE_ERRNO.  */
+#endif
+
 .globl C_SYMBOL_NAME(errno)
 .globl syscall_error
 
@@ -41,7 +45,7 @@ syscall_error:
 	movq $EAGAIN, %rax	/* Yes; translate it to EAGAIN.  */
 notb:
 #endif
-#if USE_TLS && HAVE___THREAD
+#if USE___THREAD
 # ifdef PIC
 	movq C_SYMBOL_NAME(errno@GOTTPOFF)(%rip), %rcx
 	movl %eax, %fs:0(%rcx)
@@ -60,7 +64,10 @@ notb:
 	movl %ecx, (%rax)
 # endif
 #else
-# ifndef _LIBC_REENTRANT
+# if RTLD_PRIVATE_ERRNO
+	leaq errno(%rip), %rcx
+	movl %eax, (%rcx)
+# elif !defined _LIBC_REENTRANT
 	movq C_SYMBOL_NAME(errno)@GOTPCREL(%rip), %rcx
 	movl %eax, (%rcx)
 # else
diff --git a/sysdeps/x86_64/dl-machine.h b/sysdeps/x86_64/dl-machine.h
index 21459b9ab5..740d6b7feb 100644
--- a/sysdeps/x86_64/dl-machine.h
+++ b/sysdeps/x86_64/dl-machine.h
@@ -288,7 +288,7 @@ _dl_start_user:\n\
    define the value.
    ELF_RTYPE_CLASS_NOCOPY iff TYPE should not be allowed to resolve to one
    of the main executable's symbols, as for a COPY reloc.  */
-#ifdef USE_TLS
+#if defined USE_TLS && (!defined RTLD_BOOTSTRAP || USE___THREAD)
 # define elf_machine_type_class(type)					      \
   ((((type) == R_X86_64_JUMP_SLOT					      \
      || (type) == R_X86_64_DTPMOD64					      \
@@ -389,7 +389,7 @@ elf_machine_rela (struct link_map *map, const Elf64_Rela *reloc,
 	value += sym->st_value;
 #endif
 
-#if defined RTLD_BOOTSTRAP && !(USE_TLS && HAVE___THREAD)
+#if defined RTLD_BOOTSTRAP && !USE___THREAD
       assert (r_type == R_X86_64_GLOB_DAT || r_type == R_X86_64_JUMP_SLOT);
       *reloc_addr = value + reloc->r_addend;
 #else