summaryrefslogtreecommitdiff
path: root/malloc
Commit message (Collapse)AuthorAgeFilesLines
* Call the right helper function when setting mallopt M_ARENA_MAX (BZ #21338)Wladimir J. van der Laan2017-04-031-1/+1
| | | | | | | | | | | | Fixes a typo introduced in commit be7991c0705e35b4d70a419d117addcd6c627319. This caused mallopt(M_ARENA_MAX) as well as the environment variable MALLOC_ARENA_MAX to not work as intended because it set the wrong internal parameter. [BZ #21338] * malloc/malloc.c: Call do_set_arena_max for M_ARENA_MAX instead of incorrect do_set_arena_test
* Fix getting tunable values on big-endian (BZ #21109)Siddhesh Poyarekar2017-02-091-4/+4
| | | | | | | | | The code to set value passed a tunable_val_t, which when cast to int32_t on big-endian gives the wrong value. Instead, use tunable_val_t.numval instead, which can then be safely cast into int32_t. (cherry picked from commit 8cbc826c37c0221ada65a7a622fe079b4e89a4b0)
* Increase some test timeouts.Joseph Myers2017-01-052-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch increases timeouts on some tests I've observed timing out. elf/tst-tls13 and iconvdata/tst-loading both dynamically load many objects and so are slow when testing over NFS. They had timeouts set from before the default changed from 2 to 20 seconds; this patch removes those old settings, so effectively increasing the timeout to 20 seconds (from 3 and 10 seconds respectively). malloc/tst-malloc-thread-fail.c and malloc/tst-mallocfork2.c are slow on slow systems and so I set a fairly arbitrary 100 second timeout, which seems to suffice on the system where I saw them timing out. nss/tst-cancel-getpwuid_r.c and nss/tst-nss-getpwent.c are slow on systems with a large passwd file; I set timeouts that empirically worked for me. (It seems tst-cancel-getpwuid_r.c is hitting the 100000 getpwuid_r call limit in my testing, with each call taking a bit over 0.007 seconds, so 700 seconds for the test.) * elf/tst-tls13.c (TIMEOUT): Remove. * iconvdata/tst-loading.c (TIMEOUT): Likewise. * malloc/tst-malloc-thread-fail.c (TIMEOUT): Increase to 100. * malloc/tst-mallocfork2.c (TIMEOUT): Define to 100. * nss/tst-cancel-getpwuid_r.c (TIMEOUT): Define to 900. * nss/tst-nss-getpwent.c (TIMEOUT): Define to 300.
* Fix malloc/ tests for GCC 7 -Walloc-size-larger-than=.Joseph Myers2017-01-043-0/+52
| | | | | | | | | | | | | | | | | | | | | GCC 7 has a -Walloc-size-larger-than= warning for allocations of half the address space or more. This causes errors building glibc tests that deliberately test failure of very large allocations. This patch arranges for this warning to be ignored around the problematic function calls. Tested compilation for aarch64 (GCC mainline) with build-many-glibcs.py; did execution testing for x86_64 (GCC 5). * malloc/tst-malloc.c: Include <libc-internal.h>. (do_test): Disable -Walloc-size-larger-than= around tests of malloc with negative sizes. * malloc/tst-mcheck.c: Include <libc-internal.h>. (do_test): Disable -Walloc-size-larger-than= around tests of malloc and realloc with negative sizes. * malloc/tst-realloc.c: Include <libc-internal.h>. (do_test): Disable -Walloc-size-larger-than= around tests of realloc with negative sizes.
* malloc: Run tunables tests only if tunables are enabledFlorian Weimer2017-01-011-2/+5
| | | | | Otherwise, the environment variable will not have any effect and the test will fail.
* Update copyright dates not handled by scripts/update-copyrights.Joseph Myers2017-01-013-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've updated copyright dates in glibc for 2017. This is the patch for the changes not generated by scripts/update-copyrights and subsequent build / regeneration of generated files. Please remember to include 2017 in the dates for any new files added in future (which means updating any existing uncommitted patches you have that add new files to use the new copyright dates in them). * NEWS: Update copyright dates. * catgets/gencat.c (print_version): Likewise. * csu/version.c (banner): Likewise. * debug/catchsegv.sh: Likewise. * debug/pcprofiledump.c (print_version): Likewise. * debug/xtrace.sh (do_version): Likewise. * elf/ldconfig.c (print_version): Likewise. * elf/ldd.bash.in: Likewise. * elf/pldd.c (print_version): Likewise. * elf/sotruss.sh: Likewise. * elf/sprof.c (print_version): Likewise. * iconv/iconv_prog.c (print_version): Likewise. * iconv/iconvconfig.c (print_version): Likewise. * locale/programs/locale.c (print_version): Likewise. * locale/programs/localedef.c (print_version): Likewise. * login/programs/pt_chown.c (print_version): Likewise. * malloc/memusage.sh (do_version): Likewise. * malloc/memusagestat.c (print_version): Likewise. * malloc/mtrace.pl: Likewise. * manual/libc.texinfo: Likewise. * nptl/version.c (banner): Likewise. * nscd/nscd.c (print_version): Likewise. * nss/getent.c (print_version): Likewise. * nss/makedb.c (print_version): Likewise. * posix/getconf.c (main): Likewise. * scripts/test-installation.pl: Likewise. * sysdeps/unix/sysv/linux/lddlibc4.c (main): Likewise.
* Update copyright dates with scripts/update-copyrights.Joseph Myers2017-01-0151-51/+51
|
* Initialize tunable list with the GLIBC_TUNABLES environment variableSiddhesh Poyarekar2016-12-313-1/+7
| | | | | | | | | | | | | | | | | | | | | | Read tunables values from the users using the GLIBC_TUNABLES environment variable. The value of this variable is a colon-separated list of name=value pairs. So a typical string would look like this: GLIBC_TUNABLES=glibc.malloc.mmap_threshold=2048:glibc.malloc.trim_threshold=1024 * config.make.in (have-loop-to-function): Define. * elf/Makefile (CFLAGS-dl-tunables.c): Add -fno-tree-loop-distribute-patterns. * elf/dl-tunables.c: Include libc-internals.h. (GLIBC_TUNABLES): New macro. (tunables_strdup): New function. (parse_tunables): New function. (min_strlen): New function. (__tunables_init): Use the new functions and macro. (disable_tunable): Disable tunable from GLIBC_TUNABLES. * malloc/tst-malloc-usable-tunables.c: New test case. * malloc/tst-malloc-usable-static-tunables.c: New test case. * malloc/Makefile (tests, tests-static): Add tests.
* Add framework for tunablesSiddhesh Poyarekar2016-12-313-0/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The tunables framework allows us to uniformly manage and expose global variables inside glibc as switches to users. tunables/README has instructions for glibc developers to add new tunables. Tunables support can be enabled by passing the --enable-tunables configure flag to the configure script. This patch only adds a framework and does not pose any limitations on how tunable values are read from the user. It also adds environment variables used in malloc behaviour tweaking to the tunables framework as a PoC of the compatibility interface. * manual/install.texi: Add --enable-tunables option. * INSTALL: Regenerate. * README.tunables: New file. * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE. (before-compile): Generate dl-tunable-list.h early. * config.h.in: Add HAVE_TUNABLES. * config.make.in: Add have-tunables. * configure.ac: Add --enable-tunables option. * configure: Regenerate. * csu/init-first.c (__libc_init_first): Move __libc_init_secure earlier... * csu/init-first.c (LIBC_START_MAIN):... to here. Include dl-tunables.h, libc-internal.h. (LIBC_START_MAIN) [!SHARED]: Initialize tunables for static binaries. * elf/Makefile (dl-routines): Add dl-tunables. * elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE namespace. * elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_ only when !HAVE_TUNABLES. * elf/rtld.c (process_envvars): Likewise. * elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h (_dl_sysdep_start): Call __tunables_init. * elf/dl-tunable-types.h: New file. * elf/dl-tunables.c: New file. * elf/dl-tunables.h: New file. * elf/dl-tunables.list: New file. * malloc/tst-malloc-usable-static.c: New test case. * malloc/Makefile (tests-static): Add it. * malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h. Define TUNABLE_NAMESPACE. (DL_TUNABLE_CALLBACK (set_mallopt_check)): New function. (DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define callback functions. (ptmalloc_init): Set tunable values. * scripts/gen-tunables.awk: New file. * sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h. (_dl_sysdep_start): Call __tunables_init.
* support: Introduce new subdirectory for test infrastructureFlorian Weimer2016-12-093-132/+35
| | | | | | | | | | | | | | | The new test driver in <support/test-driver.c> has feature parity with the old one. The main difference is that its hooking mechanism is based on functions and function pointers instead of macros. This commit also implements a new environment variable, TEST_COREDUMPS, which disables the code which disables coredumps (that is, it enables them if the invocation environment has not disabled them). <test-skeleton.c> defines wrapper functions so that it is possible to use existing macros with the new-style hook functionality. This commit changes only a few test cases to the new test driver, to make sure that it works as expected.
* malloc: Update comments about chunk layoutFlorian Weimer2016-10-281-10/+30
|
* sysmalloc: Initialize previous size field of mmaped chunksFlorian Weimer2016-10-281-0/+1
| | | | | With different encodings of the header, the previous zero initialization may be insufficient and produce an invalid encoding.
* malloc: Use accessors for chunk metadata accessFlorian Weimer2016-10-283-70/+91
| | | | | This change allows us to change the encoding of these struct members in a centralized fashion.
* Static inline functions for mallopt helpersSiddhesh Poyarekar2016-10-271-34/+93
| | | | | | | | | | | | | | | Make mallopt helper functions for each mallopt parameter so that it can be called consistently in other areas, like setting tunables. * malloc/malloc.c (do_set_mallopt_check): New function. (do_set_mmap_threshold): Likewise. (do_set_mmaps_max): Likewise. (do_set_top_pad): Likewise. (do_set_perturb_byte): Likewise. (do_set_trim_threshold): Likewise. (do_set_arena_max): Likewise. (do_set_arena_test): Likewise. (__libc_mallopt): Use them.
* malloc: Remove malloc_get_state, malloc_set_state [BZ #19473]Florian Weimer2016-10-265-100/+480
| | | | | | | | | | | | After the removal of __malloc_initialize_hook, newly compiled Emacs binaries are no longer able to use these interfaces. malloc_get_state is only used during the Emacs build process, so we provide a stub implementation only. Existing Emacs binaries will not call this stub function, but still reference the symbol. The rewritten tst-mallocstate test constructs a dumped heap which should approximates what existing Emacs binaries pass to glibc malloc.
* Remove redundant definitions of M_ARENA_* macrosSiddhesh Poyarekar2016-10-261-5/+0
| | | | | | | | | The M_ARENA_MAX and M_ARENA_TEST macros are defined in malloc.c as well as malloc.h, and the former is unnecessary. This patch removes the duplicate. Tested on x86_64 to verify that the generated code remains unchanged barring changed line numbers to __malloc_assert. * malloc/malloc.c (M_ARENA_TEST, M_ARENA_MAX): Remove.
* Document the M_ARENA_* mallopt parametersSiddhesh Poyarekar2016-10-261-1/+0
| | | | | | | | | | | The M_ARENA_* mallopt parameters are in wide use in production to control the number of arenas that a long lived process creates and hence there is no point in stating that this interface is non-public. Document this interface and remove the obsolete comment. * manual/memory.texi (M_ARENA_TEST): Add documentation. (M_ARENA_MAX): Likewise. * malloc/malloc.c: Remove obsolete comment.
* malloc: Manual part of conversion to __libc_lockFlorian Weimer2016-09-212-4/+4
| | | | | This removes the old mutex_t-related definitions from malloc-machine.h, too.
* Add tests-static to tests in malloc/MakefileSiddhesh Poyarekar2016-09-101-2/+1
| | | | | | | | | | This is a trivial change to add the static tests only to tests-static and then adding all of tests-static to the tests target to make it look consistent with some other Makefiles. This avoids having to duplicate the test names across the two make targets. * malloc/Makefile (tests): Remove individual static test names and just add all of tests-static.
* malloc: Automated part of conversion to __libc_lockFlorian Weimer2016-09-063-60/+60
|
* malloc: Simplify static malloc interposition [BZ #20432]Florian Weimer2016-08-2610-1/+648
| | | | | | | | Existing interposed mallocs do not define the glibc-internal fork callbacks (and they should not), so statically interposed mallocs lead to link failures because the strong reference from fork pulls in glibc's malloc, resulting in multiple definitions of malloc-related symbols.
* elf: dl-minimal malloc needs to respect fundamental alignmentFlorian Weimer2016-08-032-63/+53
| | | | | | | | | | | | | | The dynamic linker currently uses __libc_memalign for TLS-related allocations. The goal is to switch to malloc instead. If the minimal malloc follows the ABI fundamental alignment, we can assume that malloc provides this alignment, and thus skip explicit alignment in a few cases as an optimization. It was requested on libc-alpha that MALLOC_ALIGNMENT should be used, although this results in wasted space if MALLOC_ALIGNMENT is larger than the fundamental alignment. (The dynamic linker cannot assume that the non-minimal malloc will provide an alignment of MALLOC_ALIGNMENT; the ABI provides _Alignof (max_align_t) only.)
* malloc: Run tests without calling mallopt [BZ #19469]Florian Weimer2016-08-021-0/+4
| | | | | | The compiled tests no longer refer to the mallopt symbol from their main functions. (Some tests still call mallopt explicitly, which is fine.)
* malloc: Preserve arena free list/thread count invariant [BZ #20370]Florian Weimer2016-08-021-5/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | It is necessary to preserve the invariant that if an arena is on the free list, it has thread attach count zero. Otherwise, when arena_thread_freeres sees the zero attach count, it will add it, and without the invariant, an arena could get pushed to the list twice, resulting in a cycle. One possible execution trace looks like this: Thread 1 examines free list and observes it as empty. Thread 2 exits and adds its arena to the free list, with attached_threads == 0). Thread 1 selects this arena in reused_arena (not from the free list). Thread 1 increments attached_threads and attaches itself. (The arena remains on the free list.) Thread 1 exits, decrements attached_threads, and adds the arena to the free list. The final step creates a cycle in the usual way (by overwriting the next_free member with the former list head, while there is another list item pointing to the arena structure). tst-malloc-thread-exit exhibits this issue, but it was only visible with a debugger because the incorrect fix in bug 19243 removed the assert from get_free_list.
* Bump up tst-malloc-thread-fail timeout from 20 to 30sChris Metcalf2016-07-051-1/+1
| | | | | | Right now tilegx is right on the verge of timeout when it runs, so adding a bit of headroom seems like the right thing; we see failures when running tests in parallel.
* test-skeleton.c: Add write_message functionFlorian Weimer2016-06-231-11/+3
|
* malloc: Avoid premature fallback to mmap [BZ #20284]Florian Weimer2016-06-211-6/+4
| | | | | | | Before this change, the while loop in reused_arena which avoids returning a corrupt arena would never execute its body if the selected arena were not corrupt. As a result, result == begin after the loop, and the function returns NULL, triggering fallback to mmap.
* Revert __malloc_initialize_hook symbol poisoningFlorian Weimer2016-06-205-23/+6
| | | | | | It turns out the Emacs-internal malloc implementation uses __malloc_* symbols. If glibc poisons them in <stdc-pre.h>, Emacs will no longer compile.
* malloc_usable_size: Use correct size for dumped fake mapped chunksFlorian Weimer2016-06-111-1/+6
| | | | | | The adjustment for the size computation in commit 1e8a8875d69e36d2890b223ffe8853a8ff0c9512 is needed in malloc_usable_size, too.
* malloc: Remove __malloc_initialize_hook from the API [BZ #19564]Florian Weimer2016-06-106-8/+60
| | | | | | | __malloc_initialize_hook is interposed by application code, so the usual approach to define a compatibility symbol does not work. This commit adds a new mechanism based on #pragma GCC poison in <stdc-predef.h>.
* malloc: Correct size computation in realloc for dumped fake mmapped chunksFlorian Weimer2016-06-081-4/+8
| | | | | | | | For regular mmapped chunks there are two size fields (hence a reduction by 2 * SIZE_SZ bytes), but for fake chunks, we only have one size field, so we need to subtract SIZE_SZ bytes. This was initially reported as Emacs bug 23726.
* malloc: Correct malloc alignment on 32-bit architectures [BZ #6527]Florian Weimer2016-05-242-15/+3
| | | | | | | | | | | | | | | | After the heap rewriting added in commit 4cf6c72fd2a482e7499c29162349810029632c3f (malloc: Rewrite dumped heap for compatibility in __malloc_set_state), we can change malloc alignment for new allocations because the alignment of old allocations no longer matters. We need to increase the malloc state version number, so that binaries containing dumped heaps of the new layout will not try to run on previous versions of glibc, resulting in obscure crashes. This commit addresses a failure of tst-malloc-thread-fail on the affected architectures (32-bit ppc and mips) because the test checks pointer alignment.
* tst-mallocfork2: Fix race condition, use fewer resourcesFlorian Weimer2016-05-131-1/+7
| | | | | | | | | | | | | | | | The first SIGUSR1 signal could arrive when sigusr1_sender_pid was still 0. As a result, kill would send SIGSTOP to the entire process group. This would cause the test to hang before printing any output. This commit also adds a sched_yield to the signal source, so that it does not flood the parent process with signals it has never a chance to handle. Even with these changes, tst-mallocfork2 still fails reliably after the fix in commit commit 56290d6e762c1194547e73ff0b948cd79d3a1e03 (Increase fork signal safety for single-threaded processes) is backed out.
* malloc: Rewrite dumped heap for compatibility in __malloc_set_stateFlorian Weimer2016-05-132-109/+96
| | | | | | | | | | | | | | | | | This will allow us to change many aspects of the malloc implementation while preserving compatibility with existing Emacs binaries. As a result, existing Emacs binaries will have a larger RSS, and Emacs needs a few more milliseconds to start. This overhead is specific to Emacs (and will go away once Emacs switches to its internal malloc). The new checks to make free and realloc compatible with the dumped heap are confined to the mmap paths, which are already quite slow due to the munmap overhead. This commit weakens some security checks, but only for heap pointers in the dumped main arena. By default, this area is empty, so those checks are as effective as before.
* Increase fork signal safety for single-threaded processes [BZ #19703]Florian Weimer2016-05-122-1/+214
| | | | | | | | | This provides a band-aid and addresses the scenario where fork is called from a signal handler while the process is in the malloc subsystem (or has acquired the libio list lock). It does not address the general issue of async-signal-safety of fork; multi-threaded processes are not covered, and some glibc subsystems have fork handlers which are not async-signal-safe.
* malloc: Adjust header file guard in malloc-internal.hFlorian Weimer2016-05-041-3/+3
|
* malloc: Add missing internal_function attributes on function definitionsFlorian Weimer2016-04-141-0/+3
| | | | Fixes build on i386 after commit 29d794863cd6e03115d3670707cc873a9965ba92.
* malloc: Remove malloc hooks from fork handlerFlorian Weimer2016-04-142-119/+8
| | | | | | | The fork handler now runs so late that there is no risk anymore that other fork handlers in the same thread use malloc, so it is no longer necessary to install malloc hooks which made a subset of malloc functionality available to the thread that called fork.
* malloc: Run fork handler as late as possible [BZ #19431]Florian Weimer2016-04-145-42/+272
| | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, a thread M invoking fork would acquire locks in this order: (M1) malloc arena locks (in the registered fork handler) (M2) libio list lock A thread F invoking flush (NULL) would acquire locks in this order: (F1) libio list lock (F2) individual _IO_FILE locks A thread G running getdelim would use this order: (G1) _IO_FILE lock (G2) malloc arena lock After executing (M1), (F1), (G1), none of the threads can make progress. This commit changes the fork lock order to: (M'1) libio list lock (M'2) malloc arena locks It explicitly encodes the lock order in the implementations of fork, and does not rely on the registration order, thus avoiding the deadlock.
* scratch_buffer_set_array_size: Include <limits.h>Florian Weimer2016-04-071-0/+1
| | | | It is needed for CHAR_BIT.
* Fix malloc threaded tests link on non-LinuxSamuel Thibault2016-03-221-6/+3
| | | | | | * malloc/Makefile ($(objpfx)tst-malloc-backtrace, $(objpfx)tst-malloc-thread-exit, $(objpfx)tst-malloc-thread-fail): Use $(shared-thread-library) instead of hardcoding the path to libpthread.
* Fix type of parameter passed by malloc_consolidateTulio Magno Quites Machado Filho2016-03-111-1/+1
| | | | atomic_exchange_acq() expected a pointer, but was receiving an integer.
* malloc: Remove NO_THREADSFlorian Weimer2016-02-192-5/+0
| | | | | No functional change. It was not possible to build without threading support before.
* malloc: Remove max_total_mem member form struct malloc_parFlorian Weimer2016-02-193-8/+4
| | | | | Also note that sumblks in struct mallinfo is always 0. No functional change.
* malloc: Remove arena_mem variableFlorian Weimer2016-02-192-8/+0
| | | | The computed value is never used. The accesses were data races.
* tst-malloc-thread-exit: Use fewer system resourcesFlorian Weimer2016-02-191-14/+15
|
* Make shebang interpreter directives consistentMarko Myllynen2016-01-071-1/+1
|
* Update copyright dates not handled by scripts/update-copyrights.Joseph Myers2016-01-043-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've updated copyright dates in glibc for 2016. This is the patch for the changes not generated by scripts/update-copyrights and subsequent build / regeneration of generated files. * NEWS: Update copyright dates. * catgets/gencat.c (print_version): Likewise. * csu/version.c (banner): Likewise. * debug/catchsegv.sh: Likewise. * debug/pcprofiledump.c (print_version): Likewise. * debug/xtrace.sh (do_version): Likewise. * elf/ldconfig.c (print_version): Likewise. * elf/ldd.bash.in: Likewise. * elf/pldd.c (print_version): Likewise. * elf/sotruss.sh: Likewise. * elf/sprof.c (print_version): Likewise. * iconv/iconv_prog.c (print_version): Likewise. * iconv/iconvconfig.c (print_version): Likewise. * locale/programs/locale.c (print_version): Likewise. * locale/programs/localedef.c (print_version): Likewise. * login/programs/pt_chown.c (print_version): Likewise. * malloc/memusage.sh (do_version): Likewise. * malloc/memusagestat.c (print_version): Likewise. * malloc/mtrace.pl: Likewise. * manual/libc.texinfo: Likewise. * nptl/version.c (banner): Likewise. * nscd/nscd.c (print_version): Likewise. * nss/getent.c (print_version): Likewise. * nss/makedb.c (print_version): Likewise. * posix/getconf.c (main): Likewise. * scripts/test-installation.pl: Likewise. * sysdeps/unix/sysv/linux/lddlibc4.c (main): Likewise.
* Update copyright dates with scripts/update-copyrights.Joseph Myers2016-01-0438-38/+38
|
* malloc: Test various special cases related to allocation failuresFlorian Weimer2015-12-292-1/+446
| | | | | | | | | | | | | | | | This test case exercises unusual code paths in allocation functions, related to allocation failures. Specifically, the test can reveal the following bugs: (a) calloc returns non-zero memory on fallback to sysmalloc. (b) calloc can self-deadlock because it fails to release the arena lock on certain allocation failures. (c) pvalloc can dereference a NULL arena pointer. (a) and (b) appear specific to a faulty downstream backport. (c) was fixed as part of commit 10ad46bc6526edc5c7afcc57112da96917ff3629. The test for (a) was inspired by a reproducer supplied by Jeff Layton.
View Lorry Controller Status