summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Remove dead stores detected by static analysisKevin Svetlitski2023-05-115-11/+6
| | | | | | | None of these are harmful, and they are almost certainly optimized away by the compiler. The motivation for fixing them anyway is that we'd like to enable static analysis as part of CI, and the first step towards that is resolving the warnings it produces at present.
* Fix possible `NULL` pointer dereference from `mallctl("prof.prefix", ...)`Kevin Svetlitski2023-05-111-0/+3
| | | | | | | | | | | | | | | | Static analysis flagged this issue. Here is a minimal program which causes a segfault within Jemalloc: ``` #include <jemalloc/jemalloc.h> const char *malloc_conf = "prof:true"; int main() { mallctl("prof.prefix", NULL, NULL, NULL, 0); } ``` Fixed by checking if `prefix` is `NULL`.
* Fix the prof thread_name reference in prof_recent dump.Qi Wang2023-05-111-2/+4
| | | | | | As pointed out in #2434, the thread_name in prof_tdata_t was changed in #2407. This also requires an update for the prof_recent dump, specifically the emitter expects a "char **" which is fixed in this commit.
* Add config detection for JEMALLOC_HAVE_PTHREAD_SET_NAME_NP.Qi Wang2023-05-111-1/+1
| | | | and use it on the background thread name setting.
* If ptr present check if alloc_ctx.edata == NULLauxten2023-05-101-1/+1
|
* Make arenas_lookup_ctl triableauxten2023-05-101-4/+6
|
* Fix possible `NULL` pointer dereference in `VERIFY_READ`Kevin Svetlitski2023-05-091-1/+3
| | | | | Static analysis flagged this. Fixed by simply checking `oldlenp` before dereferencing it.
* Fix segfault in `extent_try_coalesce_impl`Kevin Svetlitski2023-05-091-1/+3
| | | | | | | | Static analysis flagged this. `extent_record` was passing `NULL` as the value for `coalesced` to `extent_try_coalesce`, which in turn passes that argument to `extent_try_coalesce_impl`, where it is written to without checking if it is `NULL`. I can confirm from reviewing the fleetwide coredump data that this was in fact being hit in production.
* Make eligible functions `static`Kevin Svetlitski2023-05-083-3/+5
| | | | | | The codebase is already very disciplined in making any function which can be `static`, but there are a few that appear to have slipped through the cracks.
* Remove errant `assert` in `arena_extent_alloc_large`Kevin Svetlitski2023-05-011-1/+0
| | | | | | | This codepath may generate deferred work when the HPA is enabled. See also [@davidtgoldblatt's relevant comment on the PR which introduced this](https://github.com/jemalloc/jemalloc/pull/2107#discussion_r699770967) which prevented a similarly incorrect `assert` from being added elsewhere.
* Check for equality instead of assigning in asserts in hpa_from_pai.Eric Mueller2023-04-171-4/+4
| | | | | | | | | | It appears like a simple typo means we're unconditionally overwriting some fields in hpa_from_pai when asserts are enabled. From hpa_shard_init, it looks like these fields have these values anyway, so this shouldn't cause bugs, but if something is wrong it seems better to have these asserts in place. See issue #2412.
* Inline the storage for thread name in prof_tdata_t.Qi Wang2023-04-056-72/+45
| | | | | | | | | The previous approach managed the thread name in a separate buffer, which causes races because the thread name update (triggered by new samples) can happen at the same time as prof dumping (which reads the thread names) -- these two operations are under separate locks to avoid blocking each other. Implemented the thread name storage as part of the tdata struct, which resolves the lifetime issue and also avoids internal alloc / dalloc during prof_sample.
* Remove unused mutex from hpa_centralAmaury Séchet2023-03-101-5/+1
|
* Add a header in HPA stats for the nonfull slabs.Qi Wang2023-02-171-2/+3
|
* Add an explicit name to the dedicated oversize arena.Qi Wang2023-02-171-0/+5
|
* Fix thread_name updating for heap profiling.Qi Wang2023-02-151-11/+10
| | | | | | | | | The current thread name reading path updates the name every time, which requires both alloc and dalloc -- and the temporary NULL value in the middle causes races where the prof dump read path gets NULLed in the middle. Minimize the changes in this commit to isolate the bugfix testing; will also refactor the whole thread name paths later.
* Implement prof sample hooks "experimental.hooks.prof_sample(_free)".Qi Wang2022-12-073-1/+111
| | | | | | | | | | | The added hooks hooks.prof_sample and hooks.prof_sample_free are intended to allow advanced users to track additional information, to enable new ways of profiling on top of the jemalloc heap profile and sample features. The sample hook is invoked after the allocation and backtracing, and forwards the both the allocation and backtrace to the user hook; the sample_free hook happens before the actual deallocation, and forwards only the ptr and usz to the hook.
* Inline free and sdallocx into operator deleteGuangli Dai2022-11-212-218/+11
|
* Add a configure option --enable-force-getenv.Qi Wang2022-11-041-6/+10
| | | | | | Allows the use of getenv() rather than secure_getenv() to read MALLOC_CONF. This helps in situations where hosts are under full control, and setting MALLOC_CONF is needed while also setuid. Disabled by default.
* Enable fast thread locals for dealloc-only threads.Qi Wang2022-10-251-1/+17
| | | | | | | | | | Previously if a thread does only allocations, it stays on the slow path / minimal initialized state forever. However, dealloc-only is a valid pattern for dedicated reclamation threads -- this means thread cache is disabled (no batched flush) for them, which causes high overhead and contention. Added the condition to fully initialize TSD when a fair amount of dealloc activities are observed.
* fix build for non linux/BSD platforms.David Carlier2022-10-031-3/+3
|
* Add arena-level name.Guangli Dai2022-09-163-7/+94
| | | | An arena-level name can help identify manual arenas.
* Making jemalloc max stack depth a runtime optionGuangli Dai2022-09-126-7/+160
|
* error: implicit declaration of function 'pthread_create_fptr_init' is ↵Abael He2022-09-071-33/+33
| | | | | | | | | | | | | | | | | | | | invalid in C99 ./autogen.sh \ && ./configure --prefix=/usr/local --enable-static --enable-autogen --enable-xmalloc --with-static-libunwind=/usr/local/lib/libunwind.a --enable-lazy-lock --with-jemalloc-prefix='' \ && make -j16 ... gcc -std=gnu11 -Werror=unknown-warning-option -Wall -Wextra -Wshorten-64-to-32 -Wsign-compare -Wundef -Wno-format-zero-length -Wpointer-arith -Wno-missing-braces -Wno-missing-field-initializers -pipe -g3 -Wimplicit-fallthrough -O3 -funroll-loops -fPIC -DPIC -c -D_REENTRANT -Iinclude -Iinclude -DJEMALLOC_NO_PRIVATE_NAMESPACE -o src/edata_cache.sym.o src/edata_cache.c src/background_thread.c:768:6: error: implicit declaration of function 'pthread_create_fptr_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration] pthread_create_fptr_init()) { ^ src/background_thread.c:768:6: note: did you mean 'pthread_create_wrapper_init'? src/background_thread.c:34:1: note: 'pthread_create_wrapper_init' declared here pthread_create_wrapper_init(void) { ^ 1 error generated. make: *** [src/background_thread.sym.o] Error 1 make: *** Waiting for unfinished jobs....
* Refactor the remote / cross thread cache bin stats readingGuangli Dai2022-09-061-2/+1
| | | | Refactored cache_bin.h so that only one function is racy.
* Add double free detection in thread cache for debug buildIvan Zaitsev2022-08-043-1/+18
| | | | | | Add new runtime option `debug_double_free_max_scan` that specifies the max number of stack entries to scan in the cache bit when trying to detect the double free bug (currently debug build only).
* OpenBSD build fix. still no cpu affinity.David Carlier2022-07-191-0/+4
| | | | | - enabling pthread_get/pthread_set_name_np api. - disabling per thread cpu affinity handling, unsupported on this platform.
* Add the missing opt.cache_oblivious handling.Qi Wang2022-07-141-0/+1
|
* Fix possible "nmalloc >= ndalloc" assertionAzat Khuzhin2022-07-111-6/+9
| | | | | | | | | | | | | | | In arena_stats_merge() first nmalloc was read, and after ndalloc. However with this order, it is possible for some thread to incement ndalloc in between, and then nmalloc < ndalloc, and assertion will fail, like again found by ClickHouse CI [1] (even after #2234). [1]: https://github.com/ClickHouse/ClickHouse/issues/31531 Swap the order to avoid possible assertion. Cc: @interwq Follow-up for: #2234
* New configure option '--enable-pageid' for LinuxDavid Carlier2022-06-091-0/+28
| | | | | | | The option makes jemalloc use prctl with PR_SET_VMA to tag memory mappings with "jemalloc_pg" or "jemalloc_pg_overcommit". This allows to easily identify jemalloc's mappings in /proc/<pid>/maps. PR_SET_VMA is only available in Linux 5.17 and above.
* Implement pvalloc replacementAlex Lapenkou2022-05-181-0/+46
| | | | | Despite being an obsolete function, pvalloc is still present in GLIBC and should work correctly when jemalloc replaces libc allocator.
* Improve the failure message upon opt_experimental_infallible_new.Qi Wang2022-05-171-2/+8
|
* Make the default option of zero realloc match the system allocator.Qi Wang2022-05-051-1/+6
|
* Avoid abort() in test/integration/cpp/infallible_new_true.Qi Wang2022-04-252-2/+26
| | | | | Allow setting the safety check abort hook through mallctl, which avoids abort() and core dumps.
* fix some typoscuishuang2022-04-252-2/+2
| | | | Signed-off-by: cuishuang <imcusg@gmail.com>
* Rename zero_realloc option "strict" to "alloc".Qi Wang2022-04-201-6/+6
| | | | | With realloc(ptr, 0) being UB per C23, the option name "strict" makes less sense now. Rename to "alloc" which describes the behavior.
* Do not disable SEC by default for 64k pages platformsAlex Lapenkou2022-03-241-1/+1
| | | | | | Default SEC max_alloc option value was 32k, disabling SEC for platforms with lg-page=16. This change enables SEC for all platforms, making minimum max_alloc value equal to PAGE.
* Add comments and use meaningful vars in sz_psz2ind.Charles2022-03-241-1/+1
|
* Implement PAGE_FLOOR macroAlex Lapenkou2022-03-221-1/+1
|
* Fix size class calculation for secAlex Lapenkou2022-03-221-5/+8
| | | | | | Due to a bug in sec initialization, the number of cached size classes was equal to 198. The bug caused the creation of more than a hundred of unused bins, although it didn't affect the caching logic.
* Fix FreeBSD system jemalloc TSD cleanupAlex Lapenkov2022-03-021-5/+11
| | | | | | | | | | | | | | Before this commit, in case FreeBSD libc jemalloc was overridden by another jemalloc, proper thread shutdown callback was involved only for the overriding jemalloc. A call to _malloc_thread_cleanup from libthr would be redirected to user jemalloc, leaving data about dead threads hanging in system jemalloc. This change tackles the issue in two ways. First, for current and old system jemallocs, which we can not modify, the overriding jemalloc would locate and invoke system cleanup routine. For upcoming jemalloc integrations, the cleanup registering function will also be redirected to user jemalloc, which means that system jemalloc's cleanup routine will be registered in user's jemalloc and a single call to _malloc_thread_cleanup will be sufficient to invoke both callbacks.
* Fix possible "nmalloc >= ndalloc" assertion.Azat Khuzhin2022-03-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is possible that ndalloc will be updated before nmalloc, in arena_large_ralloc_stats_update(), fix this by reorder those calls. It was found by ClickHouse CI, that periodically hits this assertion [1]. [1]: https://github.com/ClickHouse/ClickHouse/issues/31531 That issue contains lots of examples, with core dump and some gdb output [2]. [2]: https://s3.amazonaws.com/clickhouse-test-reports/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/stress_test__debug__actions_.html Here you can find binaries for that particular report [3] you need clickhouse debug build [4]. [3]: https://s3.amazonaws.com/clickhouse-builds/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/clickhouse_build_check_(actions)/report.html [4]: https://s3.amazonaws.com/clickhouse-builds/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/package_debug/clickhouse Brief info from that report: 2 0x000000002ad6dbfe in arena_stats_merge (tsdn=0x7f2399abdd20, arena=0x7f241ce01080, nthreads=0x7f24e4360958, dss=0x7f24e4360960, dirty_decay_ms=0x7f24e4360968, muzzy_decay_ms=0x7f24e4360970, nactive=0x7f24e4360978, ndirty=0x7f24e43 e4360988, astats=0x7f24e4360998, bstats=0x7f24e4363310, lstats=0x7f24e4364990, estats=0x7f24e4366e50, hpastats=0x7f24e43693a0, secstats=0x7f24e436a020) at ../contrib/jemalloc/src/arena.c:138 ndalloc = 226 nflush = 0 curlextents = 0 nmalloc = 225 nrequests = 0 Here you can see that they differs only by 1. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Fix failed assertion due to racy memory accessAlex Lapenkou2022-02-171-2/+4
| | | | | | | | | | While calculating the number of stashed pointers, multiple variables potentially modified by a concurrent thread were used for the calculation. This led to some inconsistencies, correctly detected by the assertions. The change eliminates some possible inconsistencies by using unmodified variables and only once a concurrently modified one. The assertions are omitted for the cases where we acknowledge potential inconsistencies too.
* Add a dependency checking step at the end of malloc_conf_init.Qi Wang2022-01-261-20/+19
| | | | Currently only prof_leak_error and prof_final are checked.
* Initialize prof_leak during prof init.Qi Wang2022-01-262-1/+4
| | | | | Otherwise, prof_leak may get set after prof_leak_error, and disagree with each other.
* Add prof_leak_error optionyunxu2022-01-215-0/+35
| | | | | | The option makes the process to exit with error code 1 if a memory leak is detected. This is useful for implementing automated tools that rely on leak detection.
* Avoid calculating size of size class twice & delete sc_data_global.Charles2022-01-181-5/+2
|
* Simplify arena_migrate() to take arena_t* instead of indices.Qi Wang2022-01-112-5/+4
| | | | | This makes debugging slightly easier and avoids the confusion of "should we create new arenas" here.
* Fix the extent state checking on the merge error path.Qi Wang2022-01-111-3/+17
| | | | | | With DSS as primary, the default merge impl will (correctly) decline to merge when one of the extent is non-dss. The error path should tolerate the not-merged extent being in a merging state.
* Purge all if the last thread migrated away from an arena.Qi Wang2022-01-061-0/+6
|
View Lorry Controller Status