Capability narrowing uses a lookup table from an address to the internally used wide capability. Keep the narrow capability in the table instead of just the address and check it. This allows free and realloc to check their input and only accept capabilities exactly matching what was returned by malloc. When a user adds restrictions on top of malloc returned capabilities (e.g. narrower bounds), realloc could bypass those restrictions when it simply looked up the internal wide capability for the address.
Internally obstack used a backward word at a time copy. This can be an aliasing violation and likely slower than memcpy on most targets. The code is a badly written memcpy with performance issues such as the h->object_base indirection cannot be hoisted out of the loop. On CHERI architecture this is needed for copying pointers correctly.
New code in realloc that handles when new allocation is needed for alignment reasons (for capability narrowing) used the wrong size in memcpy (size was off by SIZE_SZ unless memory tagging was enabled) and used wrongly tagged pointer for untagging the old memory. Due to this bug realloc sometimes failed to copy tail bytes of an old allocation to the new allocation when capability narrowing is enabled.
Capability narrowing requires extra alignment for large allocations, this can cause significant heap fragmentation so ensure large allocations use a single mmap and unmaped on free. The fragmentation is bug 14581. This patch disables dynamic mmap threshold when cap_narrowing is enabled and uses a threshold of 256 kbytes on morello (which means at most 64 byte alignment requirement on the heap). The dynamic threshold could quickly grow very big and stay there making the fragmentation issue common.
Morello linux does not support brk syscall, so fail early in the malloc implementation instead of trying to use the syscall which always fails.
The obstack version should have been bumped after commit 2fd4de4b15a66f821057af90714145d2c034a609 [BZ #321] which changed a ptrdiff_t struct member into a union of ptrdiff_t and a pointer and thus changed the ABI on targets where ptrdiff_t and pointer have different size or alignment. This affects CHERI targets. Old versions of obstack are used in the wild and conflict with the one provided by glibc so at least on CHERI targets it has to be bumped. On other targets we don't bump the version as it changes compile time behaviour (the ABI remains backward compatible either way).
malloc/tst-malloc-backtrace tests heap corruption. malloc/tst-dynarray uses malloc_debug wrappers that access internals.
Add more cap_ hooks to implement narrowing without depending on a global capability covering the heap. Either recording every narrowed capability in a lookup table or recording every mapping used for the heap are supported. The morello implmentation uses a lookup table for now. The lookup table adds memory overhead, failure paths and locks. Recording and removing entries from the lookup table must be done carefully in realloc so on failure the old pointer is usable and on success the old pointer is immediately reusable concurrently. The locks require fork hooks so malloc works in multi-threaded fork child.
Public interfaces return pointers with narrow bounds, this internally requires bumping the size and alignment requirement of allocations so the bounds are representible. When pointers with narrow bounds need to be turned back to have wide bounds (free, realloc), the pointer is rederived from DDC. (So this patch relies on DDC to cover all heap memory with RW permission.) Allocations above the mmap threshold waste memory for alignment and realloc often falls back to the inefficient alloc, copy, free sequence instead of mremap or other inplace solution.