Shrink heap on linux when overcommit_memory == 2

Using madvise with MADV_DONTNEED to release memory back to the kernel
is not sufficient to change the commit charge accounted against the
process on Linux.  It is OK however, when overcommit is enabled or is
heuristic.  However, when overcommit is restricted to a percentage of
memory setting the contents of /proc/sys/vm/overcommit_memory as 2, it
makes a difference since memory requests will fail.  Hence, we do what
we do with secure exec binaries, which is to call mmap on the region
to be dropped with MAP_FIXED. This internally unmaps the pages in
question and reduces the amount of memory accounted against the
process.

1 files changed, 25 insertions, 0 deletions

diff --git a/sysdeps/generic/malloc-sysdep.h b/sysdeps/generic/malloc-sysdep.h
new file mode 100644
index 0000000000..bbc48c04f0
--- /dev/null
+++ b/sysdeps/generic/malloc-sysdep.h
@@ -0,0 +1,25 @@
+/* System-specific malloc support functions.  Generic version.
+   Copyright (C) 2012 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+/* Force an unmap when the heap shrinks in a secure exec.  This ensures that
+   the old data pages immediately cease to be accessible.  */
+static inline bool
+check_may_shrink_heap (void)
+{
+  return __libc_enable_secure;
+}