Make malloc(3) superpage aware. Specifically, if getpagesizes(3) returns

a large page size that is greater than malloc(3)'s default chunk size but
less than or equal to 4 MB, then increase the chunk size to match the large
page size.

Most often, using a chunk size that is less than the large page size is not
a problem.  However, consider a long-running application that allocates and
frees significant amounts of memory.  In particular, it frees enough memory
at times that some of that memory is munmap()ed.  Up until the first
munmap(), a 1MB chunk size is just fine; it's not a problem for the virtual
memory system.  Two adjacent 1MB chunks that are aligned on a 2MB boundary
will be promoted automatically to a superpage even though they were
allocated at different times.  The trouble begins with the munmap(),
releasing a 1MB chunk will trigger the demotion of the containing superpage,
leaving behind a half-used 2MB reservation.  Now comes the real problem.
Unfortunately, when the application needs to allocate more memory, and it
recycles the previously munmap()ed address range, the implementation of
mmap() won't be able to reuse the reservation.  Basically, the coalescing
rules in the virtual memory system don't allow this new range to combine
with its neighbor.  The effect being that superpage promotion will not
reoccur for this range of addresses until both 1MB chunks are freed at some
point in the future.

Reviewed by:	jasone
MFC after:	3 weeks

2 files changed, 19 insertions, 2 deletions

diff --git a/lib/libc/stdlib/malloc.3 b/lib/libc/stdlib/malloc.3
index a621a24..308ba7b 100644
--- a/lib/libc/stdlib/malloc.3
+++ b/lib/libc/stdlib/malloc.3
@@ -32,7 +32,7 @@
 .\"     @(#)malloc.3	8.1 (Berkeley) 6/4/93
 .\" $FreeBSD$
 .\"
-.Dd August 26, 2008
+.Dd September 26, 2009
 .Dt MALLOC 3
 .Os
 .Sh NAME
@@ -245,7 +245,8 @@ will be initialized to 0x5a.
 This is intended for debugging and will impact performance negatively.
 .It K
 Double/halve the virtual memory chunk size.
-The default chunk size is 1 MB.
+The default chunk size is the maximum of 1 MB and the largest
+page size that is less than or equal to 4 MB.
 .It M
 Use
 .Xr mmap 2
@@ -561,6 +562,7 @@ _malloc_options = "X";
 .Xr alloca 3 ,
 .Xr atexit 3 ,
 .Xr getpagesize 3 ,
+.Xr getpagesizes 3 ,
 .Xr memory 3 ,
 .Xr posix_memalign 3
 .Sh STANDARDS
diff --git a/lib/libc/stdlib/malloc.c b/lib/libc/stdlib/malloc.c
index b56b003..bdc26b6 100644
--- a/lib/libc/stdlib/malloc.c
+++ b/lib/libc/stdlib/malloc.c
@@ -4795,6 +4795,21 @@ malloc_init_hard(void)
 		}
 	}
 
+	/*
+	 * Increase the chunk size to the largest page size that is greater
+	 * than the default chunk size and less than or equal to 4MB.
+	 */
+	{
+		size_t pagesizes[MAXPAGESIZES];
+		int k, nsizes;
+
+		nsizes = getpagesizes(pagesizes, MAXPAGESIZES);
+		for (k = 0; k < nsizes; k++)
+			if (pagesizes[k] <= (1LU << 22))
+				while ((1LU << opt_chunk_2pow) < pagesizes[k])
+					opt_chunk_2pow++;
+	}
+
 	for (i = 0; i < 3; i++) {
 		unsigned j;