rpmalloc

rpmalloc - General Purpose Memory Allocator

This library provides a public domain cross platform lock free thread caching 16-byte aligned memory allocator implemented in C. The latest source code is always available at https://github.com/mjansson/rpmalloc

Created by Mattias Jansson (@maniccoder) - Discord server for discussions at https://discord.gg/M8BwTQrt6c

Platforms currently supported:

• Windows
• Linux
• MacOS
• iOS
• Android

The code should be easily portable to any platform with atomic operations and an mmap-style virtual memory management API. The API used to map/unmap memory pages can be configured in runtime to a custom implementation, as can the memory page size.

This library is put in the public domain; you can redistribute it and/or modify it without any restrictions. Or, if you choose, you can use it under the MIT license. The license is declared as the SPDX expression Unlicense OR MIT.

Performance

We believe rpmalloc is faster than most popular memory allocators like tcmalloc, hoard, ptmalloc3 and others without causing extra allocated memory overhead in the thread caches compared to these allocators. We also believe the implementation to be easier to read and modify compared to these allocators, as it is a single source file of ~3300 lines of C code. All allocations have a natural 16-byte alignment.

The chart below shows throughput on the rptest benchmark, allocating randomly sized blocks in the [16, 8000] bytes range with a linear falloff distribution and cross-thread frees, as the number of threads scales from 1 to 16. Higher is better.

Peak resident memory for the same benchmark, lower is better:

The numbers were collected on a 13th Gen Intel Core i7-13800H (14 cores / 20 threads) running Ubuntu 26.04, comparing rpmalloc against the current releases of several popular allocators. The actual numbers are not to be interpreted as absolute performance figures, but rather as relative comparisons between the different allocators. For a full allocator comparison across the mimalloc-bench benchmark suite, including peak memory usage, see the BENCHMARKS file. The captured results and the script that produces these graphs live in the benchmark directory.

Required functions

The allocator is initialized automatically with default configuration on the first allocation request, so no explicit initialization is required. If you want to provide a custom memory interface or configuration you MUST call the initialization function, either rpmalloc_initialize or rpmalloc_initialize_config, before any other call into the allocator.

Before terminating your use of the allocator, you SHOULD call rpmalloc_finalize in order to release caches and unmap virtual memory, as well as prepare the allocator for global scope cleanup at process exit or dynamic library unload depending on your use case.

Using

The easiest way to use the library is simply adding rpmalloc.[h|c] to your project and compile them along with your sources. The allocator is completely self contained, you are not required to call the init/fini functions from your own code, but can do so in order to initialize and finalize the allocator in specific places or provide your own memory interface and/or configuration:

rpmalloc_initialize : Call at process start to initialize the allocator, optionally passing a custom memory interface with functions to map, commit, decommit and unmap memory pages (pass null to use the default OS backed implementation)

rpmalloc_initialize_config : Optional entry point to call at process start to initialize the allocator with a custom memory interface and/or a configuration of memory page size, huge page or transparent huge page usage, decommit behaviour, page naming and unmap-on-finalize behaviour

rpmalloc_finalize: Call at process exit to finalize the allocator

rpmalloc_thread_initialize: Call at each thread start to initialize the thread local data for the allocator (done automatically on the first allocation in a thread)

rpmalloc_thread_finalize: Call at each thread exit to finalize and release the thread heap for reuse by other threads (done automatically at thread exit on platforms supporting thread destructors)

rpmalloc_config: Get the current runtime configuration of the allocator

Then simply use the rpmalloc/rpfree and the other malloc style replacement functions. Remember all allocations are 16-byte aligned, so no need to call the explicit rpmemalign/rpaligned_alloc/rpposix_memalign functions unless you need greater alignment, they are simply wrappers to make it easier to replace in existing code.

If you wish to override the standard library malloc family of functions and have automatic initialization/finalization of process and threads, define ENABLE_OVERRIDE to non-zero (default is 1) which will include the malloc.c file in compilation of rpmalloc.c, and then rebuild the library or your project where you added the rpmalloc source. If you compile rpmalloc as a separate static library and use only the overridden libc entry points (you call plain malloc/free but never any rp* function directly), the linker may not pull the rpmalloc object out of the archive, and the overrides will silently not take effect. To force its inclusion you must reference at least one symbol from the library: the easiest way is to include rpmalloc.h in at least one source file and call rpmalloc_linker_reference somewhere - it's a dummy empty function provided for exactly this purpose. This is not needed if you call rpmalloc/rpfree (or any other rp* function) directly, link the dynamic library, or use LD_PRELOAD/DYLD_INSERT_LIBRARIES. For C++ operator overrides the behaviour is platform specific. On Windows you must #include <rpnew.h> in exactly one source file (it defines the new/delete operators with external linkage and has no include guard, so including it in more than one translation unit causes duplicate symbol errors). On other platforms the new/delete operators are overridden automatically by the library - do not include rpnew.h there, as it will conflict with the operators already provided. The list of libc entry points replaced may not be complete, use libc/stdc++ replacement only as a convenience for testing the library on an existing code base, not a final solution.

For explicit first class heaps, see the rpmalloc_heap*_ API under first class heaps section, requiring RPMALLOC_FIRST_CLASS_HEAPS to be defined to 1 - default is 0, as it imposes a very slight performance hit in deallocation path from an extra conditinal instruction.

Building

To compile as a static library run the configure python script which generates a Ninja build script, then build using ninja. The ninja build produces both a static and a dynamic library named rpmalloc.

By default the dynamic library can be used with LD_PRELOAD/DYLD_INSERT_LIBRARIES to inject in a preexisting binary, replacing any malloc/free family of function calls (when ENABLE_OVERRIDE is defined to 1). This is only implemented for Linux and macOS targets. The list of libc entry points replaced may not be complete, use preloading as a convenience for testing the library on an existing binary, not a final solution.

The latest stable release is available in the main branch. For latest development code, use the develop branch.

Configuration options

Detailed statistics are available if ENABLE_STATISTICS is defined to 1 (default is 0, or disabled), either on compile command line or by setting the value in rpmalloc.c. This will cause a slight overhead in runtime to collect statistics for memory page and huge block operations.

Detection of allocations still outstanding at rpmalloc_finalize is enabled if ENABLE_LEAK_DETECTION is defined to 1. It requires ENABLE_STATISTICS for the allocation counters and follows it by default, except when building with ENABLE_OVERRIDE where it defaults to 0: with the standard library override rpmalloc is the backing store for the C runtime's own allocations (such as per-thread TLS), which are still live at finalize and indistinguishable from application leaks. When ENABLE_ASSERTS is also enabled a detected leak triggers an assert.

Integer safety checks on all calls are enabled if ENABLE_VALIDATE_ARGS is defined to 1 (default is 0, or disabled), either on compile command line or by setting the value in rpmalloc.c. If enabled, size arguments to the global entry points are verified not to cause integer overflows in calculations.

Asserts are enabled if ENABLE_ASSERTS is defined to 1 (default is 0, or disabled), either on compile command line or by setting the value in rpmalloc.c.

To include malloc.c in compilation and provide overrides of standard library malloc entry points define ENABLE_OVERRIDE to 1 (this is the default).

To enable support for first class heaps, define RPMALLOC_FIRST_CLASS_HEAPS to 1 (default is 0, or disabled).

Huge pages

The allocator has support for huge/large pages on Windows, Linux and MacOS. To enable it, pass a non-zero value in the config value enable_huge_pages when initializing the allocator with rpmalloc_initialize_config. Explicit huge pages require system configuration (a preallocated huge page pool on Linux, the SeLockMemoryPrivilege on Windows). At initialization rpmalloc checks whether huge pages are actually available (on Linux by probing a real huge page mapping, not just reading the configured huge page size). If huge pages were requested but are not available on the system, rpmalloc_initialize/rpmalloc_initialize_config returns a non-zero value and leaves the allocator uninitialized (the effective enable_huge_pages is cleared), so you can detect this and re-initialize without huge pages if desired. On Linux, if an individual huge page backed mapping cannot later be served from the pool the allocator falls back to a normal mapping promoted to transparent huge pages. On Windows, large pages cannot be decommitted, so unused memory ranges are kept committed until unmapped when huge pages are enabled - on Linux unused memory ranges are decommitted if the kernel supports it (5.18 or later for huge page backed mappings).

On Linux and Android the allocator can alternatively use transparent huge pages without requiring any system configuration or preallocated huge page pool. Pass a non-zero value in the config value enable_thp and the allocator will advise the kernel to back the memory mappings with transparent huge pages using madvise(MADV_HUGEPAGE). Unlike enable_huge_pages this does not affect page size or commit/decommit behaviour. The two modes are mutually exclusive, with enable_huge_pages taking precedence.

Quick overview

The allocator uses separate heaps for each thread and partitions memory blocks according to a preconfigured set of size classes, up to 2MiB. Huge blocks above this limit are mapped and unmapped directly. Blocks are allocated from a page of multiple blocks, all of the same size class. Each page is one of four page types, small, medium-small, medium-large or large, depending on the size class. Each page belongs to an even larger span of pages, each of the same page type.

Implementation details

The allocator is based on a fixed span alignment of 256MiB, a fixed page alignment per page type, and 16 byte block alignment within the page. The span alignment is achieved by oversizing the memory mapping and aligning the returned virtual memory address to the required boundary (on POSIX systems the excess address range is immediately unmapped). By aligning to fixed sizes th