#2956·jemalloc

jemalloc large allocation becomes inaccessible shortly after malloc

Author: xqjcoolCreated Jul 10, 2026Updated Jul 30, 2026

Hi,

We are debugging a crash in a multi-threaded C daemon using jemalloc on Linux x86_64.

The symptom is that a large allocation returned by malloc appears to become inaccessible shortly afterward. The crash happens when reading the first field of the allocated object.

The object is roughly 6.3 MB:

c
#define BUCKET_COUNT 99767

struct bucket {
    void *next;
    pthread_rwlock_t lock;
};

struct table {
    int max_size;
    int node_count;
    int thread_safe;
    int (*compare)(void *, void *);

    struct bucket list;
    pthread_rwlock_t list_lock;

    struct bucket buckets[0];
};

struct table *table_create(void)
{
    size_t len = sizeof(struct table) + sizeof(struct bucket) * BUCKET_COUNT;

    struct table *t = malloc(len);
    if (!t)
        return NULL;

    memset(t, 0, len);

    t->max_size = BUCKET_COUNT;
    t->compare = compare_func;

    return t;
}

The returned pointer is stored in a per-thread/worker object:

worker->table = table_create();

Later, before the table is used, the program validates it:

if (!worker->table ||
    worker->table->max_size != BUCKET_COUNT ||
    worker->table->compare != compare_func) {
    ...
}

The process crashes while reading worker->table->max_size.

From the core:

worker = 0x7f06e7300000
&worker->table = 0x7f06e7336038
worker->table = 0x7f06e48ff000
si_addr = 0x7f06e48ff000

So the fault address is exactly equal to the malloc-returned pointer stored in worker->table.

This suggests the object itself is no longer mapped/readable. It does not look like a normal table/list traversal bug, because the crash happens before the table is used.

We have also seen several other crashes inside jemalloc, often around these paths:

ph_remove
je_edata_heap_remove
je_eset_remove
extent_coalesce
pac_decay_stashed
arena_decay_dirty
tcache_destroy
je_tsd_cleanup

Changing jemalloc settings seems to affect the frequency. For example:

MALLOC_CONF=abort:true,junk:true,tcache:false,narenas:1,background_thread:false

appears to reduce the frequency, but does not fully eliminate the issue.

Our current hypotheses are:

1. The large allocation is being freed or unmapped unexpectedly.
2. There is a double-free or invalid free.
3. Heap metadata is corrupted elsewhere, and jemalloc later unmaps/recycles the wrong extent.
4. A buffer overflow in another thread corrupts allocator metadata.
5. The crash is only a delayed symptom of an earlier heap corruption.

We plan to debug by recording the pointer returned by table_create() and then watching for free/munmap:

set $ptr = worker->table
set $end = (char *)$ptr + allocation_size

b free
commands
  silent
  if $rdi == $ptr
    printf "free table ptr=%p\n", $rdi
    bt 20
  end
  c
end

catch syscall munmap
commands
  silent
  if $rdi <= $ptr && $ptr < $rdi + $rsi
    printf "munmap table range addr=%p len=%lu ptr=%p\n", $rdi, $rsi, $ptr
    bt 20
  end
  c
end

watch -l worker->table
watch -l *(long *)$ptr

Questions:

- Does this failure pattern usually indicate use-after-free, invalid free, or heap metadata corruption?
- Are there jemalloc-specific debug options that help identify who freed/unmapped a large extent?
- Is the above GDB approach reasonable for catching the first bad operation?
- Are there better jemalloc options than disabling tcache/background_thread/narenas for this kind of investigation?