Optional pure-Rust allocator for the worker binaries

The value of integrating

Chroma's worker binaries run jemalloc everywhere except MSVC — rust/worker/src/bin/fn_consumer.rs gates the #[global_allocator] on #[cfg(not(target_env = "msvc"))], with tikv-jemallocator = { version = "0.6.0", features = ["profiling"] }. So the choice is already made: the platform allocator was not the right answer for this workload.

The workload is why. A worker holds index segments in memory, runs tantivy = "0.22.0" for full-text alongside vector search, spreads work with rayon = "1.10.0" and dashmap, and serves it over tonic = "0.14" and axum = "0.8" on a multi-threaded runtime — so a query's buffers are routinely allocated on one thread and freed on another, continuously, for the life of the process. That is a live set that never drains and a cross-thread free path that is always hot.

rusty_alloc is a pure-Rust allocator with 8,851 downloads on crates.io and no build script, which also removes the MSVC exception. Against jemalloc the rusty_alloc README's callgrind arms measure 0.84x, 0.89x and 0.98x whole-program instructions on lua, perl and sqlite under LD_PRELOAD; on a workload with a real live set it measures 0.92x mimalloc's instructions (liveset — 65,536 live objects, a random victim replaced each step).

Not asking you to drop jemalloc — this would sit behind an off-by-default feature.

How to integrate

[features]
rusty-alloc = ["dep:rusty_alloc-api"]   # opt in; nothing changes by default

[dependencies]
rusty_alloc-api = { version = "2.1", optional = true }

#[cfg(feature = "rusty-alloc")]
#[global_allocator]
static GLOBAL: rusty_alloc_api::RustyAlloc = rusty_alloc_api::RustyAlloc;

Nothing else changes: same collections, same distance functions, same client APIs. You would lose jemalloc's heap profiling in that arm, which is a good reason to keep the existing path as the default.

Remade With Rust & MATA

We are rebuilding the C libraries Rust projects still depend on, in safe Rust — allocators, codecs, compression, parsers, identity and storage — so a build can drop its C pieces without losing performance. The org is at https://github.com/Remade-With-Rust if anything else fits. If this is not for you, please close the issue.

• x.com/farmer_timmm