低延迟的 NUMA 感知的分支-并行线程池,在热路上不需要分配内存、系统调用、CAS 或虚假共享,适用于 C、C++、Rust 和 Zig
低延迟的 NUMA 感知的分支-并行线程池,在热路上不需要分配内存、系统调用、CAS 或虚假共享,适用于 C、C++、Rust 和 Zig
ForkUnion is a NUMA-aware fork-join thread-pool for C++, C, Rust, and Zig — built for the tightest #pragma omp parallel for-style loops, not task queues.
It already powers NUMA-sharded vector search with USearch, LLM KV-cache and attention kernels with NumKong, and unbalanced bioinformatics workloads — thousands of combinatorial tasks per core — with StringZilla.
On the hot path it makes zero heap allocations, zero system calls, zero CAS operations, and suffers no false-sharing of cache-lines.
So dispatch latency stays flat into the hundreds of cores, precisely where task-queue runtimes like Rayon collapse and even OpenMP begins to slip.
It is also unique in parking idle workers on a hardware address monitor — x86 UMONITOR/UMWAIT, Arm WFET, RISC-V Zawrs — so a worker light-sleeps on the exact cache-line it is waiting for and the silicon wakes it the instant another core writes there, with no hot spinning and no kernel futex.
That bet compounds as sockets multiply: Intel Xeon Platinum and NVIDIA Vera pack up to 8 sockets per node, where shared CAS thrashes the interconnect and private fetch_add cursors keep synchronization off it.
One parallel-for dispatch: ForkUnion is often ~3–6× faster than OpenMP and ~12–16× faster than Rayon and Taskflow — the fork-join tax, paid once per loop, cut by an order of magnitude. Full tables ↓
It is exhaustively tested for boundary-condition scheduling with miniaturized uint8_t indices, even runs on your big-endian 32-bit IBM mainframe, and ships with no_std and Miri coverage.
The core is a C++ 17 library; the C 99, Rust, and Zig APIs bind it, and all four can pin threads to NUMA nodes or individual cores and allocate node-local memory.
Despite being far more deeply tied to hardware and the OS than most alternatives, ForkUnion runs on six operating systems — Linux, FreeBSD, Windows, macOS, Android, and iOS — including asymmetric compute and memory topologies.
Topology harvesting and thread placement work on all six; NUMA-local memory placement is implemented on Linux, FreeBSD, and Windows, and elsewhere allocation falls back to a single memory domain.
ForkUnion is dead-simple to use!
There is no nested parallelism, exception handling, or "future promises"; they are banned.
The thread pool itself has a few core operations:
try_spawn to initialize worker threads, andfor_threads to launch a blocking callback on all threads.Higher-level APIs for index-addressable tasks are also available:
for_n - for individual evenly-sized tasks,for_n_dynamic - for individual unevenly-sized tasks,for_slices - for slices of evenly-sized tasks.For additional flow control and tuning, following helpers are available:
sleep(microseconds) - for longer naps,terminate - to kill the threads before the destructor is called,unsafe_for_threads - to broadcast a callback without blocking, returning a generation token,unsafe_join - to block until the completion of a broadcasted generation,is_complete - to poll a generation token for completion without blocking.Every dispatch is identified by an always-odd generation token.
On caller_exclusive_k pools you can dispatch, overlap your own work, poll is_complete, and join - the classic poll-then-join pattern.
On caller_inclusive_k pools the calling thread owes one slice of the work, which only runs inside unsafe_join, so completion can't be reached by polling alone.
The same rule shapes the RAII guard returned by for_threads and the for_n family: on exclusive pools the work starts at the guard's construction, while on inclusive pools it runs at join or destruction.
On Linux, in C++, given the maturity and flexibility of the HPC ecosystem, it provides NUMA extensions.
That includes the colocated_pool analog of the flat_pool and the linux_numa_allocator for allocating memory in a specific memory domain.
Those are out-of-the-box compatible with the higher-level APIs.
Most interestingly, for Big Data applications, a higher-level distributed_pool class will address and balance the work across all compute domains.
To integrate into your Rust project, add the following lines to Cargo.toml:
[dependencies]
forkunion = "3.0.3" # detect what the platform offers
forkunion = { version = "3.0.3", features = ["portable"] } # STL thread pool only
forkunion = { version = "3.0.3", features = ["place-memory-on-domain"] } # require NUMA-aware allocationsOr for the preview development version:
[dependencies]
forkunion = { git = "https://github.com/ashvardanian/ForkUnion.git", branch = "main-dev" }A minimal example may look like this:
use forkunion as fu;
let topology = fu::Topology::new().expect("Failed to detect hardware topology");
let mut pool = fu::spawn(&topology, 2);
pool.for_threads(&|thread_index, compute_domain_index| {
println!("Hello from thread # {} on compute domain # {}", thread_index + 1, compute_domain_index + 1);
});Higher-level APIs distribute index-addressable tasks across the threads in the pool:
pool.for_n(100, |prong| {
println!("Running task {} on thread # {}",
prong.task_index + 1, prong.thread_index + 1);
});
pool.for_slices(100, |prong, count| {
println!("Running slice [{}, {}) on thread # {}",
prong.task_index, prong.task_index + count, prong.thread_index + 1);
});
pool.for_n_dynamic(100, |prong| {
println!("Running task {} on thread # {}",
prong.task_index + 1, prong.thread_index + 1);
});A more realistic example with named threads and error handling may look like this:
use std::error::Error;
use forkunion as fu;
fn heavy_math(_: usize) {}
fn main() -> Result> {
let topology = fu::Topology::new()?;
let mut pool = fu::ThreadPool::try_named_spawn(&topology, "heavy-math", 4)?;
pool.for_n_dynamic(400, |prong| {
heavy_math(prong.task_index);
});
Ok(())
}The Topology handle reports the hardware topology, to size and place work:
let topology = fu::Topology::new()?;
for domain in 0..topology.compute_domains_count() {
let domain = fu::ComputeDomain(domain);
println!("domain {}: {} cores, level {}, allocate on memory domain {}",
domain.get(), topology.logical_cores_count_in(domain),
topology.compute_level_in(domain), topology.local_memory_of(domain).get());
}For advanced usage, refer to the NUMA section below.
For convenience Rayon-style parallel iterators pull the prelude module and check out related examples.
To integrate into your C++ project, either copy the include/ directory into your project, add a Git submodule, or CMake.
The forkunion.hpp umbrella is a list of #includes; the implementation lives beside it in include/forkunion/, one header per concern, so a platform backend can be read or replaced on its own.
For a Git submodule, run:
git submodule add https://github.com/ashvardanian/ForkUnion.git extern/forkunionAlternatively, using CMake:
FetchContent_Declare(
forkunion
GIT_REPOSITORY https://github.com/ashvardanian/ForkUnion
GIT_TAG v3.0.3
)
FetchContent_MakeAvailable(forkunion)
target_link_libraries(your_target PRIVATE forkunion::header)Then, include the header in your C++ code:
…The fu_-prefixed functions report the hardware topology, to size and place work:
…GCC supports nested functions that can capture variables from the enclosing scope:
…Compile: gcc -std=c11 test.c -lforkunion_static -lstdc++ -lpthread
The archive is C++ behind a C facade, so a C consumer names the C++ runtime itself — -lc++ under Apple's toolchain.
Clang provides blocks with ^{} syntax:
…Compile: clang -std=c11 -fblocks test.c -lforkunion_static -lc++ -lpthread -lBlocksRuntime
Most "thread-pools" are not thread-pools at all, but task-queues - think a std::queue> behind a std::mutex, executing each heap-allocated task on whatever core the OS scheduler picks.
That shape is what most alternatives are built around, and it is the wrong shape for a tight parallel for:
taskflow/taskflow, progschj/ThreadPool, bshoshany/thread-poolvit-vit/CTPL, mtrebi/thread-pooltokio-rs/tokio, rayon-rs/rayon, smol-rs/smolstd.Thread.PoolReach for ForkUnion when you have data-parallel loops, bulk-synchronous phases, or NUMA-sharded scans, and you want them to dispatch in nanoseconds with neither an allocator nor the kernel on the hot path. Reach for something else when you need task graphs, async I/O, futures and promises, work that outlives its scope, or nested parallelism — Taskflow, Tokio, and oneTBB are built for exactly that, and ForkUnion deliberately bans it.
The trade-offs line up like this, verified against each project's own source and documentation:
| Property | ForkUnion | OpenMP | Rayon | Taskflow | oneTBB |
|---|---|---|---|---|---|
| Hot-path heap allocations | None | ≈ None ¹ | Per spawned task ² | Per task node | Per task, pooled |
| Syscalls to dispatch | None | Spin, then futex | Spin/yield, then futex | Spin, then futex | Spin, then futex |
| Dynamic-schedule primitive | fetch_add cursors |
CAS retry loop ³ | Chase-Lev CAS deque ⁴ | Chase-Lev CAS deque ⁴ | Chase-Lev CAS deque ⁴ |
| NUMA pinning & local allocators | Yes | Yes ⁵ | No | No ⁶ | Partial ⁷ |
| Hardware timed-wait | Yes | x86 only, opt-in ⁸ | No | No | No |
| Languages | C++ · C · Rust · Zig | C · C++ · Fortran | Rust | C++ | C++ |
| Design center | Fork-join loops | Loops + tasks | Stealing tasks + iterators | Task-graph DAG | Stealing tasks |
¹ OpenMP reuses a persistent thread team, so a repeated
parallel forover a fixed-size team does not re-create it; the per-region worksharing descriptor is reused but not provably allocation-free on every path. ² Rayon keepsjoinand parallel-iterator jobs on the stack, butscope/spawnheap-allocate one job per task, and the Chase-Lev deque grows on the heap under deep nesting. ³ libgomp and libomp hand outschedule(dynamic)iterations with
暂无开放 Issues,或尚未同步最近议题。