百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
F

ForkUnion

> 编程语言
开源

低延迟的 NUMA 感知的分支-并行线程池,在热路上不需要分配内存、系统调用、CAS 或虚假共享,适用于 C、C++、Rust 和 Zig

366 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

低延迟的 NUMA 感知的分支-并行线程池,在热路上不需要分配内存、系统调用、CAS 或虚假共享,适用于 C、C++、Rust 和 Zig

ForkUnion

ForkUnion is a NUMA-aware fork-join thread-pool for C++, C, Rust, and Zig — built for the tightest #pragma omp parallel for-style loops, not task queues. It already powers NUMA-sharded vector search with USearch, LLM KV-cache and attention kernels with NumKong, and unbalanced bioinformatics workloads — thousands of combinatorial tasks per core — with StringZilla.

On the hot path it makes zero heap allocations, zero system calls, zero CAS operations, and suffers no false-sharing of cache-lines. So dispatch latency stays flat into the hundreds of cores, precisely where task-queue runtimes like Rayon collapse and even OpenMP begins to slip. It is also unique in parking idle workers on a hardware address monitor — x86 UMONITOR/UMWAIT, Arm WFET, RISC-V Zawrs — so a worker light-sleeps on the exact cache-line it is waiting for and the silicon wakes it the instant another core writes there, with no hot spinning and no kernel futex. That bet compounds as sockets multiply: Intel Xeon Platinum and NVIDIA Vera pack up to 8 sockets per node, where shared CAS thrashes the interconnect and private fetch_add cursors keep synchronization off it.

One parallel-for dispatch: ForkUnion is often ~3–6× faster than OpenMP and ~12–16× faster than Rayon and Taskflow — the fork-join tax, paid once per loop, cut by an order of magnitude. Full tables ↓

It is exhaustively tested for boundary-condition scheduling with miniaturized uint8_t indices, even runs on your big-endian 32-bit IBM mainframe, and ships with no_std and Miri coverage. The core is a C++ 17 library; the C 99, Rust, and Zig APIs bind it, and all four can pin threads to NUMA nodes or individual cores and allocate node-local memory. Despite being far more deeply tied to hardware and the OS than most alternatives, ForkUnion runs on six operating systems — Linux, FreeBSD, Windows, macOS, Android, and iOS — including asymmetric compute and memory topologies. Topology harvesting and thread placement work on all six; NUMA-local memory placement is implemented on Linux, FreeBSD, and Windows, and elsewhere allocation falls back to a single memory domain.

Basic Usage

ForkUnion is dead-simple to use! There is no nested parallelism, exception handling, or "future promises"; they are banned. The thread pool itself has a few core operations:

  • try_spawn to initialize worker threads, and
  • for_threads to launch a blocking callback on all threads.

Higher-level APIs for index-addressable tasks are also available:

  • for_n - for individual evenly-sized tasks,
  • for_n_dynamic - for individual unevenly-sized tasks,
  • for_slices - for slices of evenly-sized tasks.

For additional flow control and tuning, following helpers are available:

  • sleep(microseconds) - for longer naps,
  • terminate - to kill the threads before the destructor is called,
  • unsafe_for_threads - to broadcast a callback without blocking, returning a generation token,
  • unsafe_join - to block until the completion of a broadcasted generation,
  • is_complete - to poll a generation token for completion without blocking.

Every dispatch is identified by an always-odd generation token. On caller_exclusive_k pools you can dispatch, overlap your own work, poll is_complete, and join - the classic poll-then-join pattern. On caller_inclusive_k pools the calling thread owes one slice of the work, which only runs inside unsafe_join, so completion can't be reached by polling alone. The same rule shapes the RAII guard returned by for_threads and the for_n family: on exclusive pools the work starts at the guard's construction, while on inclusive pools it runs at join or destruction.

On Linux, in C++, given the maturity and flexibility of the HPC ecosystem, it provides NUMA extensions. That includes the colocated_pool analog of the flat_pool and the linux_numa_allocator for allocating memory in a specific memory domain. Those are out-of-the-box compatible with the higher-level APIs. Most interestingly, for Big Data applications, a higher-level distributed_pool class will address and balance the work across all compute domains.

Intro in Rust

To integrate into your Rust project, add the following lines to Cargo.toml:

toml
[dependencies]
forkunion = "3.0.3"                                          # detect what the platform offers
forkunion = { version = "3.0.3", features = ["portable"] }   # STL thread pool only
forkunion = { version = "3.0.3", features = ["place-memory-on-domain"] } # require NUMA-aware allocations

Or for the preview development version:

toml
[dependencies]
forkunion = { git = "https://github.com/ashvardanian/ForkUnion.git", branch = "main-dev" }

A minimal example may look like this:

rust
use forkunion as fu;
let topology = fu::Topology::new().expect("Failed to detect hardware topology");
let mut pool = fu::spawn(&topology, 2);
pool.for_threads(&|thread_index, compute_domain_index| {
    println!("Hello from thread # {} on compute domain # {}", thread_index + 1, compute_domain_index + 1);
});

Higher-level APIs distribute index-addressable tasks across the threads in the pool:

rust
pool.for_n(100, |prong| {
    println!("Running task {} on thread # {}",
        prong.task_index + 1, prong.thread_index + 1);
});
pool.for_slices(100, |prong, count| {
    println!("Running slice [{}, {}) on thread # {}",
        prong.task_index, prong.task_index + count, prong.thread_index + 1);
});
pool.for_n_dynamic(100, |prong| {
    println!("Running task {} on thread # {}",
        prong.task_index + 1, prong.thread_index + 1);
});

A more realistic example with named threads and error handling may look like this:

rust
use std::error::Error;
use forkunion as fu;

fn heavy_math(_: usize) {}

fn main() -> Result> {
    let topology = fu::Topology::new()?;
    let mut pool = fu::ThreadPool::try_named_spawn(&topology, "heavy-math", 4)?;
    pool.for_n_dynamic(400, |prong| {
        heavy_math(prong.task_index);
    });
    Ok(())
}

The Topology handle reports the hardware topology, to size and place work:

rust
let topology = fu::Topology::new()?;
for domain in 0..topology.compute_domains_count() {
    let domain = fu::ComputeDomain(domain);
    println!("domain {}: {} cores, level {}, allocate on memory domain {}",
        domain.get(), topology.logical_cores_count_in(domain),
        topology.compute_level_in(domain), topology.local_memory_of(domain).get());
}

For advanced usage, refer to the NUMA section below. For convenience Rayon-style parallel iterators pull the prelude module and check out related examples.

Intro in C++

To integrate into your C++ project, either copy the include/ directory into your project, add a Git submodule, or CMake. The forkunion.hpp umbrella is a list of #includes; the implementation lives beside it in include/forkunion/, one header per concern, so a platform backend can be read or replaced on its own. For a Git submodule, run:

bash
git submodule add https://github.com/ashvardanian/ForkUnion.git extern/forkunion

Alternatively, using CMake:

cmake
FetchContent_Declare(
    forkunion
    GIT_REPOSITORY https://github.com/ashvardanian/ForkUnion
    GIT_TAG v3.0.3
)
FetchContent_MakeAvailable(forkunion)
target_link_libraries(your_target PRIVATE forkunion::header)

Then, include the header in your C++ code:

…

The fu_-prefixed functions report the hardware topology, to size and place work:

…

GCC Nested Functions Extension

GCC supports nested functions that can capture variables from the enclosing scope:

…

Compile: gcc -std=c11 test.c -lforkunion_static -lstdc++ -lpthread The archive is C++ behind a C facade, so a C consumer names the C++ runtime itself — -lc++ under Apple's toolchain.

Clang Blocks Extension

Clang provides blocks with ^{} syntax:

…

Compile: clang -std=c11 -fblocks test.c -lforkunion_static -lc++ -lpthread -lBlocksRuntime

Alternatives & Differences

Most "thread-pools" are not thread-pools at all, but task-queues - think a std::queue> behind a std::mutex, executing each heap-allocated task on whatever core the OS scheduler picks. That shape is what most alternatives are built around, and it is the wrong shape for a tight parallel for:

  • Modern C++: taskflow/taskflow, progschj/ThreadPool, bshoshany/thread-pool
  • Traditional C++: vit-vit/CTPL, mtrebi/thread-pool
  • Rust: tokio-rs/tokio, rayon-rs/rayon, smol-rs/smol
  • Zig: std.Thread.Pool

Reach for ForkUnion when you have data-parallel loops, bulk-synchronous phases, or NUMA-sharded scans, and you want them to dispatch in nanoseconds with neither an allocator nor the kernel on the hot path. Reach for something else when you need task graphs, async I/O, futures and promises, work that outlives its scope, or nested parallelism — Taskflow, Tokio, and oneTBB are built for exactly that, and ForkUnion deliberately bans it.

The trade-offs line up like this, verified against each project's own source and documentation:

Property ForkUnion OpenMP Rayon Taskflow oneTBB
Hot-path heap allocations None ≈ None ¹ Per spawned task ² Per task node Per task, pooled
Syscalls to dispatch None Spin, then futex Spin/yield, then futex Spin, then futex Spin, then futex
Dynamic-schedule primitive fetch_add cursors CAS retry loop ³ Chase-Lev CAS deque ⁴ Chase-Lev CAS deque ⁴ Chase-Lev CAS deque ⁴
NUMA pinning & local allocators Yes Yes ⁵ No No ⁶ Partial ⁷
Hardware timed-wait Yes x86 only, opt-in ⁸ No No No
Languages C++ · C · Rust · Zig C · C++ · Fortran Rust C++ C++
Design center Fork-join loops Loops + tasks Stealing tasks + iterators Task-graph DAG Stealing tasks

¹ OpenMP reuses a persistent thread team, so a repeated parallel for over a fixed-size team does not re-create it; the per-region worksharing descriptor is reused but not provably allocation-free on every path. ² Rayon keeps join and parallel-iterator jobs on the stack, but scope/spawn heap-allocate one job per task, and the Chase-Lev deque grows on the heap under deep nesting. ³ libgomp and libomp hand out schedule(dynamic) iterations with

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

C++armatomicscompare-and-swapconcurrency

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言