#1609·ggml

Convert ggml to Rust

Author: thewh1teagleCreated Aug 27, 2026Updated Aug 27, 2026

C and C++ carry a class of memory safety bugs that a runtime like ggml, embedded in llama.cpp and whisper.cpp and shipped inside countless downstream apps, is especially exposed to: buffer overflows in tensor arithmetic, use after free across backend buffers, unsafe parsing of untrusted GGUF files. The wider ecosystem is moving toward memory safe languages for exactly this kind of infrastructure.

Proposal: consider an incremental Rust implementation of the ggml core, starting where safety matters most and C interop is simplest:

  1. GGUF parsing (untrusted input, self contained, an existing CVE surface)
  2. The graph/allocator/registry layer (ggml.c, ggml-alloc, ggml-backend), keeping the existing C API via FFI so llama.cpp and whisper.cpp are unaffected
  3. CPU kernels last, where Rust SIMD intrinsics map one to one; GPU backends stay as they are since shader code is language neutral

Downstream data point: we ported whisper.cpp's full decoding pipeline to Rust on top of unmodified ggml kernels and verified it bit identical to the C++ implementation, so incremental replacement with exact behavior is practical.

Interested whether the maintainers would accept such a direction at all, or whether this belongs in a sibling project.