cutile-rs

cuTile Rust (cutile-rs) is a tile-based system for writing memory-safe, data-race-free GPU kernels in idiomatic Rust. It extends Rust's ownership discipline across the GPU launch boundary: mutable tensors are partitioned into disjoint pieces before launch, immutable tensors are shared, and generated launchers preserve ownership while GPU work is in flight. The same model supports synchronous launches, asynchronous pipelines, and CUDA graph replay. The #[cutile::module] macro embeds a captured Rust AST for each kernel in the host binary; when a kernel is needed, cuTile Rust JIT-compiles that AST through CUDA Tile IR into a GPU cubin. Local opt-outs remain available when lower-level control is needed.

Project Status

We are excited to release this research project as a demonstration of how GPU programming can be made available in the Rust ecosystem. The software is in an early stage and under active development: you should expect bugs, incomplete features, and API breakage as we work to improve it. That being said, we hope you'll be interested to try it in your work and help shape its direction by providing feedback on your experience.

Please check out CONTRIBUTING.md if you're interested in contributing.

Quick Start

…

The #[cutile::module] macro transforms add into a GPU kernel and generates a host-side launcher. The host code constructs lazy tensor operations, partitions the mutable output into 128-element chunks, and calls .sync() to JIT-compile and execute the kernel.

Launches return all runtime arguments in parameter order (z, x, y here), including inputs and scalars. .unpartition().to_host_vec().sync()? copies the partitioned tensor's contents into a vector on the host machine.

The kernel signature carries the access discipline into device code: z is the exclusive mutable output, while x and y are shared read-only inputs. The body loads input tiles matching the output partition, adds them, and stores the result. The launch grid (8, 1, 1) is inferred from the partition: 1024÷128 = 8 tiles.

• Run a similar example via cargo run -p cutile-examples --example saxpy.
• More kernels and usage examples of the host-side API can be found here.

Setup

Requirements

• Rust: stable 1.89+ (no nightly required).
• Linux: tested on Ubuntu 24.04.

GPU and toolkit requirements for cuTile Rust:

GPU compute capability	Minimum CUDA Toolkit
sm_8x (Ampere / Ada)	13.2
sm_90 (Hopper)	13.3
sm_100+ (Blackwell, including DGX Spark / GB10 sm_121)	13.2

CUDA 13.3 is recommended. FP4 packing and block-scaled MMA require 13.3. GPUs below sm_80 (such as sm_70 and sm_75) are unsupported.

Install

Rust

To install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup default stable

CUDA

Install CUDA 13.3 for your OS by following the official instructions: https://developer.nvidia.com/cuda-downloads

Configure Environment

Set CUDA_TOOLKIT_PATH (or CUDA_HOME, consulted second) to your CUDA 13.3 install directory for a reproducible setup. If neither is set, cuTile searches standard CUDA 13.3/13.2 install locations such as /usr/local/cuda-13.3, /usr/local/cuda-13.2, /usr/local/cuda-13, /usr/local/cuda, and /opt/cuda.

Example .cargo/config.toml:

[env]
CUDA_TOOLKIT_PATH = { value = "/usr/local/cuda-13", relative = false }

Verifying Installation

Run the hello world example:

cargo run -p cutile-examples --example hello_world

If everything works, you should see: Hello, I am tile in a kernel with tiles.

Via Nix

We provide a Nix flake for easy setup and development. Flakes must be enabled in your Nix configuration, if not already, add to ~/.config/nix/nix.conf:

experimental-features = nix-command flakes

Run a command directly:

nix develop -c cargo run -p cutile-examples --example saxpy

Or open an interactive shell:

nix develop
# cutile-rs dev shell
#  ✓ CUDA  /nix/store/...-cuda-toolkit-13.3
#  ✓ Rust  1.90.0-nightly

The flake automatically locates host NVIDIA driver libraries on both NixOS and non-NixOS systems.

Tests

• cuTile IR: cargo test --package cutile-ir
• cuTile Rust Compiler: cargo test --package cutile-compiler
• cuTile Rust Library: cargo test --package cutile
• Examples: run an individual example, for example cargo run -p cutile-examples --example async_gemm
• Benchmarks: cargo bench
• Everything: ./scripts/run_all.sh (or pipe to a log file: ./scripts/run_all.sh 2>&1 | tee test_run.log)

Workspace Crates

…

Related Projects and References

• Grout: Qwen 3 inference engine in Rust by Hugging Face, built with cuTile Rust and useful as a reference for production kernel call sites.
• cuTile Python: Python kernel programming with CUDA Tile.
• TileGym: CUDA Tile kernel examples and tuning patterns, including a set of ready-to-use cuTile Rust kernels under ops/cutile_rs.
• cuda-oxide: NVlabs experimental Rust-to-CUDA compiler for writing SIMT-style GPU kernels in Rust.
• CUDA Tile IR documentation: CUDA Tile IR reference documentation.
• CUDA documentation: CUDA toolkit documentation.
• Rust NVPTX backend: rustc's target support for generating PTX for NVIDIA GPUs.

Paper

The cuTile Rust paper, Fearless Concurrency on the GPU, is available here. On NVIDIA B200, cuTile Rust reaches 7 TB/s for element-wise operations and 2 PFlop/s for GEMM, about 91% of peak memory bandwidth and 92% of dense f16 peak, respectively. The GEMM result is competitive with cuBLAS, and the B200 safety-overhead microbenchmarks show that cuTile Rust adds safety without measurable runtime overhead: safe Rust persistent GEMM reaches 2.07 PFlop/s at M=N=K=8192 (92% of the B200 dense f16 peak), within 0.3% of the corresponding low-level Tile IR variant.

The paper also evaluates Grout, a Qwen3 inference engine built with cuTile Rust in collaboration with Hugging Face. In batch-1 Qwen3 decode, Grout reaches 171 tokens/s for Qwen3-4B on NVIDIA GeForce RTX 5090 and 82 tokens/s for Qwen3-32B on B200, showing competitive state-of-the-art performance on memory-bound inference tasks as measured by our HBM roofline analysis.

Reproducibility artifacts for the paper evaluation are available here. The paper-facing measurements were run against cuTile Rust 0.2.0, and the version of Grout used for the paper is available here.

Citing

If you use cuTile Rust in research, please cite the paper:

@misc{elibol2026fearlessconcurrencygpu,
  title = {Fearless Concurrency on the GPU},
  author = {Elibol, Melih and Roesch, Jared and Gelado, Isaac and Buehler, Eric and Garland, Michael},
  year = {2026},
  eprint = {2606.15991},
  archivePrefix = {arXiv},
  primaryClass = {cs.PL},
  url = {https://arxiv.org/abs/2606.15991}
}

License

All crates are licensed under the Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0