#3884·candle

`get_or_load_func` drops the kernel name from CUDA symbol-lookup errors

Author: astoriseCreated Aug 14, 2026Updated Aug 29, 2026

Summary

When a CUDA kernel symbol is missing from a loaded module, the resulting error does not say which kernel. CudaDevice::get_or_load_func takes the name as an argument and discards it on the failure path:

rust
// candle-core/src/cuda_backend/device.rs
pub fn get_or_load_func(&self, fn_name: &str, mdl: &kernels::Module) -> Result<CudaFunc> {
    let ms = self.modules.read().unwrap();
    if let Some(mdl) = ms.mdls[mdl.index()].as_ref() {
        let func = mdl.load_function(fn_name).w()?;   // <- fn_name dropped
        ...
    }
    drop(ms);
    let mut ms = self.modules.write().unwrap();
    let cuda_module = self.context.load_module(mdl.ptx().into()).w()?;
    ms.mdls[mdl.index()] = Some(cuda_module.clone());
    let func = cuda_module.load_function(fn_name).w()?; // <- and here
    ...
}

.w() wraps cudarc's DriverError unchanged, so the caller gets:

DriverError(CUDA_ERROR_NOT_FOUND, "named symbol not found")

The name that would make this actionable was in scope at the point of failure and was thrown away.

Why this is worse than it looks

Two things conspire to make it hard to recover the information afterwards.

A backtrace does not identify the kernel. The error propagates upward as a value through ?; the eventual panic happens at whatever unwrap/expect consumes it, often several layers up in user code. RUST_BACKTRACE=1 names that consumer, not the get_or_load_func call site. The natural first debugging step yields nothing.

A missing symbol is a legitimate, reachable state, not a corrupt build. candle-kernels guards kernels by target architecture, and the guards are not uniform across a dtype. For F8E4M3 specifically, the casts sit under one threshold and the elementwise and indexing ops under a higher one:

kernels guard
cast_f8_e4m3_f32, cast_f8_e4m3_bf16, … (cast.cu) __CUDA_ARCH__ >= 800
affine_f8_e4m3 (affine.cu) __CUDA_ARCH__ >= 890
is_*_f8_e4m3, gather_*_f8_e4m3, scatter_add_f8 (indexing.cu) __CUDA_ARCH__ >= 890

So a build for, say, sm_86 has a genuinely partial F8E4M3 kernel set: some operations on such a tensor work and others fail at lookup time. That is a reasonable design — but it means "symbol not found" is a routine outcome of running a supported dtype on a supported device, and the one thing the user needs in order to understand it is the name that is currently discarded.

Concretely: we hit CUDA_ERROR_NOT_FOUND while building a model on an sm_86 device and, after reading the guards above and tracing the load path by hand, still cannot say which kernel is missing. That is the entire content of this report — not that the lookup failed, but that a failed lookup is undiagnosable from its own error.

Why not fix this in cudarc

DriverError is a newtype over the raw driver result:

rust
pub struct DriverError(pub sys::CUresult);

It has a public field, and its std::error::Error impl is behind #[cfg(feature = "std")]. Carrying a symbol name would require an owned String — allocation in a no_std-friendly FFI binding, a change to a pattern-matchable public shape, and a breaking change across every API that returns it. cudarc already surfaces the driver's own description; the caller's identifier is the caller's to attach.

Shape of the fix

Attach the name at both load_function sites, and the module with it, since "which module was searched" is the natural follow-up question:

cuda: kernel `affine_f8_e4m3` not found in module `affine`
  (this kernel may not be compiled for the target architecture)

The parenthetical matters as much as the name: for a user on a pre-sm_89 device, "not found" reads as a broken installation, when it is in fact the architecture guard doing its job.

Acceptance

  • A failed lookup names the kernel and the module it was searched in.
  • The underlying DriverError remains in the source chain, so CUDA_ERROR_NOT_FOUND is still distinguishable from other driver failures.
  • Both call sites are covered — the cached-module path and the load-then-lookup path fail identically today and must keep doing so.