`get_or_load_func` drops the kernel name from CUDA symbol-lookup errors
Summary
When a CUDA kernel symbol is missing from a loaded module, the resulting error
does not say which kernel. CudaDevice::get_or_load_func takes the name as an
argument and discards it on the failure path:
// candle-core/src/cuda_backend/device.rs
pub fn get_or_load_func(&self, fn_name: &str, mdl: &kernels::Module) -> Result<CudaFunc> {
let ms = self.modules.read().unwrap();
if let Some(mdl) = ms.mdls[mdl.index()].as_ref() {
let func = mdl.load_function(fn_name).w()?; // <- fn_name dropped
...
}
drop(ms);
let mut ms = self.modules.write().unwrap();
let cuda_module = self.context.load_module(mdl.ptx().into()).w()?;
ms.mdls[mdl.index()] = Some(cuda_module.clone());
let func = cuda_module.load_function(fn_name).w()?; // <- and here
...
}.w() wraps cudarc's DriverError unchanged, so the caller gets:
DriverError(CUDA_ERROR_NOT_FOUND, "named symbol not found")The name that would make this actionable was in scope at the point of failure and was thrown away.
Why this is worse than it looks
Two things conspire to make it hard to recover the information afterwards.
A backtrace does not identify the kernel. The error propagates upward as a
value through ?; the eventual panic happens at whatever unwrap/expect
consumes it, often several layers up in user code. RUST_BACKTRACE=1 names
that consumer, not the get_or_load_func call site. The natural first
debugging step yields nothing.
A missing symbol is a legitimate, reachable state, not a corrupt build.
candle-kernels guards kernels by target architecture, and the guards are not
uniform across a dtype. For F8E4M3 specifically, the casts sit under one
threshold and the elementwise and indexing ops under a higher one:
| kernels | guard |
|---|---|
cast_f8_e4m3_f32, cast_f8_e4m3_bf16, … (cast.cu) |
__CUDA_ARCH__ >= 800 |
affine_f8_e4m3 (affine.cu) |
__CUDA_ARCH__ >= 890 |
is_*_f8_e4m3, gather_*_f8_e4m3, scatter_add_f8 (indexing.cu) |
__CUDA_ARCH__ >= 890 |
So a build for, say, sm_86 has a genuinely partial F8E4M3 kernel set: some
operations on such a tensor work and others fail at lookup time. That is a
reasonable design — but it means "symbol not found" is a routine outcome of
running a supported dtype on a supported device, and the one thing the user
needs in order to understand it is the name that is currently discarded.
Concretely: we hit CUDA_ERROR_NOT_FOUND while building a model on an
sm_86 device and, after reading the guards above and tracing the load path by
hand, still cannot say which kernel is missing. That is the entire content of
this report — not that the lookup failed, but that a failed lookup is
undiagnosable from its own error.
Why not fix this in cudarc
DriverError is a newtype over the raw driver result:
pub struct DriverError(pub sys::CUresult);It has a public field, and its std::error::Error impl is behind
#[cfg(feature = "std")]. Carrying a symbol name would require an owned
String — allocation in a no_std-friendly FFI binding, a change to a
pattern-matchable public shape, and a breaking change across every API that
returns it. cudarc already surfaces the driver's own description; the caller's
identifier is the caller's to attach.
Shape of the fix
Attach the name at both load_function sites, and the module with it, since
"which module was searched" is the natural follow-up question:
cuda: kernel `affine_f8_e4m3` not found in module `affine`
(this kernel may not be compiled for the target architecture)The parenthetical matters as much as the name: for a user on a pre-sm_89
device, "not found" reads as a broken installation, when it is in fact the
architecture guard doing its job.
Acceptance
- A failed lookup names the kernel and the module it was searched in.
- The underlying
DriverErrorremains in the source chain, soCUDA_ERROR_NOT_FOUNDis still distinguishable from other driver failures. - Both call sites are covered — the cached-module path and the load-then-lookup path fail identically today and must keep doing so.
Source: huggingface/candle