Bug: --gpu nvidia and vulkan fail on older NVIDIA mobile GPUs (GTX 970M / Maxwell) on Windows

Contact Details

[email protected]

What happened?

Describe the bug

On Windows 10/11 using a laptop with an older NVIDIA GPU (specifically a GeForce GTX 970M, Maxwell architecture), llamafile fails to detect or utilize the GPU via either --gpu nvidia or --gpu vulkan, even though the appropriate tools (Visual Studio MSVC and CUDA Toolkit) are fully installed and configured.

Running --gpu nvidia throws: fatal error: support for --gpu nvidia was explicitly requested, but it wasn't available

Running --gpu vulkan throws: fatal error: --gpu vulkan was explicitly requested but Vulkan is not usable on this system - no Vulkan-capable device was detected...

Running --gpu auto completely ignores the discrete GPU and falls back to CPU compilation (compute: Intel Core i7-6700HQ), leaving the 6GB of VRAM untouched.

Environment Context

• OS: Windows 10 / Windows 11
• Hardware: ASUS ROG G752VT Laptop (Intel Core i7-6700HQ + NVIDIA GeForce GTX 970M 6GB VRAM)
• Architecture: Maxwell (Compute Capability 5.2)
• CUDA Compiler: nvcc --version reports CUDA 13.x successfully installed and available in System PATH.
• Llamafile version: 0.10.5

Additional Information

The GPU is fully functional, capable of running Vulkan applications, and works perfectly with other local LLM runners like Ollama. It seems the hardware graphics driver probe mechanism in llamafile either crashes during detection or strictly filters out older Compute Capability (5.x/6.x) devices, preventing any GPU offloading on highly popular legacy hardware.

Could the detection script be updated or relaxed to allow manual overrides for older CUDA/Vulkan capable hardware?

Version

Qwen3.5-0.8B-Q8_0.llamafile.exe

What operating system are you seeing the problem on?

No response

Relevant log output