IQ9-EVK NPU Limitations?
Author: dtischlerCreated Jul 28, 2026Updated Sep 16, 2026
When attempting to run Gemma4 GGUF from HuggingFace on the IQ9-EVK NPU, E4B-it-qat-q4_0-gguf works, but increasing the size to
26B-A4B-it-qat-q4_0-gguf does not work. It will load on CPU, however. Any ideas how to get the larger model working?
ubuntu@ubuntu:~$ geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf --compute npu
> What is the capital of France?
The capital of France is **Paris**.
— 9.5 tok/s • 8 tok • 0.2 s first token —
> /exit
ubuntu@ubuntu:~$ geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf --compute npu
⚠️ Oops. Model failed to load.
Try these:
- Redownload the model.
- Verify your system meets the model's requirements.
- Check your NPU / GPU driver version and update it if it's out of date.
- See help in our discord or slack.
ubuntu@ubuntu:~$ geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf --compute cpu
> What is the capital of France?
The capital of France is **Paris**.
— 13.0 tok/s • 8 tok • 0.5 s first token —
> /exitThanks!
Source: qualcomm/GenieX