Gemma 4 models cannot use
Author: Zhang-JKCreated Sep 9, 2026Updated Sep 9, 2026
version: ddf1b879dc3a1760cbcb3f3c4a7c6467850cec4a
env: Ubuntu 24.04; CUDA 12.8
Problem 1: Gemma 4 text checkpoint does not load
The gemma4 example looks up model.embed_tokens.weight, but official Gemma 4 checkpoints store the language-model weights like model.language_model.*. On google/gemma-4-E4B-it that fails with cannot find tensor model.embed_tokens.weight
Problem 2: Gemma 4 E2B/E4B model does not work as it output random words
Source: huggingface/candle