[performation] Quen3.5-4B (q4f16) 为~3x慢解码+~20x慢解码 TTFT对WebGPU上的Qwen3-4B (transfers.js 4.0.0-next

System Info

• @huggingface/transformers version: 4.0.0-next.7
• Browser: Chrome 145.0.7632.160 (Official Build) (arm64)
• OS: macOS Version 14.3.1 (Build 23D60)
• GPU: Apple, ANGLE Metal Renderer: Apple M2 Pro, Version 14.3.1 (Build 23D60))
• WebGPU: Enabled
• Backend: WebGPU (device: 'WebGPU', dtype: 'q4f16')
• Other libraries: None (vanilla HTML + ES module, no framework)

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

[Performance] Qwen3.5-4B-q4f16 is significantly slower than Qwen3-4B-q4f16 on identical translation task (WebGPU)

Environment

Item	Value
@huggingface/transformers	v4.0.0-next.7
Backend	WebGPU
Browser	Chrome 145.0.7632.160
OS	macOS Version 14.3.1 (Build 23D60)
GPU	Apple, ANGLE Metal Renderer: Apple M2 Pro, Version 14.3.1 (Build 23D60))

---

Model File Sizes (from HuggingFace Hub)

Model	ONNX file (q4f16)	Size
Qwen3-4B	model_q4f16.onnx	~2.4 GB
Qwen3.5-4B	model_q4f16.onnx	~2.5 GB

File sizes are comparable, ruling out download/IO as a factor.

Description

When running an identical Japanese→English subtitle translation task using the same quantization (q4f16) and the same batch pipeline, Qwen3.5-4B is consistently ~2–3x slower than Qwen3-4B in net token throughput on WebGPU. This is unexpected given that both models have the same parameter count (4B).

Suspected Root Cause

After investigation, the likely bottlenecks in Qwen3.5-4B are:

1. Vocabulary size: Qwen3.5-4B has 248,320 tokens vs Qwen3-4B's 151,936 tokens (~63% larger). This means the lm_head matmul on every single decode step is significantly heavier.
2. Hybrid attention pattern: Qwen3.5 uses a mixed layout of 8 × (3 sliding-window attention + 1 full attention) layers. This pattern may not be fully optimized in the current ONNX export or ONNX Runtime WebGPU kernel path.
3. Possible missing kernel optimization: MatMulNBits / GroupQueryAttention contrib ops may not be fused/optimized for Qwen3.5's specific graph shape under WebGPU.

---

Reproduction

A self-contained benchmark demo (Qwen3 vs Qwen3.5 Subtitle Translation Benchmark.html) is attached. It runs both models on the same 124-line Japanese subtitle file, measures net t/s, TTFT, and total time, and displays a side-by-side comparison. Steps:

1. Open the attached benchmark.html in Chrome 113+ with WebGPU enabled
2. Click "▶ Run Qwen3-4B-q4f16" — wait for completion
3. Click "▶ Run Qwen3.5-4B-q4f16" — wait for completion
4. Observe the speed comparison panel