Params4bit.to() 跳过 .cuda()/.xpu() 所应用的 CPU AVX512 包装逆转,因此 model.to(device) 会默默地损坏权重

作者: caiotheodoro创建于 2026年9月9日更新于 2026年9月9日

The packing (`_convert_weight_packed_for_cpu`, `bitsandbytes/functional.py`) is a real bit-layout transform for the AVX512 CPU kernel: it unpacks each nibble, transposes/regroups them into 32-row blocks, and repacks — not a `view`/reshape. `Linear4bit.forward` triggers it lazily the first time it runs on CPU with `support_avx512bf16_for_cpu` set. If that same module is later moved off CPU with `model.to(device)` (or `.to("mps")` at all, no matter what device it started on), the packed layout is never inverted, and any kernel dequantizing it after the move reads the wrong nibble order.

内容来源: bitsandbytes-foundation/bitsandbytes