[错误] 压缩张量检查点 + NEXTN/MTP 头在构建时出现问题,出现"未找到兼容的压缩张量方案",除非 mtp.* 位于 `ignore` 中(模块未在错误中命名)
作者: rnxrx创建于 2026年9月8日更新于 2026年9月17日
- LLM-compressor-style checkpoints use
config_groups: {group_0: {targets: ["Linear"], weights: int4/group, input_activations: fp8 dynamic}}plus a longignorelist of everything that was not quantized. - The MTP head is not part of the HF model graph that the quantizer traces, so it is neither quantized nor listed in
ignore. Its weights in the checkpoint are plain bf16 (typically re-attached from the base model after quantization). - SGLang's NEXTN path constructs the draft model's
Linearlayers through the sameCompressedTensorsConfig. They match theLineartarget by module type, soget_scheme_dictreturns the W4A8 args, and_get_scheme_from_partsraises because there is no dense Linear scheme for int4-weight + fp8-activation in SGLang (that combination is MoE-only viaCompressedTensorsW4AFP8MoE). get_linear_schemeonly falls back toUnquantizedLinearMethodwhenweight_quant is None, i.e. when the layer matched nothing or is inignore. A layer that matches by type but has no quantized tensors in the checkpoint is not considered. For comparison, NVIDIA ModelOpt exports of the same model putmtp.*andmodel.mtp.*inignore, so those checkpoints work; LLM-compressor cannot emit that entry because it never sees the MTP weights.- Name the module in the
NotImplementedError(thelayer_name/prefixis available inget_linear_scheme) and hint at theignorefix. Today the message gives no way to know which layer failed. - For the draft-model / NEXTN path, treat modules whose checkpoint tensors are unquantized (no
weight_packed/weight_scalepresent) as unquantized instead of raising — or auto-ignore the draft module prefix when the checkpoint carries no quantized tensors for it. - Document the
re:mtp\..*(ormtp.*)ignorerequirement for NEXTN on compressed-tensors checkpoints.
内容来源: sgl-project/sglang