[Bug] compressed-tensors checkpoint + NEXTN/MTP head fails at construction with "No compressed-tensors compatible scheme was found" unless mtp.* is in `ignore` (module not named in the error)
Author: rnxrxCreated Sep 8, 2026Updated Sep 17, 2026
Summary
Serving a compressed-tensors checkpoint with --speculative-algorithm NEXTN crashes at model construction:
File ".../sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py", line ..., in get_linear_scheme
scheme = self._get_scheme_from_parts( # type: ignore
File ".../sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py", line ..., in _get_scheme_from_parts
raise NotImplementedError("No compressed-tensors compatible scheme was found.")
NotImplementedError: No compressed-tensors compatible scheme was found.The failing module is the MTP head's mtp.layers.0.self_attn.qkv_proj (found by instrumenting prefix; the error itself does not say). Adding "re:mtp\\..*" to quantization_config.ignore in config.json fixes it. Verified on main @ 52fecfdf by reading the code path (get_linear_scheme → _get_scheme_from_parts), measured on a Qwen dev build (0.0.0.dev1+gd91c3682b).
Mechanism
- llm-compressor-style checkpoints use
config_groups: {group_0: {targets: ["Linear"], weights: int4/group, input_activations: fp8 dynamic}}plus a longignorelist of everything that was not quantized. - The MTP head is not part of the HF model graph that the quantizer traces, so it is neither quantized nor listed in
ignore. Its weights in the checkpoint are plain bf16 (typically re-attached from the base model after quantization). - SGLang's NEXTN path constructs the draft model's
Linearlayers through the sameCompressedTensorsConfig. They match theLineartarget by module type, soget_scheme_dictreturns the W4A8 args, and_get_scheme_from_partsraises because there is no dense Linear scheme for int4-weight + fp8-activation in SGLang (that combination is MoE-only viaCompressedTensorsW4AFP8MoE). get_linear_schemeonly falls back toUnquantizedLinearMethodwhenweight_quant is None, i.e. when the layer matched nothing or is inignore. A layer that matches by type but has no quantized tensors in the checkpoint is not considered.
For comparison, NVIDIA ModelOpt exports of the same model put mtp.* and model.mtp.* in ignore, so those checkpoints work; llm-compressor cannot emit that entry because it never sees the MTP weights.
Requests
- Name the module in the
NotImplementedError(thelayer_name/prefixis available inget_linear_scheme) and hint at theignorefix. Today the message gives no way to know which layer failed. - For the draft-model / NEXTN path, treat modules whose checkpoint tensors are unquantized (no
weight_packed/weight_scalepresent) as unquantized instead of raising — or auto-ignore the draft module prefix when the checkpoint carries no quantized tensors for it. - Document the
re:mtp\..*(ormtp.*)ignorerequirement for NEXTN on compressed-tensors checkpoints.
Environment
- SGLang
main@52fecfdf(code), Qwen dev build0.0.0.dev1+gd91c3682b(measurement) - Qwen3-Next-family hybrid MoE with a 1-layer MTP head; the mechanism is architecture-independent (any NEXTN model with a compressed-tensors checkpoint whose
ignoreomits the MTP head) - 2× H100 NVL,
--tp 2 --ep-size 2 --speculative-algorithm NEXTN
Source: sgl-project/sglang