#38574·sglang

[Bug] compressed-tensors checkpoint + NEXTN/MTP head fails at construction with "No compressed-tensors compatible scheme was found" unless mtp.* is in `ignore` (module not named in the error)

Author: rnxrxCreated Sep 8, 2026Updated Sep 17, 2026

Summary

Serving a compressed-tensors checkpoint with --speculative-algorithm NEXTN crashes at model construction:

  File ".../sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py", line ..., in get_linear_scheme
    scheme = self._get_scheme_from_parts(  # type: ignore
  File ".../sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py", line ..., in _get_scheme_from_parts
    raise NotImplementedError("No compressed-tensors compatible scheme was found.")
NotImplementedError: No compressed-tensors compatible scheme was found.

The failing module is the MTP head's mtp.layers.0.self_attn.qkv_proj (found by instrumenting prefix; the error itself does not say). Adding "re:mtp\\..*" to quantization_config.ignore in config.json fixes it. Verified on main @ 52fecfdf by reading the code path (get_linear_scheme_get_scheme_from_parts), measured on a Qwen dev build (0.0.0.dev1+gd91c3682b).

Mechanism

  1. llm-compressor-style checkpoints use config_groups: {group_0: {targets: ["Linear"], weights: int4/group, input_activations: fp8 dynamic}} plus a long ignore list of everything that was not quantized.
  2. The MTP head is not part of the HF model graph that the quantizer traces, so it is neither quantized nor listed in ignore. Its weights in the checkpoint are plain bf16 (typically re-attached from the base model after quantization).
  3. SGLang's NEXTN path constructs the draft model's Linear layers through the same CompressedTensorsConfig. They match the Linear target by module type, so get_scheme_dict returns the W4A8 args, and _get_scheme_from_parts raises because there is no dense Linear scheme for int4-weight + fp8-activation in SGLang (that combination is MoE-only via CompressedTensorsW4AFP8MoE).
  4. get_linear_scheme only falls back to UnquantizedLinearMethod when weight_quant is None, i.e. when the layer matched nothing or is in ignore. A layer that matches by type but has no quantized tensors in the checkpoint is not considered.

For comparison, NVIDIA ModelOpt exports of the same model put mtp.* and model.mtp.* in ignore, so those checkpoints work; llm-compressor cannot emit that entry because it never sees the MTP weights.

Requests

  1. Name the module in the NotImplementedError (the layer_name/prefix is available in get_linear_scheme) and hint at the ignore fix. Today the message gives no way to know which layer failed.
  2. For the draft-model / NEXTN path, treat modules whose checkpoint tensors are unquantized (no weight_packed / weight_scale present) as unquantized instead of raising — or auto-ignore the draft module prefix when the checkpoint carries no quantized tensors for it.
  3. Document the re:mtp\..* (or mtp.*) ignore requirement for NEXTN on compressed-tensors checkpoints.

Environment

  • SGLang main @ 52fecfdf (code), Qwen dev build 0.0.0.dev1+gd91c3682b (measurement)
  • Qwen3-Next-family hybrid MoE with a 1-layer MTP head; the mechanism is architecture-independent (any NEXTN model with a compressed-tensors checkpoint whose ignore omits the MTP head)
  • 2× H100 NVL, --tp 2 --ep-size 2 --speculative-algorithm NEXTN