[Bug]: prefix caching + MTP still corrupts output on hybrid Mamba/GDN models in v0.28.0 (#43559 closed but unfixed)

Your current environment

==============================
        System Info
==============================
OS                           : Ubuntu 24.04.4 LTS (x86_64)
GCC version                  : (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
Libc version                 : glibc-2.39

==============================
       PyTorch Info
==============================
PyTorch version              : 2.13.0+cu130
Is debug build               : False
CUDA used to build PyTorch   : 13.0
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.3 (main, Jun 19 2026, 12:46:00) [GCC 13.3.0] (64-bit runtime)
Python platform              : Linux-6.8.0-1010-gcp-x86_64-with-glibc2.39

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : True
CUDA runtime version         : 12.0.140
GPU models and configuration :
GPU 0-7: NVIDIA H100 80GB HBM3   (single GPU used for this report, TP=1)
Nvidia driver version        : 580.173.02

==============================
          CPU Info
==============================
Architecture:                x86_64
Model name:                  Intel(R) Xeon(R) Platinum 8481C CPU @ 2.70GHz
CPU(s):                      208
Hypervisor vendor:           KVM

==============================
         vLLM Info
==============================
vLLM Version                 : 0.28.0
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled; XPU: Disabled
GPU Topology: NV18 all-to-all (8x H100 SXM), TP=1 used

Relevant packages:
  vllm==0.28.0
  torch==2.13.0
  transformers==5.15.1
  flashinfer-python==0.6.16.post3
  compressed-tensors==0.17.0
  xgrammar==0.2.3

Describe the bug

Summary. --enable-prefix-caching together with MTP speculative decoding still corrupts output on hybrid (GDN/Mamba) Qwen3.5-class models in v0.28.0. A small fraction of responses degenerate into repeated punctuation (!!!!…) or come back empty. This is the same failure as #43559, which was closed as completed on 2026-08-06 — the merged scheduler-side fix (#51113) does not cover it, and the cache-manager-side fixes (#43650, #48375) are still open.

Model. Qwen3.5-architecture dense 27B VLM (Qwen3_5ForConditionalGeneration, hybrid: Gated DeltaNet linear-attention layers + full attention every 4th layer, vision tower), FP8 W8A8 compressed-tensors (FP8_DYNAMIC), with an MTP head present in the checkpoint (Qwen3_5MTP resolves at startup). TP=1, single H100.

Server command (paths redacted):

export VLLM_NO_USAGE_STATS=1
export VLLM_USE_FLASHINFER_SAMPLER=0

vllm serve <QWEN3_5_27B_FP8_WITH_MTP_HEAD> \
  --served-model-name path-mini \
  --port 8000 \
  --max-model-len 62768 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.95 \
  --enable-chunked-prefill \
  --limit-mm-per-prompt.image 10 \
  --enable-prefix-caching \
  --mamba-cache-mode align \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

Startup confirms the configuration: Resolved architecture: Qwen3_5MTP, enable_prefix_caching=True, num_spec_tokens=2, GPU KV cache size: 576,637 tokens, attention block size 800 (forced up to match the mamba page size), async scheduling enabled (default).

Workload. Multi-step GUI-agent traces: each request is a conversation of prior steps plus 1–3 screenshots (sliding 2-image history), ~4k–12k prompt tokens, ~20–100 output tokens, temperature=0, client concurrency up to 32. Consecutive steps of the same episode share a long prefix, so prefix-cache hit rate climbs through a run.

Observed.

Malformed = response is a run of repeated ! characters, or empty content. finish_reason is normal; no HTTP 5xx, no crash, no traceback — the engine stays healthy and every other response in the same run is correct.

Two observations that point at the prefix cache rather than at MTP alone:

1. Rate tracks cache hit rate. Corruption appears only once the prefix cache starts hitting, and scales with the hit rate.
2. --kv-cache-dtype fp8 appears to avoid it, but only by destroying the cache. With fp8 KV, vLLM raises the attention block size 800 → 1600 (platforms/interface.py, so that one attention page covers one mamba page). Our requests share ~1,700 tokens of prefix, so at a 1600-token matching unit almost nothing is reusable: hit rate collapses from 31.7% to ~10%, and malformed responses go to 0/1,135. That is consistent with reduced exposure, not with fp8 fixing the underlying defect. (For the same reason --block-size 800 cannot be used to compensate: the value is derived and only ever raised.)

Suspected cause (already described in #43650 / #48375): MambaManager.find_longest_cache_hit in vllm/v1/core/single_type_kv_cache_manager.py accepts drop_eagle_block and never acts on it, so the final matched block — which can hold recurrent state written over draft positions that verification later rejected — stays reachable through the prefix cache and is reused by subsequent requests sharing that prefix. FullAttentionManager drops that block. In v0.28.0 the caller (kv_cache_coordinator.py) additionally skips the eagle margin for MambaSpec with the comment "its finder never drops", i.e. caller and manager disagree about who is responsible.

Applying #43650 (6 lines, max_num_blocks -= 1 when use_eagle) on top of v0.28.0 starts cleanly and shows no regression on an unrelated accuracy check (100 grounding requests at concurrency 32: 76% vs 75% unpatched, identical coordinate error, 0 malformed). Other users report it resolving the symptom outright (#43650: 18/39 → 39/39 on a private benchmark; content: null at ~1-in-8 fixed completely).

Expected. --enable-prefix-caching + MTP on a hybrid Mamba/GDN model should be either correct or explicitly rejected at startup, not silently corrupting a fraction of responses.

Notes / questions

• Was #43559 closed by #51113 intentionally? #51113 fixes chunk alignment on the write path; the reports above (and this one) are about block dropping on the cache-read path, which neither #51113 nor #46384 appears to cover.
• If #43650 is considered too blunt because a hybrid block can be 1k+ tokens, would a narrower fix be acceptable — e.g. dropping the trailing block only when that block actually contains unverified draft positions?
• Happy to run any specific probe on this configuration; we can reproduce on demand with the workload above.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.