[Feature]: [CPU][GLM5Next][KDA] Add CPU KDA backend for GLM-5.3-Flash

The feature, motivation and pitch

Motivation

GLM-5.3-Flash (Glm5NextForConditionalGeneration) is a hybrid model containing both:

• sparse MLA layers
• KDA (Kimi Delta Attention / gated delta-rule) linear-attention layers

Even if GLM5Next's sparse MLA / KeyPool path is made CPU-compatible, the model still cannot execute end-to-end on CPU without a KDA backend.

I would like to request native CPU support for GLM5Next KDA.

Current state

The GLM5Next KDA implementation currently lives under the accelerator-specific path:

vllm/models/glm5next/nvidia/kda.py

and uses GPU-oriented Flash Linear Attention / Triton operators such as:

chunk_kda_with_fused_gate
fused_recurrent_kda

There does not appear to be a corresponding:

vllm/models/glm5next/cpu/kda.py

or other CPU KDA execution path for GLM5Next on current main.

This makes KDA an independent CPU blocker from sparse MLA / KeyPool support.

GLM-5.3-Flash has a large number of KDA layers, so this cannot reasonably be worked around by supporting only the sparse-MLA portion of the model.

Related CPU infrastructure already exists

The CPU backend already has substantial support for related linear-attention/state-space workloads.

In particular, current CPU platform code contains an accelerated GDN path using:

AMX tiles
or
AVX-512 BF16 / VDPBF16PS

and vLLM already has CPU machinery for stateful hybrid models, including:

• recurrent state caches
• convolution state
• chunked prefill
• decode
• hybrid KV/state-cache management

KDA is not identical to GDN, but they are closely related enough that it may be possible to reuse much of this infrastructure.

KDA-specific work

As I understand it, GLM5Next KDA needs CPU equivalents for the KDA-specific operations around the existing gated-delta/state machinery, including:

Q/K/V short convolution
        │
        ▼
KDA gate computation
        │
        ▼
chunked gated-delta prefill
        │
        ▼
final recurrent state
        │
        ▼
single-token recurrent decode
        │
        ▼
gated/RMSNorm output

The important parts include:

• q/k/v convolution and convolution-state update
• KDA gate calculation
• per-channel decay/gating behavior
• chunk_kda_with_fused_gate equivalent for prefill
• fused_recurrent_kda equivalent for decode
• initial-state gathering
• final-state writeback
• mixed decode/prefill handling
• state-cache addressing
• chunk boundaries
• prefix-cache/state restore correctness

Requested feature

Add a CPU KDA backend for GLM5Next.

A correctness-first implementation would be useful even before full AMX optimization.

One possible structure could be:

vllm/models/glm5next/
├── common/
├── nvidia/
├── amd/
└── cpu/
    └── kda.py

with explicit platform dispatch rather than allowing CPU to reach the NVIDIA implementation.

Possible implementation phases

Phase 1: reference CPU implementation

Provide a functional PyTorch/C++ CPU implementation for:

• short convolution
• chunked KDA prefill
• recurrent KDA decode
• KDA gate/output
• state gather/scatter

The initial goal would be correctness and end-to-end execution.

Phase 2: reuse existing CPU GDN kernels

Where the mathematical operations overlap with the existing CPU GDN backend, share or extend those implementations rather than creating duplicate infrastructure.

Phase 3: x86 optimization

Optimize the expensive GEMM/state operations with:

• AMX BF16
• AVX-512 BF16
• oneDNN / existing vLLM CPU kernels where appropriate

For Sapphire Rapids, AMX acceleration would be particularly useful.

Important correctness cases

KDA is stateful, so I think CPU tests should cover more than a single one-shot prompt.

Suggested tests:

Prefill equivalence

Compare:

one-shot prefill

against:

chunk 1
→ save state
→ chunk 2
→ save state
→ ...

The final output/state should match within expected numerical tolerance.

Prefill → decode

Verify that the state produced by chunked prefill can be consumed by recurrent decode without divergence.

Mixed batches

Verify batches containing both:

• ongoing decode requests
• new prefill requests

because these exercise different sequence/state layouts.

Prefix caching

Verify that restoring KDA state after a prefix-cache hit produces the same continuation as recomputing the prefix.

Request slot reuse

Verify that state belonging to a completed request cannot leak into a newly allocated request slot.

Suggested acceptance criteria

• GLM5Next KDA layers construct successfully on CPU.
• No CUDA/Triton/NVIDIA KDA operator is imported or invoked by the CPU path.
• Chunked prefill works.
• Recurrent decode works.
• Prefill-produced state is valid for decode.
• Multiple sequences in the same packed batch remain independent.
• Mixed prefill/decode batches work.
• Prefix caching/state restoration works.
• Output is numerically comparable to an existing NVIDIA/ROCm reference implementation.
• An optimized AMX/AVX-512 path can be selected on capable x86 systems.

Scope

This issue is specifically for KDA CPU execution.

GLM-5.3-Flash also requires CPU support for its NoPE sparse MLA / KeyPool indexer path; I am tracking that separately because the two features are independent.

Conceptually:

GLM-5.3-Flash
        │
        ├── KDA layers
        │       └── CPU backend needed  ← this issue
        │
        └── sparse MLA / KeyPool layers
                └── CPU backend needed  ← separate issue

Fixing either one alone is not sufficient for end-to-end GLM-5.3 CPU inference.

Would maintainers prefer implementing KDA by extending/reusing the existing CPU GDN backend, or keeping a separate GLM5Next/KDA CPU implementation first and sharing kernels later?

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.