[MXFP4 layerwise prefill] SM 8.6 (Ampere) excluded by capability whitelist — DSV4-Flash on dual A10 falls back to serialized full-GPU path

Summary

While evaluating KTransformers (DSV4-Flash Day-0 support) for a dual-NVIDIA-A10 (Ampere, SM 8.6) deployment, I noticed that the MXFP4 layerwise prefill pipeline is enabled only on Ada/Blackwell.

python/sglang/srt/layers/moe/kt_ep_wrapper.py (in the third_party/sglang submodule, branch main):

def _mxfp4_pipeline_backend_supported(method, layer: torch.nn.Module) -> bool:
    ...
    device = next(layer.parameters()).device
    return torch.cuda.get_device_capability(device) in ((8, 9), (12, 0))

SM 8.6 (A10/A30/A40/A6000/L40) is excluded, and there is no env override to opt in (the SGLANG_V4_USE_TRITON_KERNELS diagnostic only disables the pipeline).

Impact

On Ampere, when --kt-gpu-prefill-token-threshold > 0 (the DSV4-Flash doc uses 4096), a prefill request at/above the threshold enters the _full_gpu_gate branch, but since _mxfp4_pipeline_runtime_supported() is False it falls through to:

# Non-MXFP4 and unsupported MXFP4 backends retain the existing
# serialized full-GPU fallback.
ctx = self._build_full_context(layer)
...
result = ctx.gpu_method.apply(ctx.gpu_layer, dispatch_output)
if torch.cuda.is_available():
    torch.cuda.synchronize()

So each layer does: H2D-load all 256 experts -> compute -> hard sync -> next layer. That is a serialized per-layer transfer chain — the exact bottleneck we are trying to escape (see below). As the gate stands, Ampere deployments get no benefit from the layerwise pipeline.

Background / motivation

Our current DSV4-Flash deployment (vLLM + CPU-MoE backend, TP2 on 2x A10) plateaus at ~650 tok/s cold prefill, independent of prompt length (8k-64k). Thread-level diagnosis attributes it to a latency-bound serialized per-layer weight-transfer chain (1-3 runnable threads on the transfer side, zero threads blocked on bandwidth; GPU memory util 14-44% during prefill — i.e. the GPU idles waiting on H2D, not on compute).

We want to evaluate whether KTransformers' double-buffered layerwise prefill can break that plateau, but with the current capability gate we would land on the same serialized-chain shape on SM 8.6.

Questions

1. Is SM 8.6 excluded because the MXFP4 Marlin path is unverified/incorrect on Ampere, or simply untested?
2. Are there plans to enable the layerwise pipeline on Ampere, or to add an opt-in env override (e.g. SGLANG_V4_MXFP4_LAYERWISE_FORCE=1)?
3. If it is a verification gap: we can run the DSV4-Flash benchmark on our dual-A10 machine and report prefill throughput + correctness results. Please tell us which checks are needed to certify SM 8.6.

Environment

• GPU: 2x NVIDIA A10, 24 GB each, SM 8.6, PCIe 4.0 x16, no NVLink
• CPU: 2x AMD EPYC 7H12 (AVX2 only, no AVX-512/AMX)
• RAM: 512 GB
• Model: DeepSeek-V4-Flash (284B MoE, 13B active, 256 experts / top-6, native MXFP4, 43 layers, 160K context)
• KTransformers: approachingai/ktransformers:DSV4-specific / main branch