feat(sglang): add an SGLang PD adapter to the standalone decode sidecar
Author: tmonty12Created Aug 17, 2026Updated Sep 19, 2026
Labelsenhancementbackend::sglangrouterdep:implementingepp
Outcome
Make the standalone decode sidecar execute the supported raw-SGLang prefill/decode handoff after EPP selects a disaggregated plan.
The adapter forwards the original OpenAI Chat Completions request to the selected prefill worker and its local decode worker with one trusted SGLang bootstrap tuple. KV bytes transfer directly between SGLang workers; neither EPP nor the sidecar carries them.
Scope
- Require and version-gate a supported SGLang release whose
/v1/chat/completionsrequest accepts server-providedbootstrap_host,bootstrap_port, andbootstrap_room. - Receive the selected prefill bootstrap endpoint through the standalone disaggregated EPP-to-sidecar contract.
- Discard client-supplied bootstrap metadata, generate one trusted room ID, and inject the same bootstrap tuple into both prefill and decode requests.
- Preserve original OpenAI request semantics and return only the decode response stream to Gateway.
- Propagate prefill/decode errors, cancellation, and stream termination through the sidecar contract.
- Add focused protocol coverage for the supported SGLang request contract.
Out of scope
- Translation to SGLang
/generate; the supported path uses the native OpenAI Chat Completions endpoint. - KV-byte proxying through EPP or the sidecar.
- Paired EPP admission, worker discovery, manifests, benchmarks, or end-to-end deployment validation.
Dependencies
- Milestone 2: SGLang Aggregated Routing, including ai-dynamo/dynamo#13390.
- ai-dynamo/dynamo#13405 — decode-sidecar request contract.
- ai-dynamo/dynamo#13406 and ai-dynamo/dynamo#13407 — role-scoped worker catalogs and paired selection.
Tracking
- DEP: ai-dynamo/dynamo#11661
- Milestone: SGLang Disaggregated Routing
Source: ai-dynamo/dynamo