#13413·dynamo

feat(sglang): add concurrent prefill/decode dispatch to the standalone decode sidecar

Author: tmonty12Created Aug 17, 2026Updated Sep 19, 2026
Labelsenhancementbackend::sglangrouterdep:implementingepp

Outcome

Have the standalone SGLang decode sidecar start the selected prefill and local decode legs concurrently once it has constructed the trusted bootstrap tuple, then return the decode stream to Gateway.

Scope

  • Dispatch the selected prefill and local decode requests concurrently using the SGLang PD adapter.
  • Preserve the decode response as the only client-visible stream.
  • Coordinate prefill failure, decode failure, client cancellation, normal completion, and stream teardown with the paired reservation lifecycle.
  • Add focused coverage for concurrent dispatch and each terminal path.

Out of scope

  • SGLang bootstrap-tuple construction and request adaptation.
  • Paired EPP admission, worker discovery, manifests, benchmarks, or end-to-end deployment validation.
  • Multi-EPP disaggregated replication and recovery.

Dependencies

  • ai-dynamo/dynamo#13412 — SGLang PD adapter.
  • ai-dynamo/dynamo#13405 and ai-dynamo/dynamo#13407 — shared decode-sidecar contract and paired selection/reservation lifecycle.

Tracking

  • DEP: ai-dynamo/dynamo#11661
  • Milestone: SGLang Disaggregated Routing