#14640·dynamo

fix(ext-proc): select standalone cache-salt forwarding by tokenizer backend

Author: tmonty12Created Sep 10, 2026Updated Sep 19, 2026
Labelslanguage::rustbackend::vllmbackend::sglangrouterepp

Problem

Standalone EPP resolves an effective cache_namespace from the request body and headers, then uses it for KV-aware selection.

Today, its PickResult unconditionally uses the native-vLLM forwarding policy. That causes ext-proc to inject vLLM's top-level cache_salt extension (dynamo-cache-salt:<namespace>) even when standalone EPP uses the SGLang tokenizer/renderer.

SGLang does not currently have a compatible cache-salt request and KV-event contract. Forwarding the vLLM extension to it is therefore incorrect.

Desired behavior

Derive cache-salt forwarding from the standalone tokenizer/renderer protocol:

  • VllmRender -> CacheSaltForwarding::NativeVllm
    • Inject the resolved, Dynamo-prefixed top-level cache_salt for raw vLLM.
  • SglangRenderer -> CacheSaltForwarding::NativeSglang
    • Keep the resolved namespace for Dynamo selection only.
    • Do not inject or overwrite cache_salt in the forwarded request body.
  • Dynamo runtime continues using CacheSaltForwarding::Preserve; downstream workers resolve and tag the original cache-salt inputs.

This relies on the standalone configuration invariant that the tokenizer/renderer protocol matches the registered worker backend.

Scope

This does not add end-to-end cache-salt isolation for raw SGLang. It prevents EPP from sending vLLM's extension to SGLang while preserving correct namespace-aware selection. SGLang cache-salt forwarding and matching KV-event representation should be investigated separately.

Acceptance criteria

  • Add CacheSaltForwarding::NativeSglang.
  • Select the forwarding policy from RendererProtocol when constructing standalone EppRouter.
  • inject_body_extensions receives a cache salt only for NativeVllm.
  • Add focused coverage for the renderer-protocol-to-forwarding-policy mapping.