[Bug]: /vertex_ai/live cost/logging doesn't resolve model_group_alias

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

On /vertex_ai/live, when the client's setup.model field is a bare model_group_alias (e.g. "transcribe_live") instead of a full projects/.../locations/.../publishers/google/models/<id> resource path, the WebSocket relay itself works correctly — _build_vertex_live_setup_model_rewriter resolves the alias via llm_router.get_model_list() and forwards the right real model to Vertex, and transcription succeeds end to end.

However, the spend/logging code path is a separate function, _extract_model_from_vertex_ai_setup (litellm/proxy/pass_through_endpoints/pass_through_endpoints.py), which only recognizes a string containing "/models/":

if isinstance(model_path, str) and "/models/" in model_path:
    model_name = model_path.split("/models/")[-1]
    return model_name

A bare alias never matches this, so extraction returns None and kwargs["model"] never gets set for the call. That reaches a second function, VertexAILivePassthroughLoggingHandler.vertex_ai_live_passthrough_handler (llm_provider_handlers/vertex_ai_live_passthrough_logging_handler.py), which falls back to a hardcoded placeholder:

model: Final = kwargs.get("model", "gemini-2.0-flash-live-preview-04-09")

That placeholder isn't in litellm's model cost map, so cost calculation fails outright with This model isn't mapped yet, even though this same handler already extracted real usage tokens from Vertex's own usageMetadata frames -- the call succeeded, real tokens are already in hand, and cost still comes out $0 because of the wrong model string.

Expected: the cost/logging path resolves model_group_alias the same way _build_vertex_live_setup_model_rewriter already does (both live in the same file, right next to each other), since it uses the exact same llm_router and the exact same alias source of truth.

User Flow

Before a (hypothetical) fix: a proxy admin has registered a live-transcription model behind a model_group_alias (so the real Vertex model id stays out of client config, same as every other aliased model on the proxy), and their app's spend dashboard silently loses all cost data for it.

1. The proxy admin sets, in config.yaml:

   model_list:
     - model_name: vertex_gemini_live_transcribe__3_5
       litellm_params:
         model: vertex_ai/gemini-3.5-transcribe-live-preview
         vertex_credentials: "os.environ/GCP_SERVICE_ACCOUNT_JSON"
         vertex_location: "global"
   
   router_settings:
     model_group_alias: {
       "transcribe_live": "vertex_gemini_live_transcribe__3_5"
     }
   

2. Their app opens wss://litellm-domain/vertex_ai/live?model=transcribe_live&vertex_location=global and sends a setup frame with "model": "transcribe_live".
3. Audio streams both ways, the app receives real transcription results back from Vertex — the call fully succeeds.
4. The admin opens https://litellm-domain/ui/?page=logs and sees the request logged with Model: unknown, Cost: - (blank/$0), even though real Vertex usage was billed on Google's side.

After a (hypothetical) fix: the same setup, but the dashboard reflects real spend.

1. Same config.yaml as above, no changes needed on the admin's side.
2. Their app opens the same URL and sends the same setup frame with "model": "transcribe_live".
3. Same successful audio round-trip.
4. https://litellm-domain/ui/?page=logs now shows Model: gemini-3.5-transcribe-live-preview, Cost: a real non-zero dollar amount, with real prompt/completion token counts — the log accurately reflects what happened.

Proof the bug occurs

Version / commit: v1.100.0-rc.1 (commit 10631eb834c7802aa61611e807474170b8a4d425)

Config the proxy ran with (config.yaml, trimmed to the relevant model only):

model_list:
  - model_name: vertex_gemini_live_transcribe__3_5
    litellm_params:
      model: vertex_ai/gemini-3.5-transcribe-live-preview
      vertex_credentials: "os.environ/GCP_SERVICE_ACCOUNT_JSON"
      vertex_location: "global"

router_settings:
  model_group_alias: {
    "transcribe_live": "vertex_gemini_live_transcribe__3_5"
  }

Env vars: GCP_SERVICE_ACCOUNT_JSON set to a real GCP service account with Vertex AI access (redacted).

Source-level confirmation (independent of the log evidence below): the two code paths were traced directly against this commit.

• The alias is resolved for the outgoing frame, in _build_vertex_live_setup_model_rewriter / _resolve_alias_to_upstream_model (llm_passthrough_endpoints.py), via llm_router.get_model_list(), which the function's own docstring says "Includes router model_group_alias'es as well."
• The alias is not resolved for cost/logging, in _extract_model_from_vertex_ai_setup (pass_through_endpoints.py), which only branches on "/models/" in model_path and returns None otherwise — never consulting llm_router at all, even though it's called from the same websocket_passthrough_request function that already has llm_router in scope one call away.

Live proxy run, real Vertex call, real log output:

Request: client connects to wss://<litellm-host>/vertex_ai/live?model=transcribe_live&vertex_location=global, sends a real setup frame with {"setup": {"model": "transcribe_live", ...}}, streams a real audio sample, and receives a real transcript back (call succeeds end to end — this is not a connection failure).

Resulting /ui/?page=logs entry for that exact request:

Time	Status	Cost	Duration (s)	Model	Tokens
(real request timestamp)	Success	- (i.e. $0)	2.55	unknown	0(0+0)

For comparison, the identical request/config, with only _extract_model_from_vertex_ai_setup locally patched to fall back to _resolve_alias_to_upstream_model when there's no /models/ substring, produces the correct entry for the same live call:

Time	Status	Cost	Duration (s)	Model	Tokens
(real request timestamp)	Success	$0.000462	2.55	gemini-3.5-transcribe-live-preview	97(90+7)

Both rows are from real, successful, live Vertex calls with identical request payloads and identical config — the only difference is whether _extract_model_from_vertex_ai_setup can resolve the alias.

Suggested fix: give _extract_model_from_vertex_ai_setup an optional llm_router parameter, and when model_path doesn't contain "/models/", fall back to resolving it via _resolve_alias_to_upstream_model (already defined in the neighboring file) before giving up. Both of _extract_model_from_vertex_ai_setup's call sites in pass_through_endpoints.py already have easy access to llm_router via from litellm.proxy.proxy_server import llm_router, the same way _build_vertex_live_setup_model_rewriter obtains it via _get_llm_router().

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.100.0-rc.1

Twitter / LinkedIn details

No response