Bedrock invoke-model: gateway replaces every chat's model with the provider row's settings.model, so model configs silently run (and are priced as) the wrong model
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
Under the default Bedrock protocol (invoke-model), the AI gateway replaces the model in every request body with the provider row's settings.model (or settings.small_fast_model when the requested id contains haiku). The chat's model_config_id decides only the display name and the price table. aibridge/intercept/messages/base.go Model() still does this on main today.
Consequences we measured on a production deployment (v2.36.5, 2026-09-17):
- A 6-hour Coder Agents delegation on a model config named "Claude Fable 5" (
global.anthropic.claude-fable-5, 1,453 steps) ran entirely on Sonnet 5. CloudWatchAWS/Bedrockmetrics byModelIdfor the whole day show a single dimension,global.anthropic.claude-sonnet-5; no fable/opus id exists as a metric at all. GET /api/experimental/chats/{id}/costreported $854 at the Fable config's prices. Real spend at Sonnet rates was ~$256. Every Opus- and Fable-labelled chat had been overbilled ~3.3× and nobody received the model they selected, since at least 2026-08-27.- Non-Anthropic configs on the same row (
openai.gpt-5.6-luna, which was our default,xai.grok-4.3) were also silently Sonnet 5. Nothing in the UI, the chat, or the cost endpoint reveals the substitution.
The only workaround on invoke-model is one Bedrock provider row per model whose settings.model equals the config's model (we now run three identical-looking rows: Sonnet, Opus 5, Fable 5.1). The mantle protocol passes the model through, but Mantle does not yet serve every model (no claude-fable-5-1, no gpt-6-astra as of 2026-09-18), so it cannot replace the legacy path.
Relevant Log Output
# CloudWatch AWS/Bedrock Invocations by ModelId, 2026-09-17 18:00Z – 2026-09-18 00:30Z
global.anthropic.claude-sonnet-5 Invocations 2003 (only dimension present)
# Coder cost for the Fable-labelled chat tree over the same window
{"total_cost_micros":854160585,"priced_message_count":1453}
# Provider row (no protocol → invoke-model)
"settings": {"_type":"bedrock","_version":1,"model":"global.anthropic.claude-sonnet-5",
"small_fast_model":"global.anthropic.claude-haiku-4-5-20251001-v1:0","region":"us-east-1","role_arn":"..."}Expected Behavior
The model a chat is pinned to is the model Bedrock is asked for, under every protocol. Concretely:
- On
invoke-model, send the config's model.settings.model/settings.small_fast_modelshould only be a fallback when the request carries no model (Claude Code'sANTHROPIC_MODEL/ANTHROPIC_SMALL_FAST_MODELuse case), never an override of an explicitmodel_config. The chat daemon always knows the config's model, so it can pass it through; thehaiku-substring heuristic should not apply to a request that already names a Bedrock inference profile. - Reject the mismatch instead of hiding it. If the gateway must substitute, a model config whose
modeldiffers from what the provider row will send should fail validation on create/update (the same place #29112 already resolves inference-profile ARNs), and the chat should surface the substituted model inmessages[].usage/ the cost endpoint so pricing follows the model that actually ran. - Price what ran. Cost should be computed from the model id the upstream response reports (Bedrock returns it), not from the config the chat was created with.
Any one of these would have made the problem visible; (1) makes it go away without per-model provider rows.
Steps to Reproduce
- One Bedrock provider (default protocol) with
settings.model = global.anthropic.claude-sonnet-5. - Two enabled model configs on it:
global.anthropic.claude-sonnet-5andglobal.anthropic.claude-opus-5(or any second Anthropic id), each with its own prices. - Create a chat pinned to the Opus config, send one prompt.
- Look at CloudTrail
InvokeModelWithResponseStreamresources[].ARNor CloudWatchAWS/BedrockbyModelId: the request went to Sonnet.GET /chats/{id}/costpriced it as Opus.
Environment
- Coder v2.36.5+28c4c83 (same code on
mainas of 2026-09-18,aibridgeintercept/messages/base.goModel()) - Provider type
bedrock,role_arnauth, region us-east-1, protocol unset (invoke-model) - Coder Agents chats created via
POST /api/experimental/chatswithmodel_config_id
Additional Context
- Workaround in use: one provider row per model (
bedrock,bedrock-fable,bedrock-opus) with matchingsettings.model. It works but is invisible in the model editor and nothing warns when a config points at a row that will send a different model. - #28404 (Mantle, non-Anthropic models) and #29230 (reasoning-model override) improve the Mantle path, but deployments that need a model Mantle does not list still depend on
invoke-model, where the substitution remains. - Related: #26734, #27361 (other cases of Bedrock model ids being reinterpreted silently).
Source: coder/coder