Track `ChatAnthropicVertex` prompt-caching parity and provider consolidation
Requested by @sydney-runkle. Tracker for the missing automatic prompt-cache setup when Deep Agents is given langchain_google_vertexai.model_garden.ChatAnthropicVertex. This does not propose closing either linked PR.
Observable gap
Deep Agents currently appends its prompt-cache stack to the main agent, the automatic general-purpose subagent, and declarative inline subagents. Its always-present AnthropicPromptCachingMiddleware only accepts isinstance(request.model, ChatAnthropic). ChatAnthropicVertex is a distinct BaseChatModel, so the middleware is invoked with unsupported_model_behavior="ignore" and silently passes the request unchanged.
Consequently, an Anthropic Claude model reached through Vertex does not receive the automatic system/tool annotations or model_settings["cache_control"] that direct ChatAnthropic receives. The immediate effect is missing automatic cache-control payloads, which can increase repeated-prefix input processing cost. It is not a promise that every request would otherwise be a cache hit: provider eligibility, minimum-size rules, a stable matching prefix, and cache lifetime still decide that.
The adapter path is ready: langchain-google #1631 is merged and has ChatAnthropicVertex._format_params() remove cache_control from top-level parameters and add it to the final formatted message content block, for sync, async, and streaming paths. That makes the following a temporary opt-in workaround for callers able to pass the model option (or an equivalent custom wrap_model_call middleware that sets model_settings["cache_control"]):
cache_control = {"type": "ephemeral", "ttl": "5m"}
model.invoke(messages, cache_control=cache_control)Use a released langchain-google-vertexai containing #1631. This workaround targets the trailing message breakpoint; it is not a replacement for Deep Agents' full automatic system/tool handling.
Public-source audit
| Runtime | class/function and package | Provider/model gate and behavior | Status / source |
|---|---|---|---|
| Python | AnthropicPromptCachingMiddleware — langchain-anthropic |
Exact ChatAnthropic; tags the final system block and tool, and sets model_settings.cache_control; sync + async. |
Current and always wired by Deep Agents. source · Deep Agents wiring |
| Python | BedrockPromptCachingMiddleware — langchain-aws |
ChatBedrock / ChatBedrockConverse, Anthropic or Amazon Nova model identifiers; passes cache_control for provider-side conversion; sync + async. |
Current, optional when langchain-aws imports. source · Deep Agents wiring · wiring PR #4108 |
| Python | FireworksPromptCachingMiddleware — langchain-fireworks |
Deep Agents lazily imports it and passes unsupported_model_behavior="ignore". |
Current, optional wiring. The class/package is referenced in the helper, but a bounded public GitHub code/repository search did not locate the package source, so its model gate and payload transformation are deliberately not inferred here. |
| Python | VertexPromptCachingMiddleware — proposed langchain-google-vertexai |
Exact ChatAnthropicVertex; mirrors the Anthropic strategy: system/tool annotations plus model_settings.cache_control, sync + async. |
Proposed, not merged or released in langchain-google #1993. Deep Agents wiring is draft/open, dependent on it, in deepagents #6318. |
| JavaScript | anthropicPromptCachingMiddleware — langchain |
ChatAnthropic or Anthropic ConfigurableModel; sets modelSettings.cache_control; runtime context options. |
Current JS implementation. source |
| JavaScript | bedrockPromptCachingMiddleware — langchain |
ChatBedrockConverse / Bedrock or AWS ConfigurableModel plus Anthropic Claude or Amazon Nova identifier; sets modelSettings.cache_control; runtime context options. |
Current JS implementation. source |
No public JS Vertex or Fireworks prompt-caching middleware was found in the current langchainjs middleware tree. Gemini is intentionally not treated as the same adapter issue: Mason's consolidation proposal notes its out-of-band CachedContent shape, whereas OpenAI prefix caching is automatic.
Requested tracking / acceptance criteria
Link this as a provider follow-up to #3239 — Generalize caching, the open consolidation issue authored by @mdrxy (Mason Daugherty). Its stated direction is a neutral PromptCachingMiddleware with provider dispatch, deprecated Anthropic alias, explicit supported-provider documentation, and no semantic change on the initial Anthropic path. This issue supplies the ChatAnthropicVertex case rather than broadening #3239 without review.
A complete solution should:
- integrate the Vertex handler through the neutral/provider-dispatch design rather than add another permanent ad-hoc gate;
- preserve existing Anthropic, Bedrock, and Fireworks behavior and make support/no-op behavior explicit;
- apply the same behavior to the main agent, automatic general-purpose subagent, and declarative inline subagents;
- cover synchronous and asynchronous calls, including the adapter's actual request payload and returned usage metadata where available;
- retain optional imports: Deep Agents must not gain hard dependencies on AWS, Fireworks, or Vertex integrations, and an unavailable or older optional integration must no-op without masking unrelated import errors;
- test a real formatted Vertex request for a content-block
cache_controlmarker (not only a middlewareModelRequest), plus an unsupported-model no-op; and - document that automatic markers optimize eligible matching prefixes, not guaranteed provider cache hits.
Related public work: langchain-google #1993 and deepagents #6318, both open at filing. Hunter Lovell (@hntrl) authored those PRs.
Source: langchain-ai/deepagents