[Feature]: Enable prompt caching on the Bedrock path — cache_control is never set
openwiki never requests Bedrock prompt caching, so long runs reprocess their whole stable prefix on every model call. On a real CI run against a mid-sized repo we measured 17,031,960 input tokens across 286 calls with 0 cache reads — about 59.5K input tokens per call, essentially all of it a re-sent prefix.
Measured on 0.5.0 and re-checked against 0.5.1: the Bedrock branch is unchanged, and the
published dist contains no occurrence of cache_control, cachePoint or PromptCachingMiddleware.
The Bedrock branch of createModel (src/agent/index.ts:1217 on main) passes model,
region, maxTokens, streamIdleTimeout and retry options, and no cache configuration:
if (provider === "bedrock") {
return new ChatBedrockConverse({
model: modelId,
region: resolveProviderRegion(provider),
...maxTokensOptions, ...streamIdleTimeoutOptions, ...retryOptions,
});
}The createDeepAgent middleware array (src/agent/index.ts:486) carries only the translation,
okf-index and repository-runner middlewares — bedrockPromptCachingMiddleware is never imported.
With no cache_control supplied, @langchain/aws's applyCachePointsToConversePayload returns at
its first line (if (!cacheControl) return;) and emits no cachePoint blocks. So zero cache reads
is unconditional on Bedrock, for every model and every configuration.
Notably the OpenAI path does handle caching concerns (src/agent/openai-chatgpt-oauth.ts:72
manages prompt_cache_retention), so this looks like an oversight on the Bedrock path rather than
a deliberate choice.
Related: #696 asks for the same thing on the Anthropic direct path and scopes itself out of other providers. This issue is the Bedrock half; the two are complementary.
Requested change: set cache_control on the Bedrock model settings — the installed
@langchain/aws 1.4.5 already honours it (ChatBedrockConverse reads options.cache_control and
translates it into Converse cachePoint blocks after the system prompt, the tool definitions and
the final message). No dependency bump is needed.
Please prefer this over adopting bedrockPromptCachingMiddleware. That middleware
(langchain, libs/langchain/src/agents/middleware/provider/aws/promptCaching.ts) decides whether
a model can cache by substring-matching the model id:
modelId.toLowerCase().includes("anthropic.claude") || modelId.toLowerCase().includes("amazon.nova")An application inference profile ARN — the documented way to attribute Bedrock cost per caller,
arn:aws:bedrock:<region>:<account>:application-inference-profile/<id> — names a profile, not a
family, so it matches neither. The middleware then takes the unsupported-model branch and, under
the default unsupportedModelBehavior: "warn", emits only a console.warn, which is invisible in
any non-TTY or buffered-output setting. We verified against Bedrock directly that this is a false
negative and not a capability limit: application inference profiles cache exactly as system
profiles do, and the cache is keyed by content rather than by profile (write via ARN → read via
system profile hits, and vice versa). We intend to file that separately against langchainjs.
The direct cache_control route has no such gate: applyCachePointsToConversePayload consults the
model id only to detect amazon.nova (to adjust cache-point placement), so any other id, ARNs
included, gets the full Claude placement.
Happy to open a PR if the direction is agreeable.
Source: langchain-ai/openwiki