Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#853·openwiki

[Feature]: Enable prompt caching on the Bedrock path — cache_control is never set

Author: adi-shilo-sbCreated Sep 10, 2026Updated Sep 10, 2026

openwiki never requests Bedrock prompt caching, so long runs reprocess their whole stable prefix on every model call. On a real CI run against a mid-sized repo we measured 17,031,960 input tokens across 286 calls with 0 cache reads — about 59.5K input tokens per call, essentially all of it a re-sent prefix.

Measured on 0.5.0 and re-checked against 0.5.1: the Bedrock branch is unchanged, and the published dist contains no occurrence of cache_control, cachePoint or PromptCachingMiddleware.

The Bedrock branch of createModel (src/agent/index.ts:1217 on main) passes model, region, maxTokens, streamIdleTimeout and retry options, and no cache configuration:

javascript
if (provider === "bedrock") {
  return new ChatBedrockConverse({
    model: modelId,
    region: resolveProviderRegion(provider),
    ...maxTokensOptions, ...streamIdleTimeoutOptions, ...retryOptions,
  });
}

The createDeepAgent middleware array (src/agent/index.ts:486) carries only the translation, okf-index and repository-runner middlewares — bedrockPromptCachingMiddleware is never imported. With no cache_control supplied, @langchain/aws's applyCachePointsToConversePayload returns at its first line (if (!cacheControl) return;) and emits no cachePoint blocks. So zero cache reads is unconditional on Bedrock, for every model and every configuration.

Notably the OpenAI path does handle caching concerns (src/agent/openai-chatgpt-oauth.ts:72 manages prompt_cache_retention), so this looks like an oversight on the Bedrock path rather than a deliberate choice.

Related: #696 asks for the same thing on the Anthropic direct path and scopes itself out of other providers. This issue is the Bedrock half; the two are complementary.

Requested change: set cache_control on the Bedrock model settings — the installed @langchain/aws 1.4.5 already honours it (ChatBedrockConverse reads options.cache_control and translates it into Converse cachePoint blocks after the system prompt, the tool definitions and the final message). No dependency bump is needed.

Please prefer this over adopting bedrockPromptCachingMiddleware. That middleware (langchain, libs/langchain/src/agents/middleware/provider/aws/promptCaching.ts) decides whether a model can cache by substring-matching the model id:

javascript
modelId.toLowerCase().includes("anthropic.claude") || modelId.toLowerCase().includes("amazon.nova")

An application inference profile ARN — the documented way to attribute Bedrock cost per caller, arn:aws:bedrock:<region>:<account>:application-inference-profile/<id> — names a profile, not a family, so it matches neither. The middleware then takes the unsupported-model branch and, under the default unsupportedModelBehavior: "warn", emits only a console.warn, which is invisible in any non-TTY or buffered-output setting. We verified against Bedrock directly that this is a false negative and not a capability limit: application inference profiles cache exactly as system profiles do, and the cache is keyed by content rather than by profile (write via ARN → read via system profile hits, and vice versa). We intend to file that separately against langchainjs.

The direct cache_control route has no such gate: applyCachePointsToConversePayload consults the model id only to detect amazon.nova (to adjust cache-point placement), so any other id, ARNs included, gets the full Claude placement.

Happy to open a PR if the direction is agreeable.

Source: langchain-ai/openwiki

View original on GitHubView discussion on GitHub