#1002·morphic

The chat cache is never invalidated by the assistant answer

Author: miurlaCreated Aug 27, 2026Updated Aug 29, 2026
Labelsautomated

Problem

loadChat (lib/actions/chat.ts) is wrapped in unstable_cache(..., { tags: ['chat-<id>', 'chat'], revalidate: 60 }) and depends on revalidateTag('chat-<id>', 'max') inside upsertMessage to invalidate it. For the assistant answer, that invalidation never runs, so any reader going through the cached path can be served a snapshot that predates the most recent answer.

#1001 changes the one reader where this reaches the model (prompt construction on the streaming path). The other readers are unchanged and still exposed:

  • app/search/[id]/page.tsx renders a chat from loadChat, so a page load can render a thread whose most recent answer is missing.
  • app/api/chat/route.ts computes conversationTurn for analytics from loadChat inside a floating promise. Besides being able to undercount thread depth, this is the read that repopulates the cache entry in the middle of a turn, which is how a pre-answer snapshot becomes the cached value at all.

Evidence

Mechanism, all visible in code:

  • revalidateTag() does not invalidate on the spot. It pushes the tag onto workStore.pendingRevalidatedTags (next/dist/server/web/spec-extension/revalidate.js).
  • That queue is flushed by resolvePendingRevalidations() in next/dist/server/route-modules/app-route/module.js, which runs immediately after the route handler returns its Response.
  • The chat route returns a streaming Response, and the assistant answer is persisted afterwards in the onEnd callback of toUIMessageStreamResponse (persistStreamResults -> upsertMessage). Its revalidateTag is queued into a store that will not be flushed again.
  • unstable_cache does not block on a stale entry outside static generation: it returns the stale value and schedules the revalidation in the background, so the first reader after expiry still sees the old snapshot.
  • The user message does not have this problem, because it is upserted in prepareMessages before the handler returns. That asymmetry is why only the assistant side is ever dropped.

How to reproduce this observation

Send a turn, wait for the answer to finish, then load the chat page. Compare what is rendered against the rows in messages for that chat_id. The persisted ordering alternates user/assistant; a render that stops at the user message is the cached snapshot, not the database.

Proposed direction

  1. Drop the unstable_cache wrapper from loadChat. Chat history is per-user mutable data read roughly once per navigation, and the entry is already invalidated by every user message, so little hit rate is being given up. This removes the class rather than the instance.
  2. Keep the cache but stop reading it from the request that also writes: give the analytics turn calculation an uncached read, or derive the depth without a chat read at all, so a mid-turn read cannot install a pre-answer snapshot.
  3. Keep the cache and give the answer write an invalidation path that does not depend on the request's pending-revalidation queue.

Item 1 is the small one and is worth doing on its own. Items 2 and 3 are the judgment calls, and 3 in particular should not be adopted without deciding what replaces revalidateTag for post-response writes.

Acceptance criteria

  • A chat page loaded right after an answer completes renders that answer, without needing a second load.
  • No read used to build a prompt, or to compute conversation depth, can return a snapshot older than the last committed message for that chat.
  • No new invalidation path relies on work queued after the route handler has returned its Response.

Refs #1001.