Inline image spec blocks are rarely emitted: domain-restricted searches plus unfollowed image instruction

Problem

Answers rarely embed inline image groups any more. The INLINE IMAGE EMBEDDING section of the render prompt instructs the model to emit an image spec block whenever the search tool returns images, but on the currently configured model that instruction is mostly not followed.

The drop has two independent causes, and they need separate fixes:

1. Supply: the model now restricts almost every search to specific domains, which makes the search provider return no images array.
2. Emission: even when images are present in the tool output, the model usually does not emit the image spec block.

Evidence

Measured on persisted assistant messages in adaptive mode, split at the two relevant deploys. "Images available" means the search tool output has a top level non-empty images array.

Emission, given images are available

Conditional emission, i.e. the share of answers that embed an image spec block among those where the tool output actually carried images, fell sharply at the model switch. The related-questions fix later recovered a small part of it as a side effect, but most of the gap remains open. The residual is large, not marginal.

Supply, per search tool call (adaptive)

Over the same split, the share of adaptive search calls that pass include_domains went from negligible to the clear majority, and search_depth: advanced rose alongside it. The share of calls that come back carrying images fell correspondingly.

Domain restriction suppresses images, and the effect holds within each period, not just across them: in both the before and after windows, calls that used include_domains returned images less often than calls that did not. So this is a causal-shaped effect rather than an artifact of comparing two periods.

getSourceDirectionGuidance() in lib/agents/prompts/search-mode-prompts.ts already says to apply domain filters only when the user's intent clearly points to a source, and not to invent restrictions for ordinary queries. That instruction is now being ignored for the majority of searches.

How to reproduce these measurements

Both halves are measurable from persisted data without any special tooling:

• Supply: read include_domains and search_depth off the search tool call inputs, and check whether the corresponding tool output carries a top level images array. This needs no image-emission oracle, so it can be validated on its own.
• Emission: among assistant answers whose search output carried images, check whether the answer text contains an image spec block.

What has been ruled out

• Plumbing. The images array does reach the model. Asked to list the image URLs from the tool output verbatim, the model reproduces them exactly, so toModelOutput in lib/tools/search.ts is not dropping them.
• Prompt assembly. getAdaptiveModePrompt() contains the INLINE IMAGE EMBEDDING section.
• Crowding out by the related-questions block. Removing the related-questions section from the system prompt entirely does not restore image emission in probe runs.
• Wording strength alone. Rewriting the image section mandate-first, in the same shape as the related-questions fix, does not restore it. Appending a blunt "you MUST emit an image spec block" override at the very end of the system prompt also does not.

Why this is an issue and not a PR

No reliable local reproduction exists yet. A probe using a synthetic image-bearing search fixture fails to emit images for the previously configured model as well, which contradicts that model's known-good behavior in production. The probe harness is therefore not a valid oracle, and any prompt change validated against it would be unverified. A faithful replay that feeds a real recorded search payload back to the model is the missing piece.

Note on measurement

Detecting image supply with a substring match for "images":[{ over the serialized tool output overcounts, because the same substring occurs nested inside result content. Only a minority of substring matches correspond to a genuine top level images array. Use a position-anchored check instead, comparing the offset of "images":[{ against the offset of "results":, which was validated against real JSON parsing on a sampled set. Earlier measurements using the naive substring understate the conditional emission rate and wrongly suggest that image supply went up after the switch, when it actually went down.

Proposed direction

Treat the two halves separately.

Supply half (smaller, more tractable):

• Tighten getSourceDirectionGuidance() so domain restriction requires the user to have named a source, and probe the resulting include_domains rate directly. That rate is measurable from tool call inputs and does not depend on the image-emission oracle, so this half can be validated.

Emission half:

• First build a faithful replay harness: record real search tool payloads and replay them to the model with the production prompt and provider options, and confirm the harness reproduces the known before and after behavior for both models. Without that control, prompt edits cannot be verified.
• Only then re-tune the image instruction.

Acceptance criteria

• A replay harness reproduces the historical per-model emission behavior closely enough to be used as a regression oracle.
• include_domains is applied only when the user named a source, verified on tool call inputs.
• Conditional image emission in adaptive mode recovers toward the pre-switch level, measured with the position-anchored supply check. The target level is tracked internally rather than stated here.