#313 Incremental prepare: prototypes, numbers and open decisions

This issue collects the incremental preparation work so someone can pick it up cold. It's shelved until the markdown chat work is done. No prototype code is merged. Four bugs on main found during this work landed separately as #269, #270, #271 and #272.

Checked against main at 5810820 (#311) on 2026-09-15. The prototypes were built on 477510e (#256, 2026-09-13), and all of them conflict with main now.

References: main:path:N is a line at 5810820, 477510e:path:N a line at the prototype base, and <sha>:path:N a line on a branch commit.

Terms

Handle: what prepare() or prepareWithSegments() returns: flat arrays with one entry per segment.
Engine profile: Blink, WebKit, Gecko or Android behaviour, chosen once per process from the user agent.
Simple and complex walkers: the two line-walking loops in src/line-break.ts. walkPreparedLinesSimple (main:src/line-break.ts:311) runs when simpleLineWalkFastPath is set (:507).
Generation: a counter the prototypes add to clearMeasurementCaches(), so a kept result can tell that caches were cleared since. Main has none.
Canary: a copy of the source with one check deliberately broken. The test harness must fail on it.
Browser gate: bun run test:wrapping --browser=all, the wrapping suite run in installed Chrome, Safari and Firefox.
Owner prototype: an older unmerged commit, 940438b (branch incremental-owner-prototype), whose createIncrementalPreparer() returns a mutable owner object that hands out prepared snapshots (3.7).

Summary

prepare() redoes whole-text analysis on every call. A keystroke costs about 1 ms per 10k characters of English, Arabic or Thai, and ten times that for Japanese. Streaming re-prepares the message per token, so the total grows with the square of its length.
Three prototypes, all exactly equal to a fresh prepare() under a fake Canvas:
- Paragraph pattern (incremental-pieces): no new API. Prepare each pre-wrap line, or each markdown block, on its own. Fast. Recommended as documentation.
- Item reuse (incremental-rich-memo): prepareRichInline(items, previous?). 10-17× faster at 500 items, but 2.3-4.1% slower for callers who don't pass previous. Parked.
- Edit window (incremental-prepare-edit): prepare(…, { editable: true }) + prepareEdit(previous, text). 12-41× faster on one long paragraph, but its slowest 10% of keystrokes missed the 1 ms bar, and it adds 2.47 kB gzip and 1.5× handle memory. Parked.
An older unmerged prototype, createIncrementalPreparer(font, options?) (incremental-owner-prototype, 940438b), measured 3.6× on an 18.2k-character stream in Chrome 152, with thin exactness evidence.
Decided: no explicit edit ranges; eager vs lazy preparation is up to callers; no append API (#153).
Open: the target workload; where to document the paragraph pattern; whether results may change in place; whether the harness scripts go in the repo.

1. Problem

prepare() has two phases, and both run over the whole text:

Analysis: normalize whitespace, run Intl.Segmenter, run the merge passes, build the arrays.
Measurement: measure segments with Canvas. Only per-segment facts are cached, keyed by font and segment text (main:src/measurement.ts:132).

Where a warm keystroke's time goes:

0-2 Canvas calls.
analyzeText() is 70-80% of a 100k-character keystroke for English, mixed app text, Arabic and Thai. For Japanese it's 45%, plus about 50% in splitting CJK text into units inside measurement.
layout() is 0.5-1.6% of a warm prepare.

Warm keystroke, white-space: normal, ms:

text	1k chars	10k	100k	share of a 16.7 ms frame at 100k
en	0.106-0.622	1.16-1.21	16.6-17.2	104%
mixed app text	0.162-0.185	1.42-2.37	16.8-17.4	105%
ja	0.851-0.912	11.6-12.4	116-123	738%
ar	0.130-0.134	1.38-3.94	17.0-17.5	106%
th	0.211-0.224	2.17-2.29	23.5-24.4	147%

Cost is linear: 0.11-0.24 µs per character for en/mixed/ar/th, about 1.2 µs for ja. JavaScript alone fills a frame at about 95-104k characters for en/mixed/ar, 70k for th and 14k for ja.

Streaming a 10k-character message in 4-code-point tokens, re-preparing the whole message per token:

text	whole stream	ms per token, last 100 tokens
en	1,606 ms	1.16
mixed	1,897 ms	1.42
ja	15,073 ms	12.7
ar	1,750 ms	1.41
th	2,662 ms	2.11

A later benchmark run measured en at 2.00 s and ja at 16.5 s.

Rich inline: changing one item re-prepares every item. That's 2.07 ms (en) and 3.12 ms (mixed) per keystroke at 500 items, and 0.197-0.310 ms at 50.

Caveats for every number in this issue.

Node v23.10.0 or Bun 1.4.0 with a fake Canvas whose measureText() returns text.length * 8, on one Apple M5 Max on AC power.
"Safari" and "Firefox" rows only switch the engine profile through the user agent. They still run on V8.
No browser timing was done for the three prototypes. Treat the numbers as a floor for slower devices.
The benchmark scripts behind these numbers aren't committed. The exactness harnesses are on the branches.

2. Workloads

Streaming chat

#153 (closed, not planned) asked for prepareStream(initialText, font, options) + appendPreparedText(prepared, appendedText), for chat messages that stream token by token, terminal and log panes, live transcripts and collaborative surfaces. First version: append-only, same font and options.

Markdown can restructure earlier output as tokens arrive:

a setext === turns the paragraph above into a heading, in another font;
a table delimiter row turns the line above into a header;
an unclosed code fence swallows later text until it closes;
a closing ** splits one run into three;
a list marker narrows the block;
a reference definition that arrives later turns earlier text into a link, in another font, while that text's markdown is unchanged.

So a streaming chat needs to: append to the last item of the last block, replace the last block, replace a range of items inside a block, and occasionally change an earlier block's font or kind.

The markdown chat demo prepares messages from parseMarkdownBlocks(spec.markdown) at load (main:pages/demos/markdown-chat.model.ts:295, 411). It has no streaming path and no reuse parameter. Draft #312 prepares a window of message chunks around the viewport; that isn't streaming.

Editing

Textarea auto-height: one prepare(value, font, { whiteSpace: 'pre-wrap' }), of which only the height is used (main:README.md:33-38). Edits: insert and delete at the caret, paste, IME composition, autocorrect, select-all replace, undo.
Custom and rich editors: #173 (rich inline with pre-wrap), #151 (selection), #198 (character coordinates), #90 (source offsets). Edits add paragraph split and merge, multi-cursor, find/replace, drag and drop, and splitting or merging items when styling.
Collaborative edits: only #153 lists them. Remote splices at arbitrary offsets, batched per message.
On 2026-09-11 the maintainer asked for a plain-text incremental prepare whose later calls are basically a diff against the previous version, for live editing where most chunks stay the same.
Editors also need which lines changed, source offsets through normalization, caret x, hit testing, up/down with a goal x, and selection boxes. None exists. Source offsets (#90) come first for any API that takes positions (main:ENGINE_FOLLOWUPS.md:9).

Many paragraphs

In pre-wrap, \n resets every analysis and measurement rule that reads neighbours, once text-wide checks stop crossing newlines. So preparing each line separately can equal preparing the whole text.
Markdown blocks are separate CSS blocks, so a block's lines never depend on another block. The chat demo prepares 10,000 messages at startup.
Virtualized lists call prepare() many times, so state kept on every handle costs memory. #255 had just reduced it.

"Streaming" means something else here

In this repo "streaming" usually means line-by-line layout (layoutNextLine(), layoutNextRichInlineLineRange()): main:README.md:133, 160, main:TODO.md:6, main:RESEARCH.md:397, 972, main:CHANGELOG.md:65, and #13, #129, #140, #221, #222. None of these is about preparing text that keeps arriving.

What main's docs say

Nothing about incremental preparation. README, DEVELOPMENT, TODO, ENGINE_FOLLOWUPS, RESEARCH and CHANGELOG have no paragraph recipe and no parked entries.

main:README.md:31: "Do not rerun prepare() for the same text and configs".
main:README.md:211 describes clearCache() only as releasing memory. Nothing says to call it after a web font loads.
main:README.md:212 says setLocale() doesn't affect existing states ("no mutations to them"). That's the only place README speaks about handles changing.

3. What was tried

How the prototypes were checked

Every incremental result had to deep-equal prepare() or prepareWithSegments() of the final text under the same caches. Layout outputs, previously returned handles and Canvas call counts had to match too.
One child process per engine profile, since the profile is fixed per process.
A deterministic, counting fake Canvas, with width tables that switch at simulated font loads.
Canaries that must fail, and an independent review that read the diff.
Serial timing: keystroke suites with 2 warmup and 21 recorded interleaved rounds; A/B runs of 15 rounds × 20 samples for callers who don't use the new path; bundle sizes from bun build. Kill criteria were set before building.

Side by side

	Paragraph pattern	Item reuse	Edit window	Owner prototype
Branch	`incremental-pieces` `06a25fd`	`incremental-rich-memo` `c4a5ee4`	`incremental-prepare-edit` `8ab9d5c`	`incremental-owner-prototype` `940438b`
Public API	none; a README recipe	`prepareRichInline(items, previous?)`	`{ editable: true }` + `prepareEdit(previous, text)`	`createIncrementalPreparer(font, options?)` → `{ replace, append, clear }`
Reuses	whole `pre-wrap` lines; whole markdown blocks	each unchanged item's own preparation	everything outside a window between clean separators	analysis before a seam
Redoes	changed lines or blocks	every fact that spans items	the window, plus whole-text counts, chunks and array copies	the whole suffix from the seam, and measurement over the whole text
Handles	immutable	immutable	immutable; side state in a WeakMap	immutable snapshots from a mutable owner
Library size	+30/−4 lines	+159/−33 lines	395 engine lines; layout.ts +43/−9	+919/−266 over 20 files, with benchmarks

Compare views: incremental-pieces, incremental-rich-memo, incremental-prepare-edit, incremental-pieces-nosimple, incremental-owner-prototype.

3.1 Paragraph pattern (`incremental-pieces`): recommended, no API

What it promises. In pre-wrap, split the text after every \n, keeping the \n on each line. The lines prepare into handles whose line counts add up, and their lines equal the whole text's lines, with segment indices offset by earlier lines.

README proposal (commit 06a25fd, titled as a proposal):

For pre-wrap text that changes as you type or stream, prepare each line separately, keeping its \n, and add up the line counts. The lines match preparing the whole text, so only changed lines need prepare() again:

let handles = new Map<string, PreparedText>()

function textareaHeight(value: string, width: number): number {
  const next = new Map<string, PreparedText>()
  let lineCount = 0
  for (const line of value.split(/(?<=\n)/)) { // each line keeps its '\n'
    const prepared = next.get(line) ?? handles.get(line) ?? prepare(line, '16px Inter', { whiteSpace: 'pre-wrap' })
    next.set(line, prepared)
    lineCount += layout(prepared, width, 20).lineCount
  }
  handles = next
  return Math.max(1, lineCount) * 20
}

Drop the handles after changing the font or options, <html lang> or setLocale(), and after calling clearCache() when a web font loads. With prepareWithSegments(), a position is the line's index plus a LayoutCursor inside that line. Markdown blocks, paragraphs and chat messages are separate blocks anyway: prepare each on its own and prepare again only the ones whose text or style changed.

Library fixes it needed. At 477510e three results crossed \n: WebKit's explicit bidi control check read the whole text; the simple and complex walkers ended an overflowing line differently; Gecko's numeric affix check read a newline or space as a mark's base. All are on main now (section 4.9).

Chat block reuse (06a25fd:pages/demos/markdown-chat.model.ts, +29/−9). parseMarkdownBlocks(markdown, previous = []) re-lexes the whole message on each token and reuses a block's prepared result when its preparation inputs equal an earlier block's. The key is JSON.stringify(items) over { text, font, break, extraWidth } for inline blocks, and JSON.stringify(['code', source]) for code. Keying on raw markdown isn't exact, because of late reference definitions.

Exactness. scripts/pieces-differential.ts (982 lines) uses two width tables that switch at simulated font loads, a 4px emoji correction, and <html lang> cycling through root, ja and ko.

At scale 1: 11,858-13,246 whole-text comparisons and 206,653-223,286 width checks per profile, 0 differences.
One Gecko probe made 5 Canvas calls against 3 (1 of 308). That's a cost-only cache key issue on main (section 4.9, item 6).
Every canary failed as it should: main's source, each fix alone, a split that drops \n, and stale handles across a font or language change.
bun test: 249 pass.

Keystroke at 100k characters, pre-wrap, ms:

text	generic split	README Map recipe	app knows the edited line	whole prepare
en	0.427-0.476	0.536-0.579	0.017-0.048	16.0-16.9
mixed	0.400-0.452	0.490-0.534	0.020-0.041	16.9-17.9
ja	0.819-8.97 (alone 0.909-1.012)	0.827-8.90 (alone 0.890-0.962)	0.104-0.179	114-126
ar	0.436-0.487	0.534-0.574	0.019-0.049	17.4-18.0
th	0.427-0.552	0.514-0.588	0.028-0.101	23.2-24.4

The ja 8.97 ms step showed up only in the full suite, with no known cause.
For ja, laying out all 611 lines (0.52 ms) costs more than preparing the changed line (0.08-0.14 ms).

Other numbers.

Callers who don't use it: 0 flagged of 56 Chrome, 10 Safari-profile and 10 Firefox-profile cells. prepare()/prepareWithSegments() 0.930-1.040×, layout() 0.953-1.027×.
First render at 100k: 1.03-1.09× a cold whole prepare with the same Canvas calls, but ctx.font is set 331-611 times instead of once. The fake Canvas makes that free; the real cost is unknown.
10k-character stream in 4-code-point tokens, total: generic split 0.103-0.518 s, append-aware split 0.045-0.407 s, README Map 0.117-0.547 s, whole prepare 2.00-16.5 s.
The README Map at 100k retains 1.10-1.14× one whole handle.
Size, all from the fixes: +141 B gzip on the layout entry, +145 B on rich-inline.

Review findings not fixed on the branch. Negative maxWidth broke the promise; main now lays it out like 0 (#272). The walker fix changes rich-inline line counts, which the branch CHANGELOG contradicted. The Gecko mark fix also covers + and \ and removes some emergency breaks. The branch's size figures included gzip header names.

Verdict. Recommend as a documented pattern. No API.

3.2 Deleting the simple walker (`incremental-pieces-nosimple`, `08c9e5f`): rejected

Routing every API through the complex walker (line-break.ts +7/−343) made layout() 1.974-2.021× slower on all 4 simple-path documents, and cold prepareRichInline en/500 1.312× slower. It isn't only a speed choice: at 477510e the complex walker moved the space after an overflowing first word to the next line, so walkLineRanges() differed from the patched walkers on incremental-pieces in 4,574 of 84,000 width runs on handles that had used the simple walker. Line counts never differed.

3.3 Chat block reuse: optional demo change

Streamed markdown messages, marked lexer, one layout() per token:

message	whole re-prepare	demo reusing blocks	demo from scratch	edit window on the text
en 10k, many blocks	1,844 ms	457 ms	2,122 ms	106 ms
ja 10k, many blocks	14,120 ms	1,037 ms	–	868 ms
en 5k, one growing paragraph	432 ms	484 ms	476 ms	33.6 ms
ja 5k, one growing paragraph	3,941 ms	4,030 ms	4,043 ms	3,587 ms

One growing paragraph gains nothing from block reuse.
Lexing is 211 of 357 ms (en) and 389 of 878 ms (ja) of the block strategy.
Adding item reuse on top: 0.981-1.048×.
The harness compares streamed chat frames at 3 widths with frames built from scratch. The stale-font canary fails as it should.

3.4 Item reuse (`incremental-rich-memo`): parked

API (c4a5ee4 README):

prepareRichInline(items: RichInlineItem[], previous?: PreparedRichInline): PreparedRichInline
// After an edit, pass the previous result to reuse the preparation of items whose text, font and letterSpacing didn't change; the result is the same either way

How it works (c4a5ee4:src/rich-inline.ts:535, 571-583).

An item's core is what preparation computes from that item alone: its handle, whole width, establishesLine, boundary whitespace flags, WebKit boundary context, and first and last space segments.
A flow keeps its cores, the page language and the generation (getMeasurementGeneration(), bumped in clearMeasurementCaches()).
Leading and trailing items match by position. Middle items match through a Map keyed by font, letterSpacing and text, and every hit is checked again. Cores are reused only if the generation and the page language both match.
Gaps, joined windows, WebKit contexts, Gecko flags and the carry chain rerun main's code.
Any flow may be passed as previous, even an unrelated one.

Exactness. scripts/rich-inline-memo-check.ts (791 lines): 38,668 small and 704 large edit chains per profile, 0 differences, 414,010 cores reused. Canvas calls were identical in all 3,405 samples. Three canaries failed in 12 of 12 runs. An independent harness with deep-frozen previous flows found 0 differences in 18,340 steps.

Keystroke in a flow, warm, ms:

flow	main	item reuse	speedup
en, 50 items	0.176-0.191	0.023-0.024	7.4-8.3×
en, 500 items	1.99-3.13	0.184-0.202	10.0-16.9×
mixed, 50 items	0.290-0.327	0.029-0.032	9.2-10.1×
mixed, 500 items	2.82-3.58	0.235-0.245	11.5-14.8×

A keystroke in a 500-item flow goes from 12-21% of a frame to 1.1-1.5%.

Flags.

Callers who never pass previous, hot at 500 items: 1.023-1.041×, and 5 of 10 cells meet the "more than 2% slower with non-overlapping IQR" rule. The earlier five-bundle A/B flagged 0 of 8. The likely cause is the per-item core objects: retained heap grows 9-11%. Whether that can be removed wasn't tested.
Size: +7 B gzip on layout, +459 B on rich-inline. The limit was 800 B.
After a web font loads without clearCache(), a reused flow can differ from a fresh one on desktop Blink and Gecko: at width 1, the third line is 7.375 wide against 6.375. Entry geometry is cached by the identity of its fit-advance array (477510e:src/layout.ts:473-492

Incremental prepare: prototypes, numbers and open decisions