Bug: Markdown chat code box counts the tab that a soft-wrapped line ends on
Description
Follow-up to #267. A fenced code line in the Markdown chat demo that soft-wraps right after a tab still makes its code box wider than its text. The box can also run past the message lane.
#267 added measureCodeLineStats() in pages/demos/markdown-chat.model.ts so that a code box doesn't count the preserved space a soft-wrapped line ends on. It only checks for 'preserved-space':
const isSoftWrapAfterSpace =
line.end.graphemeIndex === 0 &&
line.end.segmentIndex < prepared.segments.length &&
prepared.kinds[last] === 'preserved-space'
const width = isSoftWrapAfterSpace ? line.width - prepared.widths[last]! : line.width
A tab is a 'tab' segment, so a line that wraps after a tab keeps the tab's advance in line.width, and the box takes that width. CSS Text 3 hangs any preserved white space at the end of a pre-wrap line unless a forced break follows (§4.1.2), and hanging glyphs don't count toward the box's size (§8.2). Tabs are preserved white space too. A line ending in a space already gets this treatment, but a tab doesn't.
The tab advances to the next tab stop, so the extra width can be up to eight spaces. When the tab stop falls past the code text width, Pretext keeps the tab on the line and counts it, so the box ends past the lane.
Adding 'tab' to the existing check isn't enough on its own: tab segments are stored with prepared.widths[i] === 0, and their advance depends on where they sit on the line (getTabAdvance() in src/line-break.ts). So line.width - prepared.widths[last] would subtract 0.
This reproduces at main 5dbc9bd611 in headless Chrome and headless Firefox on macOS.
Reproduction
Each message is one fenced code line: a run of q, then a separator, then after_tab_tail. The separator is the only break opportunity. The first four messages use a tab. The last two use a space (controls).
const fence = (n, separator) => ({ role: 'assistant', markdown: '```\n' + 'q'.repeat(n) + separator + 'after_tab_tail' + '\n```' })
// fence(35, '\t'), fence(39, '\t'), fence(77, '\t'), fence(79, '\t'), fence(35, ' '), fence(77, ' ')
At chat width 360 the code text width is 292px (lane 316 − padding 24), so n = 35 and n = 39 wrap after the tab. At chat width 640 it is 572px, so n = 77 and n = 79 wrap after the tab.
Steps:
- Replace
pages/demos/markdown-chat.data.tswithmarkdown-chat.data.tsfrom the files below. bun start, then open/demos/markdown-chatwith a viewport 416px wide (chat width 360), then 696px wide (chat width 640). The snippet in step 3 prints--chat-width. If a classic scrollbar changes it, adjust the width.- Paste
devtools-snippet.jsfrom the files below into the console. It scrolls the thread and prints one row per code box.
The snippet's counting rules:
- model text width: the box's inline
style.width− 24 (CODE_BLOCK_PADDING_X * 2). This is the widest line width Pretext reported, after the demo's adjustment. - glyph width: a
Rangeover each.code-linetext node, with trailing spaces and tabs left out on every line except the last. The box's glyph width is the maximum over its lines. - wider: model text width − glyph width.
- past lane: box
getBoundingClientRect().right− bubbleright(assistant messages have no bubble padding).
Also below: a scripted run that reports the same numbers. build.mjs builds the demo from a checkout with the seed file swapped in. harness.html loads the demo in an iframe at each chat width and measures it, plus a native reference. run-chrome.mjs (playwright-core) and run-firefox.ts (scripts/browser-automation.ts, WebDriver BiDi) run the harness headless. run-snippet.mjs runs devtools-snippet.js on the built page in headless Chrome. report.mjs holds the static server and printer the three runners share. Save all of them in one directory.
bun build.mjs <pretext checkout> <out dir>
bun run-chrome.mjs <out dir> <pretext checkout>/node_modules/playwright-core/index.mjs
bun run-firefox.ts <out dir> <pretext checkout>
markdown-chat.data.ts// Drop-in replacement for pages/demos/markdown-chat.data.ts.
// Each message is one fenced code line: a run of 'q', a separator, then
// 'after_tab_tail', so the only soft-wrap opportunity is after the separator.
// The first four use a tab; the last two use a space instead (controls).
// Chat width 360 gives a 292px code text width, so n = 35 and 39 wrap after the tab.
// Chat width 640 gives 572px, so n = 77 and 79 wrap after the tab.
export type MarkdownChatSeed = {
role: 'assistant' | 'user'
markdown: string
}
const fence = (n: number, separator: string): MarkdownChatSeed => ({
role: 'assistant',
markdown: '```\n' + 'q'.repeat(n) + separator + 'after_tab_tail' + '\n```',
})
const SPECS: MarkdownChatSeed[] = [
fence(35, '\t'),
fence(39, '\t'),
fence(77, '\t'),
fence(79, '\t'),
fence(35, ' '),
fence(77, ' '),
]
export function createMarkdownChatSpecs(_count: number): MarkdownChatSeed[] {
return SPECS
}
devtools-snippet.js// Paste into the DevTools console on /demos/markdown-chat after swapping in
// markdown-chat.data.ts from this repro. Scrolls the thread and prints one row
// per painted code box.
//
// model text width: the box's inline style width - 24 (CODE_BLOCK_PADDING_X * 2),
// i.e. the widest line Pretext reported, after the demo drops a trailing
// soft-wrap space.
// glyph width: a Range over each .code-line's text node, with trailing spaces
// and tabs dropped on every line but the last; the box's is the max over lines.
// wider: model text width - glyph width.
// past lane: box right - bubble right (assistant messages have no bubble padding).
await (async () => {
const vp = document.getElementById('chat-viewport')
const frame = () => new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))
const seen = new Map()
const grab = () => {
for (const row of document.querySelectorAll('.msg')) {
const bubble = row.querySelector('.msg-bubble').getBoundingClientRect()
const laneRight = bubble.right - (row.classList.contains('msg--user') ? 16 : 0)
row.querySelectorAll('.code-box').forEach((box, i) => {
const key = `${Number.parseFloat(row.style.top)}#${i}`
if (seen.has(key)) return
const lines = [...box.querySelectorAll('.code-line')]
const glyph = Math.max(...lines.map((line, k) => {
const t = line.firstChild
if (t === null) return 0
const len = k === lines.length - 1 ? t.length : t.data.replace(/[ \t]+$/, '').length
const range = document.createRange()
range.setStart(t, 0)
range.setEnd(t, len)
return range.getBoundingClientRect().width
}))
const r = box.getBoundingClientRect()
const modelText = Number.parseFloat(box.style.width) - 24
seen.set(key, {
lines: lines.map(l => JSON.stringify(l.textContent)).join(' | '),
modelText: +modelText.toFixed(2),
glyph: +glyph.toFixed(2),
wider: +(modelText - glyph).toFixed(2),
pastLane: +(r.right - laneRight).toFixed(2),
})
})
}
}
for (let y = 0; ; y = Math.min(vp.scrollHeight, y + vp.clientHeight / 2)) {
vp.scrollTop = y
await frame()
grab()
if (y >= vp.scrollHeight - vp.clientHeight) break
}
vp.scrollTop = 0
const chatWidth = document.documentElement.style.getPropertyValue('--chat-width')
const rows = [...seen.values()]
console.log('chat width', chatWidth)
console.table(rows)
return { chatWidth, rows }
})()
build.mjs// Builds pages/demos/markdown-chat.html from a Pretext checkout with this directory's
// markdown-chat.data.ts in place of the demo's own seeds, and copies harness.html next to it.
// Nothing in the checkout is modified.
// Usage: bun build.mjs <pretext checkout> <out dir>
import { copyFileSync, mkdirSync, readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
const [tree, outdir] = process.argv.slice(2)
const here = dirname(new URL(import.meta.url).pathname)
const seeds = readFileSync(join(here, 'markdown-chat.data.ts'), 'utf8')
const r = await Bun.build({
entrypoints: [join(tree, 'pages/demos/markdown-chat.html')],
outdir,
plugins: [{ name: 'tab-seeds', setup(b) { b.onLoad({ filter: /markdown-chat\.data\.ts$/ }, () => ({ contents: seeds, loader: 'ts' })) } }],
})
if (!r.success) throw new Error(r.logs.join('\n'))
mkdirSync(outdir, { recursive: true })
copyFileSync(join(here, 'harness.html'), join(outdir, 'harness.html'))
console.log(`built ${outdir}`)
harness.html<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Code box tab soft-wrap harness</title>
<style>
body { margin: 0; }
iframe { border: 0; display: block; }
#native { position: absolute; left: 0; top: 0; white-space: pre-wrap; overflow-wrap: break-word; line-height: 18px; visibility: hidden; }
</style>
</head>
<body>
<iframe id="chat" width="416" height="2400"></iframe>
<div id="native"></div>
<script>
// Loads the Markdown chat demo (built with the tab seeds) in an iframe whose width
// sets --chat-width, then reports every painted code box. Result goes to location.hash.
// Counting rules:
// - model text width: box inline style width - 24 (CODE_BLOCK_PADDING_X * 2)
// - glyph width: Range over each .code-line text node, trailing spaces and tabs dropped on
// every line but the last; box glyph width = max over lines
// - wider: model text width - glyph width
// - past lane: box rect right - bubble rect right (assistant messages)
// Native reference: the same code text in a white-space: pre-wrap div of the code text width
// and the demo's code font. Each character is assigned to a line by its rect's vertical
// centre; a line's extent is the right edge of its last non-space, non-tab character
// minus the div's left; lines = number of distinct line tops.
// Same order as markdown-chat.data.ts.
const TEXTS = [[35, '\t'], [39, '\t'], [77, '\t'], [79, '\t'], [35, ' '], [77, ' ']].map(([n, sep]) => 'q'.repeat(n) + sep + 'after_tab_tail')
const frames = (win, k) => new Promise(res => { let left = k; const step = () => (--left <= 0 ? res() : win.requestAnimationFrame(step)); win.requestAnimationFrame(step) })
const iframe = document.getElementById('chat')
async function load(width) {
iframe.width = String(width)
await new Promise(res => { iframe.onload = res; iframe.src = `./markdown-chat.html?w=${width}&t=${Date.now()}` })
const win = iframe.contentWindow
for (let i = 0; i < 200 && win.document.querySelector('.msg') === null; i++) await frames(win, 1)
await frames(win, 3)
return win
}
function nativeLines(text, width, font) {
const el = document.getElementById('native')
el.style.width = `${width}px`
el.style.font = font
el.textContent = text
const t = el.firstChild
const left = el.getBoundingClientRect().left
const lines = new Map()
for (let i = 0; i < t.length; i++) {
const range = document.createRange()
range.setStart(t, i)
range.setEnd(t, i + 1)
const rects = [...range.getClientRects()].filter(r => r.width > 0 || r.height > 0)
if (rects.length === 0) continue
const r = rects[0]
const line = Math.floor((r.top + r.height / 2 - el.getBoundingClientRect().top) / 18)
const entry = lines.get(line) ?? { text: '', extent: 0 }
entry.text += t.data[i]
if (t.data[i] !== ' ' && t.data[i] !== '\t') entry.extent = Math.max(entry.extent, r.right - left)
lines.set(line, entry)
}
return [...lines.entries()].sort((a, b) => a[0] - b[0]).map(([, v]) => v)
}
async function measure(target) {
let width = target + 56
let win
let chatWidth
for (let attempt = 0; attempt < 4; attempt++) {
win = await load(width)
chatWidth = Number.parseFloat(win.document.documentElement.style.getPropertyValue('--chat-width'))
if (chatWidth === target) break
width += target - chatWidth
}
const doc = win.document
const vp = doc.getElementById('chat-viewport')
const seen = new Map()
for (let y = 0; ; y = Math.min(vp.scrollHeight, y + Math.max(100, vp.clientHeight - 300))) {
vp.scrollTop = y
await frames(win, 2)
for (const row of doc.querySelectorAll('.msg')) {
const bubble = row.querySelector('.msg-bubble').getBoundingClientRect()
row.querySelectorAll('.code-box').forEach((box, i) => {
const key = `${Number.parseFloat(row.style.top)}#${i}`
if (seen.has(key)) return
const lines = [...box.querySelectorAll('.code-line')]
const glyph = Math.max(...lines.map((line, k) => {
const t = line.firstChild
if (t === null) return 0
const len = k === lines.length - 1 ? t.length : t.data.replace(/[ \t]+$/, '').length
if (len === 0) return 0
const range = doc.createRange()
range.setStart(t, 0)
range.setEnd(t, len)
return range.getBoundingClientRect().width
}))
const r = box.getBoundingClientRect()
seen.set(key, {
top: Number.parseFloat(row.style.top),
lines: lines.map(l => l.textContent),
modelText: Number.parseFloat(box.style.width) - 24,
glyph,
boxRight: r.right,
laneRight: bubble.right,
})
})
}
if (y >= vp.scrollHeight - vp.clientHeight) break
}
const codeFont = doc.documentElement.style.getPropertyValue('--code-font')
const boxes = [...seen.values()].sort((a, b) => a.top - b.top)
const codeTextWidth = target - 44 - 24
return {
target, iframeWidth: width, chatWidth, codeFont, codeTextWidth,
boxes: boxes.map((b, i) => {
const text = TEXTS[i]
const native = nativeLines(text, codeTextWidth, codeFont)
return {
...b, text,
wider: b.modelText - b.glyph,
pastLane: b.boxRight - b.laneRight,
nativeLines: native.map(l => l.text),
nativeExtent: Math.max(...native.map(l => l.extent)),
}
}),
}
}
async function main() {
const out = { userAgent: navigator.userAgent, runs: [] }
try {
for (const target of [360, 640]) out.runs.push(await measure(target))
const ctx = document.createElement('canvas').getContext('2d')
out.canvasWidths = {}
for (const f of ['"SF Mono"', 'ui-monospace', 'Menlo', 'monospace']) { ctx.font = `500 12px ${f}`; out.canvasWidths[f] = ctx.measureText('q\tWiM_').width }
} catch (e) {
out.error = String(e && e.stack || e)
}
location.hash = `result=${encodeURIComponent(JSON.stringify(out))}`
}
main()
</script>
</body>
</html>
report.mjs// Shared: serve a built directory on an OS-assigned port, and print a harness report.
import { join } from 'node:path'
export function serve(dir) {
return Bun.serve({
hostname: '127.0.0.1',
port: 0,
async fetch(req) {
const f = Bun.file(join(dir, decodeURIComponent(new URL(req.url).pathname)))
return (await f.exists()) ? new Response(f) : new Response('not found', { status: 404 })
},
})
}
export function print(label, report) {
console.log(`${label}: ${report.userAgent}`)
if (report.error) console.log(` ERROR ${report.error}`)
console.log(` canvas width of 'q\\tWiM_' at 500 12px: ${JSON.stringify(report.canvasWidths)}`)
for (const run of report.runs) {
console.log(` chat width ${run.chatWidth} (iframe ${run.iframeWidth}), code text width ${run.codeTextWidth}, font ${run.codeFont}`)
for (const b of run.boxes) {
console.log(` ${JSON.stringify(b.text).padEnd(98)} pretext lines ${b.lines.length} modelText ${b.modelText.toFixed(2)} glyph ${b.glyph.toFixed(2)} wider ${b.wider.toFixed(2)} pastLane ${b.pastLane.toFixed(2)} | native lines ${b.nativeLines.length} extent ${b.nativeExtent.toFixed(2)} ${JSON.stringify(b.nativeLines)}`)
}
}
}
run-chrome.mjs// Headless installed Chrome through playwright-core, DPR 1.
// Usage: bun run-chrome.mjs <built dir> <playwright-core index.mjs> [out json]
import { writeFileSync } from 'node:fs'
import { print, serve } from './report.mjs'
const [dir, playwright, outFile] = process.argv.slice(2)
const { chromium } = await import(playwright)
const server = serve(dir)
const browser = await chromium.launch({ executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', headless: true })
try {
const context = await browser.newContext({ viewport: { width: 1200, height: 2600 }, deviceScaleFactor: 1 })
const page = await context.newPage()
page.on('pageerror', e => console.error(`pageerror: ${e.message}`))
await page.goto(`http://127.0.0.1:${server.port}/harness.html`)
await page.waitForFunction(() => location.hash.startsWith('#result='), undefined, { timeout: 90_000 })
const href = page.url()
const report = JSON.parse(decodeURIComponent(href.slice(href.indexOf('#result=') + 8)))
report.browserVersion = browser.version()
if (outFile) writeFileSync(outFile, `${JSON.stringify(report, null, 1)}\n`)
print(`headless Chrome ${report.browserVersion}`, report)
} finally {
await browser.close()
server.stop()
}
run-firefox.ts// Headless installed Firefox through the checkout's scripts/browser-automation.ts (WebDriver BiDi).
// Usage: bun run-firefox.ts <built dir> <pretext checkout> [out json]
import { writeFileSync } from 'node:fs'
import { print, serve } from './report.mjs'
const [dir, tree, outFile] = process.argv.slice(2)
const { createBrowserSession, sleep } = await import(`${tree}/scripts/browser-automation.ts`)
const server = serve(dir)
const session = createBrowserSession('firefox', { foreground: false, headless: true })
try {
await session.navigate(`http://127.0.0.1:${server.port}/harness.html`)
const deadline = Date.now() + 90_000
let href = ''
while (Date.now() < deadline) {
await sleep(250)
href = await session.readLocationUrl()
if (href.includes('#result=')) break
}
if (!href.includes('#result=')) throw new Error('no report')
const report = JSON.parse(decodeURIComponent(href.slice(href.indexOf('#result=') + 8)))
if (outFile) writeFileSync(outFile, `${JSON.stringify(report, null, 1)}\n`)
print('headless Firefox', report)
} finally {
await session.close()
server.stop()
}
run-snippet.mjs// Runs devtools-snippet.js on the built demo page in headless Chrome at viewport widths
// 416 and 696 (chat widths 360 and 640), DPR 1.
// Usage: bun run-snippet.mjs <built dir> <playwright-core index.mjs>
import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { serve } from './report.mjs'
const [dir, playwright] = process.argv.slice(2)
const { chromium } = await import(playwright)
const snippet = readFileSync(join(dirname(new URL(import.meta.url).pathname), 'devtools-snippet.js'), 'utf8').replace(/^await /m, 'return ')
const server = serve(dir)
const browser = await chromium.launch({ executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', headless: true })
try {
for (const width of [416, 696]) {
const context = await browser.newContext({ viewport: { width, height: 900 }, deviceScaleFactor: 1 })
const page = await context.newPage()
await page.goto(`http://127.0.0.1:${server.port}/markdown-chat.html`)
await page.waitForSelector('.msg')
const result = await page.evaluate(`(async () => { ${snippet} })()`)
console.log(`headless Chrome ${browser.version()}, viewport ${width}, chat width ${result.chatWidth}`)
for (const r of resu
Source: chenglou/pretext