Intermittent panic in display_width.rs: large-file slicing at byte offset not floored to UTF-8 char boundary (CJK content)
Summary
Fresh panics intermittently when opening a large (~50 MB) UTF-8 text file containing dense CJK characters. The panic is a string slice taken at a byte offset that falls inside a multi-byte character:
thread 'main' panicked at crates/fresh-editor-core/src/primitives/display_width.rs:39:6:
end byte index 130512 is not a char boundary; it is inside '信' (bytes 130501..130513) of ...(The string context in the panic message is redacted here — it was a user data file.)
Why it looks like a race
The failing offset is in the region where large-file mode slices the buffer by a byte boundary. If the slice index were a fixed property of the file, the crash would be deterministic — it is not: reopening the same file often succeeds, so the index most likely comes from a live snapshot of the background scan/index progress (bytes-scanned-so-far), which the display-width code then uses to slice without aligning to a UTF-8 char boundary. Pure-ASCII files can never hit this (every byte offset is a boundary); dense CJK content makes any given offset land mid-character with roughly 2-in-3 odds.
Reproduction (intermittent)
- Create/open a ~50 MB UTF-8
.txtwith dense CJK text (e.g. a Chinese application log). - Open it in Fresh and scroll immediately while the background scan is still running.
- Sometimes panics at
display_width.rs:39(or nearby slicing sites); sometimes opens fine.
Suggested fix
Floor the slice index to a UTF-8 char boundary before slicing (a small floor_char_boundary helper — str::is_char_boundary loop), or have the scan-progress snapshot only publish offsets already aligned to line/char boundaries. Given there may be several byte-offset slicing sites in the large-file path, a shared helper at the slicing points is probably the robust shape.
Environment
Fresh 0.5.1 (npm build), Windows 11 (10.0.26100), system locale zh-CN.
Source: sinelaw/fresh