[Bug] Non-CJK text (Russian) also corrupted during streaming — syllables lost/duplicated, state.db is intact, restart fixes it
Describe the bug
Hermes Desktop (Windows) renders assistant messages with lost or duplicated fragments (individual syllables) during/after streaming. The corruption affects not only CJK — the same happens with Russian / Cyrillic text.
- In the visible chat bubble, part of a word's syllable goes missing, then later repeats.
- The same text is always complete in
state.db— read it back and the message is intact. The corruption is purely a render-layer merge bug. - Restarting the app (or reloading the session) makes the identical text render correctly.
Only the chat message content is affected; the rest of the UI is fine.
Example (Russian)
| Rendered in bubble | Actual (state.db) |
|---|---|
| «Чтолось» | «Что осталось» |
| «Имп схем НЕ построен» | «Импорт схем НЕ построен» |
| «точню направление» | «Уточню направление» |
| «Вече Новода» | «Вече Новгорода» |
Root cause (likely)
This is the same mechanism as #793 (CJK), which I reproduce here for non-CJK:
mergeStreamedWithFinal() in src/renderer/src/screens/Chat/dashboardEventAdapter.ts reconciles the streamed text against final_response at end of stream via isLossyChunkCopy() (lossyText.ts). That uses a greedy indexOf over repeating fragments — for any text containing repeated substrings (common in Russian too), it matches the wrong position → false "lost block" → overwrites the streamed text with an incomplete final → characters get dropped/duplicated. commonSuffixLength() boundary handling is inexact for mixed-script text.
Suggested fix: make isLossyChunkCopy match forward from j with exact-boundary/character-level verification (currently full.indexOf(probe, j)), or skip the lossy reconciliation branch and do a simple final ⊇ streamed containment check.
Related
- #793 — same CJK text-corruption root cause
- #508 — merged fix for duplicate/split turns (message ordering), does not fix character-level loss
- #550 / #517 / #527 — duplicate-message family, also not this
Expected vs actual
Expected: streamed text merges into the identical content that lands in state.db.
Actual: visible bubble shows corrupted (lost/duplicated characters) while state.db holds the full correct text.
Steps to reproduce: any long-assistant-turn chat with the desktop app on a Russian-language model; corruption appears during/after streaming, disappears on app restart.
Desktop bundle can lag behind hermes-agent source — worth verifying against current main too.
Source: fathah/hermes-desktop