Popup command input corrupts multi-byte UTF-8 typed via fcitx5 IME (orphan lead bytes, dropped chars) on 0.9.1
Environment
- herdr 0.9.1 stable (
herdr-binpackage, Arch Linux) - Wayland (Hyprland), host terminal: foot
- IME: fcitx5 (Chinese, Rime),
XMODIFIERS=@im=fcitx,QT_IM_MODULE=fcitx LANG=en_US.UTF-8
Setup
A custom command keybinding opens a popup running a trivial bash script that reads one line and logs the raw bytes it received on stdin:
[[keys.command]]
key = "prefix+/"
type = "popup"
command = "bash /path/to/log-input.sh"
width = "60%"
height = 10#!/usr/bin/env bash
IFS= read -r line
printf 'read rc=%s len=%s hex=%s\n' "$?" "${#line}" \
"$(printf '%s' "$line" | od -An -tx1 | tr -d ' \n')"Symptom
Chinese text typed into the popup via fcitx5 frequently arrives on the script's stdin as invalid UTF-8: orphan lead bytes mid-string (bytes dropped from the middle of multi-byte sequences), and occasionally wrong-but-valid characters (bytes dropped/reordered rather than truncated).
Three captures from one session (expected text vs received bytes):
| Typed intent | Received bytes (excerpt) | Damage |
|---|---|---|
…上周六到… (Backspace used during composition) |
e4 b8 8a e5 91 a8 e5 e5 8a ad e5 88 b0 |
orphan e5, one char lost |
…到本周…现在 (no backspace, ~40-char sentence) |
…e6 9c ac e5 91 a8 e7 e4 bb 99 e5 ad 90 e7 8e b0 e5 9c a8 |
orphan e7 lead byte |
…上周五… |
…e5 91 a8 e6 e4 ba 94 |
orphan e6 lead byte |
Each of these lines was typed normally (compose with fcitx5, commit, continue typing, Enter). Backspace during IME composition reproduces it most reliably; longer sentences hit it even without visible backspace. Pasting the same text into the popup (terminal paste) delivered clean UTF-8 in 3/3 attempts. The same IME setup types flawless Chinese into regular panes.
Downstream impact
Forwarding the line to herdr agent start -- <args> fails herdr's own argv validation:
error: argument 12 is not valid UTF-8Arbitrary popup scripts that don't validate will silently consume corrupt input.
Notes
- Possibly related family: #3178 (remote rendering mojibake, fixed), #3499 (Korean IME drops syllable in a pane, fixed), #2894 (Chinese IME over direct SSH). This report is about the local popup command input path on 0.9.1.
- Workaround we use: sanitize with
iconv -f UTF-8 -t UTF-8 -c(note it exits non-zero even with-cwhile still emitting the cleaned prefix), or paste instead of typing.
Happy to provide full hex dumps or test a preview build.
Source: herdrdev/herdr