Helpers crash with UnicodeEncodeError printing → on non-UTF-8 stdout (Windows)
Summary
Several helpers print non-ASCII glyphs (→, —, …) to stdout. When stdout resolves to a non-UTF-8 encoding — which is the default on Windows, where Python falls back to the locale codepage (cp1252) for a redirected/piped stream — the print itself raises and the helper dies mid-run.
Repro
Windows 11, Python 3.12, no PYTHONIOENCODING set:
python helpers/render.py edit/edl.json -o final.mp4 --build-subtitlesTraceback (most recent call last):
File "helpers\render.py", line 615, in main
segment_paths = extract_all_segments(
File "helpers\render.py", line 238, in extract_all_segments
print(f"extracting {len(ranges)} segment(s) → {clips_dir.name}/")
File "...\Lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '→'
in position 24: character maps to <undefined>render.py fails at its first progress line, so no segments are extracted at all. pack_transcripts.py fails after writing takes_packed.md — the artifact is correct on disk, but the helper still exits non-zero, which breaks any agent step that checks the exit code.
Setting the console codepage to UTF-8 (chcp 65001) does not fix it: for a non-tty stream Python uses locale.getpreferredencoding(), not the console codepage.
Scope
15 call sites across 4 helpers:
| file | sites | chars |
|---|---|---|
helpers/render.py |
9 | → — … |
helpers/grade.py |
4 | → — |
helpers/pack_transcripts.py |
1 | → |
helpers/transcribe_batch.py |
1 | → |
Not Windows-specific in principle — any environment where stdout is not UTF-8 (e.g. LANG=C with output piped) hits the same thing.
Suggested fix
Reconfiguring stdout in each helper's entry point keeps the glyphs and is a one-liner per file:
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8", errors="replace")Replacing the glyphs with ASCII (->, --, ...) also works and needs no runtime guard, if you'd rather keep output plain.
Happy to send a PR for whichever you prefer.
Workaround
Setting PYTHONIOENCODING=utf-8 in the environment avoids all 15 sites without touching the repo.
Note
Filed separately from #124 (Windows subtitle path escaping) to keep that fix single-purpose. The two are independent: #124 makes subtitled renders work at all, this one makes the helpers survive their own progress output.
Source: browser-use/video-use