[Bug]: search-specs CLI crashes with UnicodeEncodeError on non-UTF-8 consoles

Author: aniruddhaadak80Created Sep 14, 2026Updated Sep 15, 2026
Labelstriage: needs-review

What happened?

forge/stage1_intake/search_specs.py prints query text and result snippets with ensure_ascii=False straight to print(). When stdout is not UTF-8 (Windows console with a non-UTF-8 code page, or any host with PYTHONIOENCODING set to a legacy codec), any non-ASCII query or snippet crashes the CLI with UnicodeEncodeError instead of returning results. I expected the CLI to emit UTF-8 output and exit 0 regardless of the console encoding, since every source file and cache in this pipeline is UTF-8.

Reproduction steps

  1. From the skill root, force a non-UTF-8 stdout encoding (portable on any OS):

    PYTHONIOENCODING=ascii python3 forge/stage1_intake/search_specs.py --collection cs2 rough --json

    Any query whose snippets contain non-ASCII text (for example the Vietnamese fixture terms) triggers it.

  2. Observe a traceback ending in UnicodeEncodeError and exit code 1, instead of the JSON payload.

The repo's own suite covers this path: test_human_output_is_nonempty_and_stable_on_cache_hits and test_untruncated_snippet_has_no_ellipses in forge/tests/test_search_specs.py search fixture corpora containing Vietnamese terms and an ellipsis snippet. Both pass on UTF-8 hosts and fail on Windows default consoles.

Reference image or input characteristics

Not image-related. Input: any search query plus an indexed corpus containing non-ASCII text (Vietnamese terms, ellipsis U+2026). Output: the JSON or human-readable result listing passed through print() with ensure_ascii=False.

Relevant output and evidence

Child-process stderr on a cp1252 Windows console:

File "forge/stage1_intake/search_specs.py", line 143, in _print_human
    print(f"Query: {payload['query']}")
UnicodeEncodeError: 'charmap' codec can't encode characters: character maps to <undefined>

The JSON path fails identically in _print_json (json.dumps(..., ensure_ascii=False) written to a cp1252 stream). The same print pattern exists across other forge CLIs, but this issue is scoped to search_specs.py; the rest can follow the same remedy once agreed.

Environment

  • img2threejs version or commit: upstream/main 6e60b5e
  • Python version: 3.14.3 (Windows; repo toolchain is python3, invoked here via the py launcher)
  • Host agent: OpenCode
  • Operating system: Windows 11 (cp1252 console); portable repro via PYTHONIOENCODING=ascii on any OS