Compiler output loses every non-ASCII byte when stdout is not a terminal

When the output stream is not a terminal, sccache passes the compiler's stdout and stderr through strip-ansi-escapes before writing them. That crate parses with vte, whose Perform::execute forwards only '\n'. Bytes 0x80-0x9f are treated as C1 control codes and dropped; 0xa0-0xc1 and 0xf5-0xff have no ground-state entry and are dropped; '\r' is dropped too. Only bytes that form valid UTF-8 survive.

Reproduces on Linux with gcc, no special locale needed:

printf '#warning caf\xe9 na\xefve\nint f(void){ return 1; }\n' > lat1.c
gcc -c -Wall lat1.c -o /dev/null 2> bare.txt
sccache gcc -c -Wall lat1.c -o /dev/null 2> sccache.txt
cmp bare.txt sccache.txt

gcc echoes the raw 0xe9 and 0xef from the source into its diagnostic. bare.txt is 94 bytes containing those two bytes; sccache.txt is 98 bytes with each replaced by U+FFFD. Redirecting is what matters - on a terminal sccache writes the bytes through unchanged, so this only shows up in piped or logged builds.

Where it actually hurts is Windows on a codepage that is not UTF-8. MSVC emits /showIncludes as multi-byte text, and ninja's deps = msvc finds header dependencies by matching msvc_deps_prefix against that output. With the prefix mangled ninja records no dependencies at all, so touching a header rebuilds nothing. Measured on a 3-file CMake project, Japanese Windows, CUDA/MSVC 2022:

Silent and inverted: a cache hit leaves the dependency graph empty, so a rebuild that was owed does not happen.

A fix - a byte-level scanner that only looks for a literal ESC - is on https://github.com/beru/sccache/tree/msvc-nonascii-output-passthrough

Generated with Claude Code