[Bug] Evaluate(save_as_csv=...) raises UnicodeEncodeError on Windows when outputs contain non-ASCII
Environment
- dspy 3.3.1
- Python 3.12.7
- Windows 11,
locale.getpreferredencoding(False)=cp1252
What happens
An evaluation completes normally, then the results file fails to write and the whole run's output is lost:
Average Metric: 1.0 / 1 (100.0%)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 47-48: character maps to <undefined>
Reproduction
No LM needed — the program is a plain function:
import dspy, tempfile, pathlib
ANSWER = "Don't guess — the capital is 北京 (Běijīng)."
def program(**kwargs):
return dspy.Prediction(answer=ANSWER)
def metric(example, pred, trace=None):
return 1.0
devset = [dspy.Example(question="capital of China?", answer=ANSWER).with_inputs("question")]
out = pathlib.Path(tempfile.mkdtemp())
ev = dspy.Evaluate(devset=devset, metric=metric, num_threads=1)
ev(program, save_as_csv=str(out / "results.csv")) # UnicodeEncodeError
ev(program, save_as_json=str(out / "results.json")) # fine
Root cause
dspy/evaluate/evaluate.py:206 opens the file in text mode without an encoding, so Python falls back to the platform default:
with open(save_as_csv, "w", newline="") as csvfile:
On Linux and macOS that default is UTF-8. On Windows it is the ANSI codepage — cp1252 on Western installs, cp932/936/949 on Japanese/Chinese/Korean ones — which cannot represent most characters a language model emits. Curly apostrophes and em dashes are included, so ordinary English output hits this too, not only CJK. CI does not catch it because the runners default to UTF-8.
Why save_as_json is unaffected
The JSON branch at line 220 has the same missing encoding, but json.dump defaults to ensure_ascii=True and escapes everything outside ASCII, so no non-ASCII character ever reaches the stream. Only the CSV branch writes raw text.
Suggested fix
Pass encoding="utf-8" on both opens: the CSV one to fix the bug, the JSON one so it does not become a bug if ensure_ascii=False is ever passed.
I have the fix and a regression test ready, and would like to take this one if a maintainer is happy to assign it.
Source: stanfordnlp/dspy