#288·gpt-oss

Eval progress helper crashes on an empty sample set

Author: sylvesterkaczmarekCreated Aug 16, 2026Updated Aug 16, 2026

Summary

gpt_oss.evals.report.map_with_progress() creates its worker pool with ThreadPool(min(num_threads, len(xs))). When xs is empty, that becomes ThreadPool(0), which raises ValueError: Number of processes must be at least 1.

Empty eval inputs are otherwise representable. For example, HealthBenchEval(num_examples=0) samples zero examples and then routes the empty list through map_with_progress().

Impact

An empty evaluation fails in the threading helper before the eval can return an empty aggregate/result. The failure is unrelated to model sampling or grading and makes zero-example/debug configurations brittle.

Proposed resolution

Return an empty list immediately when xs is empty, before constructing a thread pool. Add regression coverage for both progress-enabled and progress-disabled calls so empty inputs never invoke the mapped function or create workers.