当测试命令以非零状态退出且没有输出时, Benchmark 会将此练习记录为已解决
作者: shaurya416创建于 2026年9月25日更新于 2026年9月28日
benchmark/benchmark.py:1038-1048 (run_unit_tests, defined at :981), read by its caller at :881-892; solved exercises are counted at :508. Code as of 5dc9490 (current main).
success = result.returncode == 0
res = result.stdout
res = cleanup_test_output(res, testdir)
...
if not success:
print(f"Tests failed: {testdir}")
return res try:
errors = run_unit_tests(original_dname, testdir, history_fname, test_files)
except subprocess.TimeoutExpired:
errors = "Tests timed out!"
timeouts += 1
if errors:
test_outcomes.append(False)
else:
test_outcomes.append(True)
breakrun_unit_tests reports a failing run by returning the runner's output and a passing run by returning None. The return code decides only whether that text is returned; the caller then decides pass or fail from whether the text is non-empty. A test command that exits non-zero with empty output returns "", so the exercise is recorded as solved (test_outcomes.append(True)) straight after the function has printed "Tests failed".
内容来源: Aider-AI/aider