#10362·dspy

[Bug] bootstrap_trace partial-credit crashes with TypeError, and a 0.0 format_reward is rescored to the failure penalty

Author: plthiyaguCreated Sep 10, 2026Updated Sep 10, 2026

What happened?

Two related defects in the format-reward path shared by bootstrap_trace and GRPO:

1. The partial-credit branch crashes on every partial parse.

dspy/teleprompt/bootstrap_trace.py (patched_forward) computes partial format credit as:

python
present = list(parsed_result.keys()) if parsed_result else None
expected = list(failed_signature.output_fields.keys())
...
format_reward=format_failure_score
+ (failure_score - format_failure_score) * (present / expected)

present and expected are lists, so present / expected raises TypeError: unsupported operand type(s) for /: 'list' and 'list' the moment an adapter reports a partial parse (e.g. JSONAdapter.parse raises AdapterParseError(parsed_result=fields) whenever some but not all output fields parse). The intended fraction is len(present) / len(expected).

2. A legitimate format_reward of 0.0 is silently replaced by the failure penalty.

Both consumers use or where None is the documented sentinel (FailedPrediction.format_reward: float | None = None):

  • bootstrap_trace.py (wrapped_metric): return prediction.format_reward or format_failure_score
  • grpo.py: score = trace_instance[2].format_reward or self.format_failure_score

With the in-tree defaults (failure_score=0, format_failure_score=-1), the fixed partial-credit formula yields 0.0 exactly when every expected field parsed — the best possible partial score — and 0.0 or -1 rescores it to the worst. Same falsy-zero class as #10321 (seed=0), applied to training rewards: GRPO then optimizes against corrupted signal.

Steps to reproduce

python
import dspy
from dspy.teleprompt.bootstrap_trace import bootstrap_trace_data

class TwoFieldSignature(dspy.Signature):
    text: str = dspy.InputField()
    number: int = dspy.OutputField()
    explanation: str = dspy.OutputField()

# Any LM response that parses only one of the two output fields, e.g.
# '{"number": 1}' under JSONAdapter, raises AdapterParseError with
# parsed_result={'number': 1} -> present truthy -> TypeError above.

Expected

  • A partial parse produces format_failure_score + (failure_score - format_failure_score) * len(present) / len(expected).
  • Only format_reward=None falls back to format_failure_score; 0.0 is a real reward and must survive.

I have a fix with a regression test ready and will open a PR.

DSPy version

main (ca54a855)