Bug: remove_eda_part greedily removes stdout between multiple EDA blocks
Summary
remove_eda_part removes too much output when stdout contains more than one EDA block.
It should remove only text inside each explicit EDA delimiter pair:
=== Start of EDA part ===
...
=== End of EDA part ===However, the current regex is greedy and uses re.DOTALL, so it removes from the first start marker to the last end marker. Normal stdout between two separate EDA blocks is deleted.
This affects multiple data-science evaluators that call remove_eda_part before constructing feedback prompts or diagnostics.
To Reproduce
Check out RD-Agent
mainat commit6762f84f9bc0f5c6486c50a00e128a57ac6c3683.Install RD-Agent from source.
Create
test/scenarios/data_science/test_remove_eda_part_greedy.py:
from rdagent.components.coder.data_science.utils import remove_eda_part
def test_remove_eda_part_preserves_text_between_multiple_eda_blocks():
stdout = (
"before\n"
"=== Start of EDA part ===\n"
"eda one\n"
"=== End of EDA part ===\n"
"between\n"
"=== Start of EDA part ===\n"
"eda two\n"
"=== End of EDA part ===\n"
"after\n"
)
cleaned = remove_eda_part(stdout)
assert "eda one" not in cleaned
assert "eda two" not in cleaned
assert "before" in cleaned
assert "between" in cleaned
assert "after" in cleaned- Run:
python -m pytest test/scenarios/data_science/test_remove_eda_part_greedy.py -q- Observe that the assertion for
"between"fails.
Expected Behavior
Only the two delimited EDA blocks should be removed. The normal output between them should remain.
For example, the cleaned output should still contain:
before
between
afterActual Behavior
The cleaned output is:
"before\n\nafter\n"The normal "between" line is deleted.
AssertionError: assert 'between' in 'before\n\nafter\n'Screenshot
Not applicable; this is a deterministic unit-level reproduction.
Environment
- Name of current operating system: macOS
- Processor architecture: arm64
- Python version:
3.11.15 - RD-Agent version:
0.8.0,main@6762f84f9bc0f5c6486c50a00e128a57ac6c3683 - Container: not used in this reproduction
Additional Notes
The current implementation is:
return re.sub(
r"=== Start of EDA part ===(.*)=== End of EDA part ===",
"",
stdout,
flags=re.DOTALL,
)Because (.*) is greedy under re.DOTALL, it spans across multiple EDA blocks.
A possible fix is to make the block match non-greedy, or otherwise parse each delimiter pair independently, so non-EDA stdout between blocks is preserved.
Source: microsoft/RD-Agent