#1510·SWE-agent

BUG: BinaryTrajectoryComparison skips extra sampling for indented edit actions

Author: yifanxiong272Created Aug 16, 2026Updated Aug 17, 2026

Describe the bug

Summary

In sweagent/agent/action_sampler.py, BinaryTrajectoryComparison.contains_edits checks parsed actions with action.startswith(...) without removing leading whitespace.

ThoughtActionParser preserves indentation inside the action code block, while the execution path strips the parsed action before running it. An action such as edit file.txt is therefore normalized to edit file.txt for execution, but contains_edits returns False.

Impact

BinaryTrajectoryComparison.get_completions uses contains_edits to decide whether to request additional completions up to max_n_samples.

When an edit action is indented inside the model's code block, the configured edit-triggered extra-sampling branch is skipped even though the execution path normalizes it to the same edit command.

Steps/commands/code to Reproduce

  1. Check out SWE-agent main at commit 3ea751c087f32b16e039a2233dd6eefecef325d5.

  2. Set up the source checkout:

bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
  1. Add the following test to tests/test_parsing.py:
python
def test_binary_trajectory_comparison_detects_indented_edit():
    from sweagent.agent.action_sampler import BinaryTrajectoryComparison
    from sweagent.tools.parsing import ThoughtActionParser
    from sweagent.tools.tools import ToolConfig, ToolHandler

    tools = ToolHandler(ToolConfig(parse_function=ThoughtActionParser()))
    sampler = BinaryTrajectoryComparison.__new__(BinaryTrajectoryComparison)
    sampler._tools = tools
    completion = {
        "message": "Plan\n```\n  edit file.txt\n```",
    }

    _, action = tools.parse_actions(completion)

    assert tools.guard_multiline_input(action).strip() == "edit file.txt"
    assert sampler.contains_edits([completion]) is True
  1. Run:
bash
python -m pytest tests/test_parsing.py -q
  1. Observe that the newly added assertion fails while the eight existing tests pass.

Error message/results

Expected result

ThoughtActionParser returns an action containing the leading indentation. The execution normalization produces:

python
"edit file.txt"

Because this is one of the edit command prefixes recognized by BinaryTrajectoryComparison, contains_edits should return:

python
True

Actual result

The execution-normalization assertion passes, but contains_edits returns False.

The focused run reports:

........F                                                                [100%]

FAILED tests/test_parsing.py::test_binary_trajectory_comparison_detects_indented_edit
AssertionError: assert False is True
 +  where False = contains_edits([{'message': 'Plan\n```\n  edit file.txt\n```'}])

1 failed, 8 passed

At the tested commit, edit detection operates on the unnormalized action:

python
def contains_edits(self, completions: list[dict[str, Any]]) -> bool:
    keywords = [
        "edit",
        "str_replace_editor insert",
        "str_replace_editor str_replace",
    ]
    for completion in completions:
        _, action = self._tools.parse_actions(completion)
        if any(action.startswith(keyword) for keyword in keywords):
            return True
    return False

The execution path applies whitespace normalization before sending the same action to the environment:

python
run_action: str = self.tools.guard_multiline_input(
    step.action
).strip()

For the focused input, the parser returns " edit file.txt\n". contains_edits does not recognize it, while the execution path produces "edit file.txt".

One possible fix direction is to normalize leading whitespace before testing the edit prefixes:

python
normalized_action = action.lstrip()
if any(
    normalized_action.startswith(keyword)
    for keyword in keywords
):
    return True

Regression coverage should include unindented and indented forms of edit, str_replace_editor insert, and str_replace_editor str_replace, together with a non-edit action.

System Information

  • SWE-agent: 1.1.0, main@3ea751c087f32b16e039a2233dd6eefecef325d5
  • SWE-ReX: 1.4.0
  • Python: 3.12.13
  • Operating system: macOS 15.7.3, arm64
  • Installation: source checkout with editable development installation
  • Parser mode: thought_action
  • Model/provider: not applicable; reproduced through deterministic parsing and action-sampling code

Checklist

  • I'm running with the latest docker container/on the latest development version (i.e., I ran git pull))
  • I have copied the full command/code that I ran (as text, not as screenshot!)
  • If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
  • I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.