#1475·RD-Agent

Bug: analyze_error returns a matched error as both node and raw string

Author: yifanxiong272Created Sep 2, 2026Updated Sep 5, 2026
Labelsbug

Bug Description

CoSTEERRAGStrategyV2.analyze_error can return two entries for the same parsed error content when the knowledge graph contains both:

  • one existing error node that matches the current error content; and
  • at least one other error node that does not match it.

The method comment describes the output as:

python
# Raised errors, existed error nodes + not existed error nodes(here, they are strs)

So a parsed error content that already exists in the graph should be represented by the existing error node, while a parsed error content that does not exist should remain a string.

However, the current implementation loops over every existing error node and appends either the matching node or the raw string for each graph node. For one matched content, this can produce both:

python
"same error content"

and:

python
UndirectedNode(content="same error content", label="error")

in the returned list.

To Reproduce

Steps to reproduce the behavior:

  1. Check out RD-Agent main at commit:
2c878f9d2453dced35061165786d1f31bbff0ab6
  1. Install RD-Agent from source and install pytest:
bash
python -m pip install -e .
python -m pip install pytest
  1. Create test/test_costeer_analyze_error_duplicate_match.py:
python
from types import SimpleNamespace

from rdagent.components.coder.CoSTEER.knowledge_management import (
    CoSTEERRAGStrategyV2,
)
from rdagent.components.knowledge_management.graph import (
    UndirectedNode,
)


class GraphStub:
    def __init__(self, nodes):
        self.nodes = nodes

    def get_all_nodes_by_label_list(self, labels):
        assert labels == ["error"]
        return self.nodes


def test_analyze_error_returns_matched_node_once():
    matched_content = (
        "The source dataframe and the ground truth dataframe "
        "have different rows count."
    )
    missing_content = (
        "Some values differ by more than the tolerance of 1e-6."
    )

    unrelated_node = UndirectedNode(
        content="A different previous error.",
        label="error",
    )
    matched_node = UndirectedNode(
        content=matched_content,
        label="error",
    )

    strategy = CoSTEERRAGStrategyV2.__new__(
        CoSTEERRAGStrategyV2
    )
    strategy.knowledgebase = SimpleNamespace(
        graph=GraphStub(
            [
                unrelated_node,
                matched_node,
            ]
        )
    )

    feedback = (
        matched_content
        + "\n"
        + missing_content
    )

    result = strategy.analyze_error(
        feedback,
        feedback_type="value",
    )

    assert result == [
        matched_node,
        missing_content,
    ]
  1. Run:
bash
python -m pytest \
  test/test_costeer_analyze_error_duplicate_match.py \
  -q
  1. Observe that the test fails because the returned list contains three items instead of two.

Representative observed result:

python
[
    "The source dataframe and the ground truth dataframe have different rows count.",
    UndirectedNode(
        content=(
            "The source dataframe and the ground truth dataframe "
            "have different rows count."
        ),
        label="error",
    ),
    "Some values differ by more than the tolerance of 1e-6.",
]

The first two entries represent the same parsed error content.

Expected Behavior

analyze_error should return exactly one output item for each distinct parsed error content.

For a parsed error that already has a matching graph node, the returned item should be that existing node:

python
matched_node

For a parsed error that does not have a matching graph node, the returned item should remain the original string:

python
missing_content

For the reproduction above, the result should be:

python
[
    matched_node,
    missing_content,
]

Screenshot

Not applicable. This is a deterministic unit-level reproduction.

Environment

Note: Users can run rdagent collect_info to get system information and paste it directly here.

  • Name of current operating system: macOS
  • Processor architecture: arm64
  • System, version, and hardware information: macOS 15.7.3, arm64
  • Version number of the system: 15.7.3
  • Python version: 3.13.2
  • Container ID: Not applicable
  • Container Name: Not applicable
  • Container Status: Not applicable
  • Image ID used by the container: Not applicable
  • Image tag used by the container: Not applicable
  • Container port mapping: Not applicable
  • Container Label: Not applicable
  • Startup Commands: Not applicable
  • RD-Agent version: main@2c878f9d2453dced35061165786d1f31bbff0ab6
  • Package version: Source checkout

Additional Notes

The current implementation parses error_contents, retrieves all graph error nodes, and then loops over both collections:

python
all_error_nodes = (
    self.knowledgebase.graph
    .get_all_nodes_by_label_list(["error"])
)
if not len(all_error_nodes):
    return error_contents
else:
    error_list = []
    for error_content in error_contents:
        for error_node in all_error_nodes:
            if error_content == error_node.content:
                error_list.append(error_node)
            else:
                error_list.append(error_content)
            if error_list[-1] in error_list[:-1]:
                error_list.pop()

    return error_list

The problem is that the else branch runs once for every nonmatching graph node. If an unrelated error node is checked before or after the matching node, the raw string is appended in addition to the matched node.

The ad-hoc duplicate removal does not remove this pair because the string and UndirectedNode object are not equal.

This result is later saved into working trace error analysis:

python
self.knowledgebase.working_trace_error_analysis.setdefault(
    target_task_information,
    [],
).append(error_analysis_result)

and reused by error_query, which turns non-node entries back into graph nodes by content before querying similar successful knowledge.

A possible fix is to decide once per parsed error_content: find the first matching node, append that node if found, otherwise append the string.

For example:

python
error_list = []

for error_content in error_contents:
    matched_node = next(
        (
            error_node
            for error_node in all_error_nodes
            if error_node.content == error_content
        ),
        None,
    )

    error_list.append(
        matched_node
        if matched_node is not None
        else error_content
    )

Regression coverage should include:

  • no existing error nodes;
  • one parsed error with a matching existing node;
  • one parsed error with only nonmatching existing nodes;
  • multiple parsed value-check errors with one match and one nonmatch;
  • graph nodes ordered with the unrelated node before the matching node;
  • graph nodes ordered with the matching node before the unrelated node;
  • preservation of parsed error-content order;
  • no duplicate string/node pair for the same error content.

Targeted issue and pull-request searches found no existing report for this analyze_error matched-node duplication root.