Bug: ModelMultiProcessEvolvingStrategy raises IndexError when queried_knowledge is None

Author: yifanxiong272Created Aug 29, 2026Updated Aug 29, 2026

Labelsbug

Summary

ModelMultiProcessEvolvingStrategy.implement_one_task declares queried_knowledge as optional and defaults it to None:

python

queried_knowledge: CoSTEERQueriedKnowledge | None = None

The method also contains explicit fallback branches for None. However, the fallback for queried_former_failed_knowledge is an empty list, which is immediately accessed at indexes 0 and 1.

Calling the method with its declared default therefore raises:

IndexError: list index out of range

before prompt construction or model invocation begins.

To Reproduce

Check out RD-Agent main at commit 6762f84f9bc0f5c6486c50a00e128a57ac6c3683.
Install RD-Agent from source.
Create test/scenarios/data_science/test_model_strategy_none_knowledge.py:

python

from types import SimpleNamespace

import rdagent.components.coder.data_science.model as model_module
from rdagent.components.coder.data_science.model import (
    ModelMultiProcessEvolvingStrategy,
)


def test_model_strategy_accepts_none_queried_knowledge(
    monkeypatch,
):
    class FakeTemplate:
        def r(self, *args, **kwargs):
            return "prompt"

    monkeypatch.setattr(
        model_module,
        "T",
        lambda *args, **kwargs: FakeTemplate(),
    )
    monkeypatch.setattr(
        model_module.DS_RD_SETTING,
        "spec_enabled",
        True,
    )
    monkeypatch.setattr(
        model_module.PythonBatchEditOut,
        "extract_output",
        lambda *args, **kwargs: {
            "model_generated.py": "new code"
        },
    )

    class FakeBackend:
        def build_messages_and_create_chat_completion(
            self,
            **kwargs,
        ):
            return "response"

    monkeypatch.setattr(
        model_module,
        "APIBackend",
        FakeBackend,
    )

    strategy = object.__new__(
        ModelMultiProcessEvolvingStrategy
    )
    strategy.scen = SimpleNamespace(
        get_scenario_all_desc=lambda eda_output=None: (
            "scenario"
        ),
    )

    target_task = SimpleNamespace(
        name="model_1",
        get_task_information=lambda: "model task",
    )

    class FakeWorkspace:
        file_dict = {
            "model_1.py": "old code",
            "feature.py": "feature code",
            "load_data.py": "loader code",
            "spec/model.md": "model spec",
        }

        def get_codes(self, pattern):
            return {"model_1.py": "old code"}

    result = strategy.implement_one_task(
        target_task=target_task,
        queried_knowledge=None,
        workspace=FakeWorkspace(),
    )

    assert result == {
        "model_1.py": "new code",
    }

Run:

bash

python -m pytest \
    test/scenarios/data_science/test_model_strategy_none_knowledge.py \
    -q

Observe that the test fails before any backend call is made.

Expected Behavior

Passing the declared optional value queried_knowledge=None should not cause an incidental list-indexing failure.

Consistent with the method's existing None branches, no queried knowledge can be represented as empty successful and failed knowledge collections, allowing implementation to continue.

If None is intentionally unsupported, the method should instead reject it immediately with a clear exception such as:

python

ValueError(
    "queried_knowledge is required"
)

and its annotation and default value should reflect that requirement.

Actual Behavior

The method raises:

IndexError: list index out of range

The failure occurs before prompt construction or model invocation:

rdagent/components/coder/data_science/model/__init__.py:54

The relevant operation is:

python

queried_former_failed_knowledge[0]

Impact

Direct callers and integrations that invoke implement_one_task using its declared default cannot use the strategy without constructing a CoSTEERQueriedKnowledge object.

The repository's commented development example also calls this method with queried_knowledge=None, so following that example encounters the same failure.

The normal higher-level ModelCoSTEER flow typically supplies queried knowledge. The impact is therefore concentrated on direct strategy reuse, development utilities, tests, and integrations using the method-level interface.

Screenshot

Not applicable; this is a deterministic unit-level reproduction.

Environment

Name of current operating system: macOS
Processor architecture: arm64
Python version: 3.11.15
RD-Agent version: 0.8.0, main@6762f84f9bc0f5c6486c50a00e128a57ac6c3683
Package version: pytest 9.1.1
Container: not used in this reproduction

Additional Notes

The current implementation first handles None explicitly:

python

queried_former_failed_knowledge = (
    queried_knowledge.task_to_former_failed_traces[
        model_information_str
    ]
    if queried_knowledge is not None
    else []
)

It then assumes that the result is a two-element structure:

python

queried_former_failed_knowledge = (
    [
        knowledge
        for knowledge
        in queried_former_failed_knowledge[0]
        if knowledge.implementation.file_dict.get(
            f"{target_task.name}.py"
        )
        != workspace.file_dict.get(
            f"{target_task.name}.py"
        )
    ],
    queried_former_failed_knowledge[1],
)

The None fallback and the following data-shape assumption are inconsistent.

If None is supported, a shape-compatible fallback could be used:

python

queried_former_failed_knowledge = (
    queried_knowledge.task_to_former_failed_traces[
        model_information_str
    ]
    if queried_knowledge is not None
    else ([], None)
)

The same fallback/indexing pattern appears in other data-science evolving strategies, so corresponding paths may also need review.

Regression coverage should include:

queried_knowledge=None;
an empty queried-knowledge result;
existing failed-trace knowledge;
filtering knowledge whose implementation matches the workspace;
a clear early validation error if knowledge is intentionally required.

Source: microsoft/RD-Agent

View original on GitHub View discussion on GitHub