[Bug]: LLM and pydantic selectors silently route to the last choice on 0-indexed answers
Bug Description
Both selector paths in llama_index.core.selectors ask the model to answer with 1-based choice numbers (_build_choices_text enumerates choices as (1), (2), ...), then subtract 1 from the model's answer without validating it:
_structured_output_to_selector_result(llm_selectors.py):SingleSelection(index=answer.choice - 1, ...)_pydantic_output_to_selector_result(pydantic_selectors.py):output.index -= 1
Models do sometimes answer with 0-indexed or over-range numbers. When that happens:
choice = 0becomesindex = -1, and Python's negative indexing silently selects the last choice. InRouterQueryEnginethis isself._query_engines[result.ind], so the query goes to the wrong engine with no error. The same silent misrouting applies anywhereSelectorResult.ind/indsis used to index into the choice list.- An over-range choice (e.g.
choice = 5with 3 choices) passes through and crashes downstream with an unhelpfulIndexError: list index out of range.
This is the same failure mode that #22827 fixed for StructuredLLMRerank (a document_number of 0 mapping to nodes_batch[-1]); the selectors never got the same guard.
Proposed fix, mirroring #22827: validate selections against the number of choices, drop out-of-range entries, and raise a clear ValueError when nothing valid remains (a single selector cannot return an empty result). I have the patch and regression tests ready - happy to send the PR.
Version
llama-index-core 0.14.24 (also verified on current main, fd4a517)
Steps to Reproduce
from llama_index.core.selectors.pydantic_selectors import _pydantic_output_to_selector_result
from llama_index.core.selectors.llm_selectors import _structured_output_to_selector_result
from llama_index.core.base.base_selector import SingleSelection
from llama_index.core.output_parsers.base import StructuredOutput
from llama_index.core.output_parsers.selection import Answer
# A 0-indexed answer (model meant the FIRST choice)
r = _pydantic_output_to_selector_result(SingleSelection(index=0, reason="first"), 3)
print(r.selections[0].index) # -1
engines = ["engine_a", "engine_b", "engine_c"]
print(engines[r.selections[0].index]) # engine_c - silently the LAST engine
r2 = _structured_output_to_selector_result(
StructuredOutput(raw_output="", parsed_output=[Answer(choice=0, reason="first")]), 3
)
print(r2.selections[0].index) # -1
# Over-range passes through, then crashes at the indexing site
r3 = _pydantic_output_to_selector_result(SingleSelection(index=99, reason="over"), 3)
print(engines[r3.selections[0].index]) # IndexError
Output:
-1
engine_c - silently the LAST engine
-1
Traceback (most recent call last):
...
IndexError: list index out of range
Source: run-llama/llama_index