Improve MCP tool retrieval diagnostics and fallback observability

Self check

• I searched existing issues (incl. closed) — no duplicate.

What's the problem?

Currently, MCP tool retrieval is safe but mostly opaque.

When mcp_tool_retrieval_enabled is enabled and the number of MCP tools exceeds the configured threshold, AgentStreamExecutor._select_tools_for_injection() uses build_retrieval_query() and select_mcp_tools() to choose a subset of MCP tools.

The current behavior is good in several ways: Built-in tools are always injected. MCP tools fall back to full injection when retrieval is disabled, below threshold, or when embedding/index data is unavailable. The selected MCP tool set only grows within a run, so previously used tool schemas do not disappear.

However, the retrieval decision itself is not observable. select_mcp_tools() currently returns only a set[str] or None, so it is hard to know: which MCP tools were ranked, which tools were selected in the top-k set, whether retrieval fell back to full injection, why fallback happened, such as missing query vector, empty index, dimension mismatch, or invalid top_k.

This makes MCP routing harder to debug and harder to evaluate without reading logs or stepping through the code.

What would you like?

I would like to contribute a small, focused PR that improves MCP retrieval observability without changing the existing tool selection behavior. My proposed scope is: 1.Keep the existing select_mcp_tools() public behavior unchanged. 2.Add lightweight internal retrieval decision metadata, such as selected tool names, candidate count, top-k, scores, and fallback reason. 3.Emit a backward-compatible tool_retrieval event from the agent stream so clients or logs can inspect the retrieval decision. 4.Extend the existing deterministic tests in tests/test_mcp_tool_retrieval.py to cover ranking metadata and fallback reasons. 5.Avoid UI changes and new dependencies in the first PR.

The goal is to make MCP tool routing easier to debug and evaluate while preserving the current fail-open behavior.

Would this kind of observability-focused contribution fit the project roadmap? If yes, do you have a preferred event schema or naming convention for the retrieval metadata?

Contribution

• I'd be interested in helping implement this.