feature: Support tools param on /v1/chat/completions
Author: Akazai-ACreated Sep 1, 2026Updated Sep 17, 2026
Labelsenhancementstatus: needs triage
Did you check the docs?
- I have read all the NeMo-Guardrails docs
Is your feature request related to a problem? Please describe.
Requests to /v1/chat/completions containing the standard OpenAI tools parameter
return HTTP 200, but:
tools/tool_choiceare silently dropped — no warning, no error, and the model responds with plain text as if no tools were provided.message.tool_callsis never present in the response — tool calls are stripped even when the upstream model emits them.
There is no way to tell from the client side that tool calling isn't supported — requests just quietly degrade to text-only responses. This makes the endpoint unusable as a drop-in guardrails proxy for tool-calling agents, which is one of the main advertised use cases (agentic security, tool input/output rails).
Tool calling params were added to the server in 0.23.0 (#1942), but limited to `non-streaming. Related open work: #1615, #2056, #2268.
Describe the solution you'd like
/v1/chat/completionsacceptstools,tool_choice, andparallel_tool_calls, forwards them to the model, and returnsmessage.tool_callswithfinish_reason: "tool_calls"per the OpenAI spec — with tool input/output rails applied.- Streaming support (per #2056), since most agent frameworks and clients stream by default.
- Where unsupported, reject requests containing
toolswith a 422 and a clear error message instead of silently dropping them.
Describe alternatives you've considered
- Calling the LLM directly and using the checks endpoint for input/output validation
- Disabling streaming in our client
Additional context
No response
Source: NVIDIA-NeMo/Guardrails