#2352·Guardrails

feature: Support tools param on /v1/chat/completions

Author: Akazai-ACreated Sep 1, 2026Updated Sep 17, 2026
Labelsenhancementstatus: needs triage

Did you check the docs?

  • I have read all the NeMo-Guardrails docs

Is your feature request related to a problem? Please describe.

Requests to /v1/chat/completions containing the standard OpenAI tools parameter return HTTP 200, but:

  1. tools / tool_choice are silently dropped — no warning, no error, and the model responds with plain text as if no tools were provided.
  2. message.tool_calls is never present in the response — tool calls are stripped even when the upstream model emits them.

There is no way to tell from the client side that tool calling isn't supported — requests just quietly degrade to text-only responses. This makes the endpoint unusable as a drop-in guardrails proxy for tool-calling agents, which is one of the main advertised use cases (agentic security, tool input/output rails).

Tool calling params were added to the server in 0.23.0 (#1942), but limited to `non-streaming. Related open work: #1615, #2056, #2268.

Describe the solution you'd like

  1. /v1/chat/completions accepts tools, tool_choice, and parallel_tool_calls, forwards them to the model, and returns message.tool_calls with finish_reason: "tool_calls" per the OpenAI spec — with tool input/output rails applied.
  2. Streaming support (per #2056), since most agent frameworks and clients stream by default.
  3. Where unsupported, reject requests containing tools with a 422 and a clear error message instead of silently dropping them.

Describe alternatives you've considered

  • Calling the LLM directly and using the checks endpoint for input/output validation
  • Disabling streaming in our client

Additional context

No response