Legacy OpenAI_Chat: temperature=0.7 causes 400 with models that only support default

Legacy OpenAI_Chat: temperature=0.7 causes 400 with models that only support default

Description

When using Vanna legacy OpenAI_Chat (e.g. with Azure OpenAI), some models do not support the temperature parameter and return 400 Bad Request if any value other than the API default is sent.

Example error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'temperature' does not support 0.7 with this model. Only the default (1) value is supported.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_value'}}

Observed with:

• Azure OpenAI deployment gpt-5-nano
• Likely same class of models as in #855 (e.g. o3-mini) where temperature is not supported

Root cause: In vanna/legacy/openai/openai_chat.py, self.temperature defaults to 0.7 and is always passed to client.chat.completions.create(..., temperature=self.temperature). There is no way to omit the parameter for models that only accept the API default.

Workaround we used

We override via config when building our Vanna instance:

OpenAI_Chat.__init__(
    self,
    client=AzureOpenAI(...),
    config={
        "model": AZURE_DEPLOYMENT,
        "temperature": 1,  # required for gpt-5-nano; only 1 is supported
        **config,
    },
)

This works but is a workaround: we had to hardcode 1 for this model. Other models might require the parameter to be omitted entirely rather than set to 1.

Suggested fix (align with #855)

Make the temperature parameter optional in the legacy OpenAI chat client:

• If config["temperature"] is None (or not set and a new default of None is used), do not include temperature in the request to the OpenAI API.
• If config["temperature"] is a number, pass it as today.

That way:

• Models that only support the default get no temperature argument.
• Models that support tuning can still get temperature via config.

Same idea as Make Temperature parameter optional #855, but specifically for the legacy openai_chat path (e.g. vanna/legacy/openai/openai_chat.py), where the call is synchronous and always sends temperature=self.temperature.

Environment

• Vanna: 2.x with legacy API (vanna.legacy.openai.openai_chat, vanna.legacy.chromadb, vanna.legacy.flask)
• Backend: Azure OpenAI
• Model/deployment: gpt-5-nano (Azure)
• Python: 3.12

References

• #855 – Make Temperature parameter optional (same idea for newer models)
• #262 – Allow setting temperature via config (we use this for the workaround)