ai-proxy: hunyuan original-protocol chat silently drops native fields (MaxTokens, Stop, ...) - token limits ignored; embeddings impossible with native creds + OpenAI protocol
Summary
Two hunyuan request-handling gaps, both verified on a live gateway:
- Under
protocol: original, chat request bodies are re-marshalled throughhunyuanTextGenRequest, which silently drops every native field the struct does not model — includingMaxTokensandStop.protocol: originalis documented as native passthrough; in practice a 7-field struct decides what survives. A user settingMaxTokens: 100gets no error and no limit — the field is stripped before the request reaches Tencent. - With native TC3 credentials configured, OpenAI-protocol embeddings requests (
/v1/embeddings) fail with a hard 500 — the OpenAI→hunyuan converter assumes every request is a chat completion.
Environment
- Higress images: official
v2.2.4controller/pilot/gateway, k8s v1.34 (kind) - ai-proxy wasm built from
main@943b31bc(includes #4220) - Provider config:
type: hunyuanwithhunyuanAuthId/hunyuanAuthKeyset (native TC3 mode)
Finding 1: silent native-field loss under protocol: original
Request (protocol: original, native hunyuan body):
{"Model":"hunyuan-turbo","Messages":[{"Role":"user","Content":"hi"}],"MaxTokens":100,"TopP":0.9,"Temperature":0.7,"Stop":["END"],"EnableEnhancement":false}Body actually sent upstream (from the plugin's own request-body log on the gateway):
{"Model":"hunyuan-turbo","Messages":[{"Role":"user","Content":"hi"}],"TopP":0.9,"Temperature":0.7}MaxTokens, Stop and EnableEnhancement are gone; the client still receives HTTP 200 from the conversation. hunyuanTextGenRequest (hunyuan.go:50-58) models only Model/Messages/Stream/StreamModeration/TopP/Temperature/EnableEnhancement, and the original-protocol branch unmarshals into it and re-serialises (hunyuan.go:173-178), so any other documented native parameter is silently discarded.
Impact: token limits ignored (cost control defeated), stop sequences ignored, and any future Tencent-side parameter is dropped by default — with no error and no log indicating loss.
The same branch is also the cause of the embeddings Input loss fixed in #4872; that PR special-cases embeddings, but chat keeps the lossy re-marshal.
Finding 2: OpenAI-protocol embeddings impossible with native credentials
With hunyuanAuthId/hunyuanAuthKey set and the default OpenAI protocol, POST /v1/embeddings with a standard OpenAI body:
{"model":"hunyuan-embedding","input":["hello world"]}fails with HTTP 500 and the plugin log:
[onHttpRequestBody] failed to process request body, apiName=openai/v1/embeddings,
err=no message found in the request body: {"model":"hunyuan-embedding","input":["hello world"]}OnRequestBody treats the request as a chat completion (decodeChatCompletionRequest) whenever native credentials are configured, regardless of apiName. Tencent exposes an Embeddings TC3 action (already used by #4872), so converting OpenAI embeddings to the native action would make this path work; today it is a hard 500 with a chat-specific error message.
Expected behavior
protocol: originalshould pass the request body through untouched (the plugin's own doc position for this mode), signing the raw bytes — at minimum when no context file is configured, since context insertion is the only reason a parse is needed.- OpenAI-protocol embeddings with native credentials should either be converted to the native
Embeddingsaction or rejected with an accurate "unsupported" error, not a chat-parse 500.
Source: higress-group/higress