[ai-proxy] qwen provider 多模态请求仅支持 qwen-vl 前缀模型,其他多模态模型报 InvalidParameter || [ai-proxy] qwen provider multi-modal request only supports qwen-vl prefix model, other multi-modal models report InvalidParameter
- I have searched the issues of this repository and believe this is not a duplicate.
1. Issue Description
ai-proxy 插件的 qwen provider 在代理通义千问多模态请求时,是否走多模态 API 是通过硬编码的模型名前缀判断的,导致只有 qwen-vl 前缀的模型(即 qwen-vl-plus / qwen-vl-max)可以正常使用多模态能力,其他所有多模态模型(如 qwen2.5-vl-*、qwen3-vl-*、qvq-*,以及不带 vl 命名的多模态模型)一律返回百炼的 invalidParameter 错误。
2. Describe what happened
按官方文档《使用 OpenAI 协议代理通义千问服务》配置 qwen provider(默认 qwenEnableCompatible: false),发送多模态请求(content 为数组,含 image_url 与 text):
model映射为qwen-vl-plus/qwen-vl-max时:正常返回model映射为其他多模态模型(实测qwen3.5-flash)时:报错invalidParameter
qwen3.5-flash 的完整报错响应:
{
"request_id": "71f8df41-c634-9692-a304-953692658940",
"code": "InvalidParameter",
"message": "url error, please check url! For details, see: https://help.aliyun.com/zh/model-studio/error-code#error-url"
}报错信息为 url error,即请求虽然到达了百炼,但被路由到了与该模型不匹配的 API 端点(文本生成接口),与下文根因分析一致。
3. Describe what you expected to happen
按百炼模型列表,任意支持多模态的模型的多模态请求都应被正确代理,而不应受模型命名前缀限制。
4. How to reproduce it (as minimally and precisely as possible)
- 配置 ai-proxy 插件(qwen provider,未开启
qwenEnableCompatible):
provider:
type: qwen
apiTokens:
- "YOUR_DASHSCOPE_API_TOKEN"
modelMapping:
"*": ""- 发送多模态请求(与官方文档"多模态模型 API 请求示例"相同):
curl http://<your-domain>/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwen3.5-flash",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": { "url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg" }
},
{ "type": "text", "text": "这个图片是哪里?" }
]
}
]
}'- 将
model换成qwen-vl-plus重发,即可正常返回——两者形成对比。
5. Anything else we need to know?
根因分析(plugins/wasm-go/extensions/ai-proxy/provider/qwen.go,main 分支):
// qwen.go:54
qwenVlModelPrefixName = "qwen-vl"
// qwen.go:190-191 (onChatCompletionRequestBody)
// Use the qwen multimodal model generation API
if strings.HasPrefix(request.Model, qwenVlModelPrefixName) {
util.OverwriteRequestPathHeader(headers, qwenMultimodalGenerationPath)
}
// qwen.go:395-396 (流式响应处理) 同样基于该前缀判断仅当模型名以 qwen-vl 开头时,插件才将请求改写为 DashScope 原生多模态接口 /api/v1/services/aigc/multimodal-generation/generation;否则按文本生成接口转换请求体。多模态请求被发到文本端点后,百炼返回 invalidParameter。
而 qwen2.5-vl-*(前缀 qwen2.5-vl)、qwen3-vl-*、qvq-* 以及后续不带 vl 命名的多模态模型都无法命中该前缀。官方文档其实也已明示了这一限制("多模态模型 API 请求示例(适用于 qwen-vl-plus 和 qwen-vl-max 模型)")。
修复建议:将判断依据从"模型名前缀"改为"请求内容"——在请求转换阶段检测 content,当任一 message 的 content 为数组且包含 image_url / video_url / audio_url 等多模态 part 时,自动走 multimodal-generation 路径(或将多模态请求直接路由到 compatible-mode 端点)。现有前缀判断可作为兜底保留。这样对用户零配置,且天然兼容未来任意模型命名。
临时规避:配置 qwenEnableCompatible: true,走 /compatible-mode/v1/chat/completions 兼容模式接口(请求/响应不改写),所有模型的多模态请求均可正常使用。
6. Environment:
- Higress version: 独立版 2.1.9(all-in-one / standalone)
- OS: Windows 11(Docker Compose 部署)
- Others: ai-proxy 插件版本 2.0.2(wasm 镜像
higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:2.0.2),已核实 GitHub main 分支代码仍存在同样逻辑
- I have searched the issues of this repository and believe this is not a duplicate.
1. Issue Description
When the qwen provider of the ai-proxy plug-in proxies Tongyi Qianwen multimodal request, whether to use the multimodal API is determined by the hard-coded model name prefix, resulting in only models with the qwen-vl prefix (i.e. qwen-vl-plus / qwen-vl-max) can use the multimodal capability normally, and all other multimodal models (such as qwen2.5-vl-*, qwen3-vl-*, qvq-*, and multimodal models without vl names) will always return Bailian's invalidParameter error.
2. Describe what happened
According to the official document "Use OpenAI protocol to proxy Tongyi Qianwen service" configure qwen provider (default qwenEnableCompatible: false) and send multi-modal requests (content is an array, including image_url and text):
- When
modelis mapped toqwen-vl-plus/qwen-vl-max: Return normally - When
modelis mapped to other multi-modal models (actual measurementqwen3.5-flash): errorinvalidParameter
The complete error response of qwen3.5-flash:
{
"request_id": "71f8df41-c634-9692-a304-953692658940",
"code": "InvalidParameter",
"message": "url error, please check url! For details, see: https://help.aliyun.com/zh/model-studio/error-code#error-url"
} The error message is url error, which means that although the request reached Bailian, it was routed to an API endpoint (text generation interface) that does not match the model, which is consistent with the root cause analysis below.
3. Describe what you expected to happen
According to the Bailian model list, multimodal requests for any model that supports multimodality should be correctly proxied and should not be restricted by the model naming prefix.
4. How to reproduce it (as minimally and precisely as possible)
- Configure ai-proxy plug-in (qwen provider,
qwenEnableCompatibleis not enabled):
provider:
type: qwen
apiTokens:
- "YOUR_DASHSCOPE_API_TOKEN"
modelMapping:
"*": "" - Send a multimodal request (same as the official document "Multimodal Model API Request Example"):
curl http://<your-domain>/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwen3.5-flash",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": { "url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg" }
},
{ "type": "text", "text": "Where is this picture?" }
]
}
]
}' - Replace
modelwithqwen-vl-plusand resend, it will return normally - the two are in contrast.
5. Anything else we need to know?
Root cause analysis (plugins/wasm-go/extensions/ai-proxy/provider/qwen.go, main branch):
// qwen.go:54
qwenVlModelPrefixName = "qwen-vl"
// qwen.go:190-191 (onChatCompletionRequestBody)
// Use the qwen multimodal model generation API
if strings.HasPrefix(request.Model, qwenVlModelPrefixName) {
util.OverwriteRequestPathHeader(headers, qwenMultimodalGenerationPath)
}
// qwen.go:395-396 (streaming response processing) is also judged based on this prefix Only when the model name starts with qwen-vl, the plug-in rewrites the request to the DashScope native multimodal interface /api/v1/services/aigc/multimodal-generation/generation; otherwise, the request body is converted according to the text generation interface. After the multimodal request is sent to the text endpoint, Bailian returns invalidParameter.
However, qwen2.5-vl-* (prefix qwen2.5-vl), qwen3-vl-*, qvq-* and subsequent multi-modal models named without vl cannot hit this prefix. The official documentation actually states this limitation ("Multimodal model API request example (applicable to qwen-vl-plus and qwen-vl-max models)").
Repair suggestion: Change the judgment basis from "model name prefix" to "request content" - detect content during the request conversion phase. When the content of any message is an array and contains multi-modal parts such as image_url / video_url / audio_url, automatically take the multimodal-generation path (or route multi-modal requests directly to the compatible-mode endpoint). The existing prefix judgment can be retained as a back-up. This requires zero configuration for users and is naturally compatible with any model naming in the future.
Temporary circumvention: Configure qwenEnableCompatible: true, use /compatible-mode/v1/chat/completions compatibility mode interface (request/response are not rewritten), multi-modal requests of all models can be used normally.
6. Environment:
- Higress version: standalone version 2.1.9 (all-in-one / standalone)
- OS: Windows 11 (Docker Compose deployment)
- Others: ai-proxy plug-in version 2.0.2 (wasm mirror
higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-proxy:2.0.2), it has been verified that the same logic still exists in the GitHub main branch code
Source: higress-group/higress