[Feature]: model/openaimodel: support image/file input (genai InlineData/FileData parts)
Author: shuqingzaiCreated Aug 14, 2026Updated Sep 15, 2026
Labelsenhancementneeds review
Problem
The OpenAI model adapter (model/openaimodel) only supports text content parts. When a user passes an image or file (e.g. genai.Part.InlineData / genai.Part.FileData) to an agent using an OpenAI model, convertContents falls through to the default branch and returns:
openai: unsupported content part *genai.PartRelevant code: model/openaimodel/request.go:116 — the part switch in convertContents handles only Text, FunctionCall, and FunctionResponse. The behavior is even pinned by a test asserting failure: TestBuildOpenAIParams_UnsupportedPart (model/openaimodel/request_test.go:123).
Expected behavior
Support OpenAI-format image/file input in the Responses API:
genai.Part.InlineData(*genai.BlobwithData+MimeType) →responses.ResponseInputImageParam(input_image, base64data:...orfile_id) — covers images, and audio/video viainput_audio/input_videowhere applicable.genai.Part.FileData(remoteURI+MimeType) →responses.ResponseInputFileParam(input_file), e.g. PDFs.- A message containing both text and image parts should be emitted as a single message with a content list mixing
input_textandinput_imageitems (the existingnewMessageonly buildsResponseInputTextParam).
Repro sketch
req := &model.LLMRequest{
Contents: []*genai.Content{{
Role: string(genai.RoleUser),
Parts: []*genai.Part{
{Text: "What is in this image?"},
{InlineData: &genai.Blob{MimeType: "image/png", Data: pngBytes}},
},
}},
}
params, err := buildOpenAIParams("fallback", req) // currently: unsupported content partNotes
- The OpenAI Go SDK already provides
responses.ResponseInputImageParamandresponses.ResponseInputFileParam(used by the Chat Completions→Responses conversion in the same dependency). - adk-python already implements this exact behavior —
_inline_data_part_to_response_contentand_file_data_part_to_response_contentinsrc/google/adk/labs/openai/_openai_responses_llm.py; perAGENTS.md, adk-python is the source of truth for feature behavior, so adk-go should align:
- The existing
TestBuildOpenAIParams_UnsupportedParttest would need to be updated (it currently asserts the error case).
Source: google/adk-go