#1333·adk-go

[Feature]: model/openaimodel: support image/file input (genai InlineData/FileData parts)

Author: shuqingzaiCreated Aug 14, 2026Updated Sep 15, 2026
Labelsenhancementneeds review

Problem

The OpenAI model adapter (model/openaimodel) only supports text content parts. When a user passes an image or file (e.g. genai.Part.InlineData / genai.Part.FileData) to an agent using an OpenAI model, convertContents falls through to the default branch and returns:

openai: unsupported content part *genai.Part

Relevant code: model/openaimodel/request.go:116 — the part switch in convertContents handles only Text, FunctionCall, and FunctionResponse. The behavior is even pinned by a test asserting failure: TestBuildOpenAIParams_UnsupportedPart (model/openaimodel/request_test.go:123).

Expected behavior

Support OpenAI-format image/file input in the Responses API:

  • genai.Part.InlineData (*genai.Blob with Data + MimeType) → responses.ResponseInputImageParam (input_image, base64 data:... or file_id) — covers images, and audio/video via input_audio/input_video where applicable.
  • genai.Part.FileData (remote URI + MimeType) → responses.ResponseInputFileParam (input_file), e.g. PDFs.
  • A message containing both text and image parts should be emitted as a single message with a content list mixing input_text and input_image items (the existing newMessage only builds ResponseInputTextParam).

Repro sketch

go
req := &model.LLMRequest{
  Contents: []*genai.Content{{
    Role: string(genai.RoleUser),
    Parts: []*genai.Part{
      {Text: "What is in this image?"},
      {InlineData: &genai.Blob{MimeType: "image/png", Data: pngBytes}},
    },
  }},
}
params, err := buildOpenAIParams("fallback", req) // currently: unsupported content part

Notes

  • The OpenAI Go SDK already provides responses.ResponseInputImageParam and responses.ResponseInputFileParam (used by the Chat Completions→Responses conversion in the same dependency).
  • adk-python already implements this exact behavior — _inline_data_part_to_response_content and _file_data_part_to_response_content in src/google/adk/labs/openai/_openai_responses_llm.py; per AGENTS.md, adk-python is the source of truth for feature behavior, so adk-go should align:

https://github.com/google/adk-python/blob/370027a770b413ab7991afa3395dbf8be1eb89e5/src/google/adk/labs/openai/_openai_responses_llm.py#L343-L380

  • The existing TestBuildOpenAIParams_UnsupportedPart test would need to be updated (it currently asserts the error case).