Align dspy.Image validation with Audio/File (add model_validator; make I/O boundary explicit)
Summary
dspy.Image normalizes/validates its input only through a custom __init__, while dspy.Audio and dspy.File use a custom __init__ plus a @pydantic.model_validator(mode="before") (validate_input) that delegates to a shared encode_* function. This asymmetry — introduced/left by #10069 — has an observable consequence and leaves Image's I/O boundary implicit.
Observable inconsistency
A bare data-URI string validates cleanly for Audio/File through the pydantic validation path (nested models, TypeAdapter, output-parse fallback) but fails for Image, even though direct construction of the same value works:
from pydantic import TypeAdapter, BaseModel
import dspy
img_uri = "data:image/png;base64,iVBORw0KGgo...=="
aud_uri = "data:audio/wav;base64,AA=="
file_uri = "data:text/plain;base64,aGk="
TypeAdapter(dspy.Image).validate_python(img_uri) # ValidationError: Input should be a valid dictionary...
TypeAdapter(dspy.Audio).validate_python(aud_uri) # OK
TypeAdapter(dspy.File).validate_python(file_uri) # OK
class Wrap(BaseModel):
x: dspy.Image
Wrap.model_validate({"x": img_uri}) # ValidationError (Audio/File equivalents pass)
dspy.Image(img_uri) # OK (direct positional construction)Why it happens
Pydantic v2 detects Image's custom __init__ (__pydantic_custom_init__) and routes dict validation through it, which is why validate_python({"url": ...}) reaches encode_image and the resource-loading boundary holds. But a bare string never gets there: with no mode="before" validator, pydantic's core schema rejects it ("Input should be a valid dictionary or instance of Image") before any normalization runs. Audio/File's validate_input catches the bare string and normalizes it.
Why it's worth fixing
- Consistency —
Imagecannot be validated from a bare string anywhere pydantic drives validation (nested models,TypeAdapter, parse fallbacks), even for a legitimate data URI; the other two types can. - Explicit security boundary —
Image's parse-path safety currently rests on pydantic'scustom_initrouting dict-validation through__init__. That's an emergent property of pydantic internals rather than a stated invariant; a pydantic upgrade or refactor could shift it silently.Audio/Filemake the boundary explicit invalidate_input.
Proposed change
Give Image a @pydantic.model_validator(mode="before") that delegates to encode_image (handling the {"url": ...} legacy dict, bare data-URI/URL strings, bytes, PIL images, and Image instances), mirroring Audio/File. This closes the bare-string gap and makes the I/O boundary an explicit, testable guarantee.
Optional refinement while touching this: the only reason all three types still need a custom __init__ is positional construction (Image(x)), since BaseModel.__init__ is kwargs-only. The cleanest shape is __init__ does only positional/deprecated-kwarg marshaling (source → {"url": source}), and the before-validator is the single normalization + I/O-boundary point. Audio/File currently encode in both __init__ and the validator (the validator re-checks an already-normalized dict as a no-op); they could be aligned to the same "marshal-then-normalize" split so all three are identical, with each type's logic centralized in one place.
Scope / testing notes
- Re-run
tests/adaptersandtests/signaturesplus the resource-loading boundary tests (tests/adapters/test_resource_loading.py) to confirm both the constructor and parser paths still refuse local paths and non-data-URI strings after the rewiring. - Consider adding a constructor-side "no host I/O" test to complement the existing validator-path guard (tracked alongside the reframing discussed in #10069).
Follow-up to #10069.
Source: stanfordnlp/dspy