[BUG]: ImageLoader cache key lowercases the whole URL, returning the wrong image for case-differing paths
Describe the Bug
ImageLoader.load_image builds its HTTP(S) cache key by lowercasing the entire URL:
key = normalized_url.lower()Per RFC 3986, only the scheme and host are case-insensitive. The path, query, and fragment are case-sensitive, and object stores such as S3 and GCS treat them that way. Two URLs that differ only in path or query case therefore collapse onto a single cache entry, and the second request is served the first request's decoded image without any origin fetch.
The same key also feeds self._inflight, so concurrent requests for case-differing URLs dedupe onto one in-flight fetch as well.
There is no error and no log line at the point of the collision. The vision model simply receives the wrong image and answers about it. Because ImageLoader is a shared, process-wide instance in the vLLM encode worker (encode_worker_handler.py, default cache_size=8), one request's image bytes can be served for a different request's URL while both are inside the LRU window.
This was flagged as a Major review comment on #3634 but was never addressed. It is still present on main today (6562e7d04).
Steps to Reproduce
The defect is on a pure-Python path, so it reproduces with no GPU and no network. Drop this into components/src/dynamo/common/tests/multimodal/test_image_loader.py and run it:
async def test_case_differing_paths_do_not_collide(loader: ImageLoader) -> None:
"""URLs differing only in path case are distinct resources; both must fetch."""
async def _fetch(url, *args, **kwargs):
return _png_of((255, 0, 0)) if "Cat" in url else _png_of((0, 0, 255))
mock = AsyncMock(side_effect=_fetch)
with patch(_FETCH_BYTES_PATH, mock):
a = await loader.load_image("https://example.com/Cat.png")
b = await loader.load_image("https://example.com/cat.png")
assert mock.call_count == 2
assert a.getpixel((0, 0)) == (255, 0, 0)
assert b.getpixel((0, 0)) == (0, 0, 255)
def _png_of(color):
buf = BytesIO()
Image.new("RGB", (2, 2), color).save(buf, format="PNG")
return buf.getvalue()pytest components/src/dynamo/common/tests/multimodal/test_image_loader.py -k case_differing -qThe same behavior reproduces end to end against a real aiohttp origin serving a red image at /Cat.png and a blue image at /cat.png: the second load_image call returns red and the server records only one request. With the cache disabled the same pair correctly returns red then blue with two origin hits, which isolates the cause to the key construction rather than the validator or the fetcher.
Expected Behavior
https://example.com/Cat.png and https://example.com/cat.png are distinct resources. Each should be fetched from the origin and each should decode to its own image. Only scheme and host case should be normalized away, so https://EXAMPLE.com/img.png and https://example.com/img.png still share one cache entry.
Actual Behavior
mock.call_count == 1 (expected 2)The second URL is never fetched. load_image("https://example.com/cat.png") returns the image decoded from /Cat.png. Ordering decides the winner: whichever spelling arrives first populates the entry, and every later case variant within the LRU window receives it.
Environment
The defect is in Python source and does not depend on the runtime environment.
- ai-dynamo: source checkout of
mainat6562e7d04(also verified at5593e8857) - Python: 3.12
- Operating System: macOS 15 (arm64); the code path is platform independent
- GPU / CUDA: not required, the failing path is pure Python and never reaches a device
Additional Context
Suggested fix, which keeps the existing case-insensitivity for the parts of a URL that genuinely are case-insensitive:
# Scheme and host are case-insensitive (RFC 3986); path, query, and
# fragment are not.
prefix_len = len(parsed_url.scheme) + 3 + len(parsed_url.netloc)
key = normalized_url[:prefix_len].lower() + normalized_url[prefix_len:]Two supporting notes:
- The Rust frontend's
MediaLoaderhashes the full URL string without case folding, so the Python path is the outlier. - The LRU is in-memory and rebuilt on every boot, so changing the key format needs no migration.
I have this fix plus regression tests ready and will open a PR referencing this issue. Running the fix against the existing test_image_loader.py suite leaves all current tests passing.
Source: ai-dynamo/dynamo