[Bug] JSONAdapter schema mutation causes repeated cache misses with optional output fields (OpenAI/LiteLLM)

What happened?

With JSONAdapter and an output schema containing default=None, repeated identical requests can keep missing DSPy's cache even though a response is already stored. This also reproduces with a memory-only cache, so it does not require a disk-cache issue.

The cache key changes during LiteLLM's OpenAI response-format conversion:

1. _get_structured_outputs_response_format() creates a Pydantic class and overrides model_json_schema with a lambda returning the same precomputed mutable dictionary.
2. request_cache computes the lookup key from that schema and deep-copies the request before calling the provider.
3. LiteLLM's type_to_response_format_param() invokes OpenAI's strict schema conversion, which mutates the shared dictionary (for example, removing default: null).
4. The deep copy does not isolate this change: copy.deepcopy(request)["response_format"] is request["response_format"], since the value is a class.
5. cache.put() computes a different key from the now-mutated schema. On the next JSONAdapter call, a fresh class has the original schema again, so lookup misses and the existing write-key entry is overwritten.

Expected: three identical requests should call the provider once and reuse the response twice.

Actual in the reproduction: optional output -> three provider calls and one repeatedly overwritten cache entry; required-output control -> one provider call and two cache hits.

Related to the closed Gemini report #8393. This report demonstrates the same mutable-schema failure mechanism through LiteLLM/OpenAI strict schema conversion, with a credential-free executable reproduction and a control case.

I reproduced this with the installed versions below. I also inspected main at c251b89c157d5f0dae916e55967ed071aa248ad3: the shared schema dictionary and request deep copies / recomputed cache keys are still present. I have not run the reproduction against a main installation.

Potential fix directions are to isolate the schema returned for downstream conversion or to freeze the request's cache representation/key before the provider call. Merely deep-copying the dictionary containing the Pydantic class does not isolate its schema.

Steps to reproduce

Run this as a standalone Python script in an environment with the versions below. No credentials or LLM API calls are needed. It uses the actual DSPy schema builder/cache decorator and LiteLLM schema conversion; only the provider response is simulated. The import-time disk cache is isolated in a temporary directory, and the reproduction itself uses memory-only caching.

# %%
"""Reproduce JSONAdapter cache-key mutation without credentials or API calls."""

import copy
import os
from tempfile import TemporaryDirectory
from typing import Any

# Isolate DSPy's import-time disk cache from any existing user cache.
cache_directory = TemporaryDirectory(prefix="dspy-repro-")
os.environ["DSPY_CACHEDIR"] = cache_directory.name
os.environ["LITELLM_LOCAL_MODEL_COST_MAP"] = "True"

import dspy
from dspy.adapters.json_adapter import _get_structured_outputs_response_format
from dspy.clients.cache import request_cache
from litellm.llms.base_llm.base_utils import type_to_response_format_param


class OptionalAnswer(dspy.Signature):
    """Return an optional answer."""

    question: str = dspy.InputField()
    answer: str | None = dspy.OutputField(default=None)


class RequiredAnswer(dspy.Signature):
    """Return a required answer as a control case."""

    question: str = dspy.InputField()
    answer: str = dspy.OutputField()


def demonstrate(signature: type[dspy.Signature]) -> None:
    """Repeat identical requests with freshly generated JSONAdapter schemas."""
    dspy.configure_cache(enable_disk_cache=False, enable_memory_cache=True)
    provider_calls = 0

    @request_cache(cache_arg_name="request")
    def fake_completion(*, request: dict[str, Any]) -> dict[str, str]:
        """Run the real schema conversion, but never contact a provider."""
        nonlocal provider_calls
        provider_calls += 1
        type_to_response_format_param(request["response_format"])
        return {"answer": "example"}

    for attempt in range(1, 4):
        schema_model = _get_structured_outputs_response_format(signature)
        request = {
            "model": "openai/example",
            "messages": [{"role": "user", "content": "Same question"}],
            "response_format": schema_model,
        }
        assert copy.deepcopy(request)["response_format"] is schema_model
        key_before = dspy.cache.cache_key(request)
        fake_completion(request=request)
        key_after = dspy.cache.cache_key(request)
        print(
            f"{signature.__name__}: attempt={attempt}, "
            f"provider_calls={provider_calls}, "
            f"key_changed={key_before != key_after}, "
            f"stored_entries={len(dspy.cache.memory_cache)}"
        )


# %%
demonstrate(OptionalAnswer)
demonstrate(RequiredAnswer)

Observed output:

OptionalAnswer: attempt=1, provider_calls=1, key_changed=True, stored_entries=1
OptionalAnswer: attempt=2, provider_calls=2, key_changed=True, stored_entries=1
OptionalAnswer: attempt=3, provider_calls=3, key_changed=True, stored_entries=1
RequiredAnswer: attempt=1, provider_calls=1, key_changed=False, stored_entries=1
RequiredAnswer: attempt=2, provider_calls=1, key_changed=False, stored_entries=1
RequiredAnswer: attempt=3, provider_calls=1, key_changed=False, stored_entries=1

DSPy version

• DSPy: 3.3.0
• LiteLLM: 1.97.0
• OpenAI Python SDK: 2.54.0
• Pydantic: 2.13.4
• diskcache: 5.6.3
• Python: 3.12.13
• OS: macOS, Apple Silicon