[Bug] Cache shares mutable results with callers and stores values it cannot read back
What happened?
On a cache miss, request_cache stores the object returned by the decorated function and returns that same object to the caller. The cache therefore does not own an independent snapshot of the result. If the caller mutates the first result, it also mutates the value that future cache hits will return.
There is a related failure mode when the result cannot be deep-copied. The first call succeeds and the result enters the memory cache, but an identical second call fails when the cache tries to deep-copy that stored value. This can happen with completed, non-streaming results, so it is independent of the raw streaming behavior reported in #10345.
Steps to reproduce
Reproduction: caller mutation changes the cached value
import dspy
from dspy.clients.cache import request_cache
dspy.configure_cache(enable_memory_cache=True, enable_disk_cache=False)
dspy.cache.reset_memory_cache()
calls = 0
@request_cache()
def get_response(*, prompt):
global calls
calls += 1
return {"text": "ORIGINAL"}
first = get_response(prompt="hello")
first["text"] = "MUTATED"
second = get_response(prompt="hello")
assert calls == 1
assert second["text"] == "ORIGINAL" # Fails: the cached value is "MUTATED".
Reproduction: a non-copyable result breaks the matching cache hit
import threading
import dspy
from dspy.clients.cache import request_cache
dspy.configure_cache(enable_memory_cache=True, enable_disk_cache=False)
dspy.cache.reset_memory_cache()
calls = 0
@request_cache()
def get_response(*, prompt):
global calls
calls += 1
return {"text": "SUCCESS", "lock": threading.RLock()}
assert get_response(prompt="hello")["text"] == "SUCCESS"
get_response(prompt="hello") # Raises TypeError: cannot pickle '_thread.RLock' object.
The first call returns normally and inserts the result. The second call fails while preparing the matching memory-cache entry, before the decorated function can run again.
Expected behavior
The cache should own an independent snapshot of every stored result. Mutating an object returned to the caller should not change later cache hits.
If DSPy cannot create a retrievable snapshot of a result, the call should still return that result normally but should treat it as uncacheable. If an existing memory or disk entry cannot be prepared for return, DSPy should treat it as a cache miss and safely remove the unreadable entry so the request can be recomputed.
Additional context
I found this while investigating #10345, but it does not require stream=True or an LM provider. #10345 exposed the same cache-layer assumption using LiteLLM's live stream wrapper; the examples above reproduce the cache behavior with ordinary Python values.
I prepared #10348 with regression tests and a proposed fix. It stores an independent snapshot, skips values that cannot be copied, and safely evicts unreadable entries while preserving valid concurrent disk-cache replacements.
DSPy version
main at f70d08a5b934400d236078143e3704934f2d5dd1 (3.3.1)
Source: stanfordnlp/dspy