[Bug] Closing a partially consumed streamify async stream raises BaseExceptionGroup[GeneratorExit]
What happened?
Closing a partially consumed async generator returned by dspy.streamify() raises a BaseExceptionGroup containing GeneratorExit, rather than closing normally.
This was reproduced after receiving an actual generated token from OpenAI gpt-4.1-mini, through native Chat, native Responses, and LiteLLM Chat. Fully consuming the streams succeeded. This affects explicit early-close cleanup, such as a consumer stopping generation; a resource leak has not been established.
Steps to reproduce
With OPENAI_API_KEY configured, run:
import asyncio
import dspy
async def main():
lm = dspy.LM("openai/gpt-4.1-mini", cache=False, temperature=0, max_tokens=400)
try:
with dspy.context(lm=lm):
stream = dspy.streamify(
dspy.Predict("question -> answer"),
stream_listeners=[
dspy.streaming.StreamListener(signature_field_name="answer")
],
)(question="Write 100 numbered short sentences about paper airplanes.")
async for item in stream:
if isinstance(item, dspy.streaming.StreamResponse):
print("Received real token:", repr(item.chunk))
break
await stream.aclose()
print("Closed cleanly")
finally:
if hasattr(lm, "close"):
lm.close()
asyncio.run(main())
Expected: aclose() returns normally and prints Closed cleanly.
Actual: process exits with status 1:
Received real token: '1'
BaseExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
... dspy/streaming/streamify.py, in async_streamer
yield output
GeneratorExit
async_streamer yields inside an AnyIO task group. Closing the generator injects GeneratorExit, which reaches the task group's exceptional exit and is wrapped in an exception group.
DSPy version / environment
- Merged
mainat 453823209. - Also reproduced with the complete
3.3.1source tree in a separate worktree using the same installed dependencies. This is not a regression from the recently merged LocalInterpreter cancellation fix. - Linux x86-64, Python 3.11.6; AnyIO 4.14.2, LiteLLM 1.96.0, OpenAI SDK 2.53.0.
- Real provider calls, caching disabled; no mocked LM responses.
AI assistance
Amp performed the reproductions and drafted this report during isaacbmiller's interactive release-robustness investigation. isaacbmiller explicitly authorized filing this issue after reviewing the finding. Relevant prompts: “Use a real LM, not just run pytests”, “what broke?”, and “I think its fine to create an issue for”.
Source: stanfordnlp/dspy