A cancelled SSE stream deletes the whole session's message queue, so the next stream 404s (503 behind a proxy)

Describe the bug

A cancelled SSE stream deletes the message queue belonging to the session, not to the stream. Any stream that then opens for that session_hash before a fresh queue/join is answered with session_not_found, and the server re-raises an HTTPException into a response that has already started.

pending_messages_per_session[session_hash] is:

• created in exactly one place — Queue.push (queueing.py), i.e. queue/join;
• deleted in exactly one place — sse_stream's except BaseException in routes.py, when the exception is asyncio.CancelledError:

if isinstance(e, asyncio.CancelledError):
    del blocks._queue.pending_messages_per_session[session_hash]
    await blocks._queue.clean_events(session_hash=session_hash)

• and required by sse_stream on entry:

if session_hash not in blocks._queue.pending_messages_per_session:
    raise HTTPException(status_code=status.HTTP_404_NOT_FOUND)

A client disconnect cancels the streaming task, so anything that drops one stream — a backgrounded tab, a navigation, a proxy closing an idle connection — removes state shared by every stream on that session.

Because the 404 is raised after the StreamingResponse has begun, the status cannot be set:

  File "gradio/routes.py", line 1516, in sse_stream
    raise HTTPException(status_code=status.HTTP_404_NOT_FOUND)
fastapi.exceptions.HTTPException: 404: Not Found
The above exception was the direct cause of the following exception:
  File "starlette/_exception_handler.py", line 56, in wrapped_app
    raise RuntimeError("Caught handled exception, but response already started.") from exc

Locally the client sees 200 plus an in-band unexpected_error frame. Deployed behind a proxy (Hugging Face Spaces, in our case) the broken response reaches the browser as 503 on GET /gradio_api/queue/data, with POST /gradio_api/queue/join still returning 200 — the user sends a message and nothing comes back, not even their own message echoed.

The failure is invisible to every non-browser probe. curl, gradio_client, in-page fetch and our own deploy smoke check each join and read once on a fresh session and cancel nothing, so all of them pass against an app that answers no one. That cost us about two days of misdiagnosis, including two retracted root causes.

Have you searched existing issues?

• I have searched and found no existing issues

Reproduction

No browser needed. Tested on a clean venv with gradio==6.18.0.

import time
import gradio as gr
import httpx

def slow(msg):
    time.sleep(0.2)
    return f"echo: {msg}"

with gr.Blocks() as demo:
    inp, out = gr.Textbox(), gr.Textbox()
    gr.Button("go").click(slow, inp, out)

demo.launch(prevent_thread_lock=True, quiet=True, server_port=7898)
time.sleep(2)
BASE = "http://127.0.0.1:7898/gradio_api"
S = "repro-session-0002"
q = demo._queue.pending_messages_per_session

httpx.post(f"{BASE}/queue/join", json={"data": ["hi"], "fn_index": 0,
                                       "session_hash": S, "trigger_id": 2}, timeout=10)
print("1. join                     | registered:", S in q)

# A stream is opened and abandoned, as a backgrounded tab or a proxy timeout does.
try:
    with httpx.stream("GET", f"{BASE}/queue/data", params={"session_hash": S}, timeout=5) as r:
        for _ in r.iter_lines():
            break
except Exception:
    pass
time.sleep(2)
print("2. stream abandoned         | registered:", S in q)

# The frontend reopens its long-lived stream. It has nothing to send, so no new join.
with httpx.stream("GET", f"{BASE}/queue/data", params={"session_hash": S}, timeout=10) as r:
    first = ""
    if r.status_code == 200:
        for line in r.iter_lines():
            first = line
            break
    print("3. stream reopened, NO join ->", r.status_code, "|", first[:130])

Screenshot

No response

Logs

1. join                     | registered: True
2. stream abandoned         | registered: False        <- the SESSION's queue is gone
3. stream reopened, NO join -> 200 | data: {"msg":"unexpected_error","event_id":null,"message":"404: Not Found","session_not_found":true,"success":false}

Server-side, step 3 also logs RuntimeError: Caught handled exception, but response already started.

A subsequent queue/join does re-create the queue, so this is not permanent for a session that always joins before streaming. It bites whenever a stream opens for a session between the cancellation and the next join — which is the normal reconnect path for a long-lived frontend stream.

System Info

gradio 6.18.0 (reproduced); the same code is present unchanged in 6.28.0 — both the
delete and the 404 are byte-identical, so no released version fixes this.
Deployed on Hugging Face Spaces, single replica, Python 3.13, gradio[oauth,mcp].

Severity

Blocking usage of gradio

Suggested fix

The per-stream cancellation path should not delete per-session state. Options, roughly in order of how local the change is:

1. Don't delete on CancelledError — let the existing LRUCache bound the map, as it already does for every other session. This is a one-line change and is what we are running as a local workaround: subclass LRUCache with a no-op __delitem__ and swap it in. OrderedDict.popitem doesn't route through __delitem__, so eviction still bounds the cache.
2. Refcount the streams per session and delete only when the last one goes.
3. Re-create the queue in sse_stream instead of raising 404 — this makes a reconnect self-healing, but changes session_not_found semantics for genuinely unknown sessions.

Happy to open a PR for (1) if that's the direction you'd prefer.