Source chat SSE has no keepalive and cancel is a no-op

Author: lfnovoCreated Sep 5, 2026Updated Sep 7, 2026
Labelsbugreadyarea: chat

Summary

Two related gaps in the source chat streaming path (POST /api/sources/{id}/chat SSE):

  1. No keepalive while the model generates. The backend emits user_message, then runs source_chat_graph.invoke in a thread and only writes the next SSE event when the LLM finishes. For slow local models this means minutes with zero bytes on the wire, which proxies (including the Next.js rewrite in front of FastAPI) may cut as idle. The frontend then sees done without an ai_message and the reply only shows up after a refetch.

  2. Cancel is a no-op. useSourceChat keeps an abortControllerRef, but sourceChatApi.sendMessage never receives a signal, so cancelStreaming() does nothing. The backend thread is not interruptible either, so leaving the page does not stop the generation.

Surfaced while investigating #1264 (the user reported the same "answer appears only after navigating away" symptom in Chat with Sources).

Where

  • api/routers/source_chat.py — SSE generator (~lines 333-405)
  • open_notebook/graphs/source_chat.pymodel.invoke(payload)
  • frontend/src/lib/hooks/use-source-chat.tsabortControllerRef, cancelStreaming
  • frontend/src/lib/api/source-chat.tssendMessage (no signal param)

Proposed change

  • Backend: while the invoke thread runs, yield an SSE comment (: ping) every ~15s so the connection never goes idle.
  • Frontend: pass the AbortController.signal to fetch, wire cancelStreaming to it, abort on unmount.
  • Optional follow-up: if the client disconnects, stop the generation server-side (requires an async/streaming invoke instead of to_thread).

Acceptance criteria

  • A source chat against a model that takes 3+ minutes to answer keeps the SSE connection open through the Next proxy and renders the reply when it arrives.
  • Clicking cancel closes the fetch and stops the typing indicator.