Source chat SSE has no keepalive and cancel is a no-op
Summary
Two related gaps in the source chat streaming path (POST /api/sources/{id}/chat SSE):
No keepalive while the model generates. The backend emits
user_message, then runssource_chat_graph.invokein a thread and only writes the next SSE event when the LLM finishes. For slow local models this means minutes with zero bytes on the wire, which proxies (including the Next.js rewrite in front of FastAPI) may cut as idle. The frontend then seesdonewithout anai_messageand the reply only shows up after a refetch.Cancel is a no-op.
useSourceChatkeeps anabortControllerRef, butsourceChatApi.sendMessagenever receives asignal, socancelStreaming()does nothing. The backend thread is not interruptible either, so leaving the page does not stop the generation.
Surfaced while investigating #1264 (the user reported the same "answer appears only after navigating away" symptom in Chat with Sources).
Where
api/routers/source_chat.py— SSE generator (~lines 333-405)open_notebook/graphs/source_chat.py—model.invoke(payload)frontend/src/lib/hooks/use-source-chat.ts—abortControllerRef,cancelStreamingfrontend/src/lib/api/source-chat.ts—sendMessage(nosignalparam)
Proposed change
- Backend: while the invoke thread runs, yield an SSE comment (
: ping) every ~15s so the connection never goes idle. - Frontend: pass the
AbortController.signaltofetch, wirecancelStreamingto it, abort on unmount. - Optional follow-up: if the client disconnects, stop the generation server-side (requires an async/streaming invoke instead of
to_thread).
Acceptance criteria
- A source chat against a model that takes 3+ minutes to answer keeps the SSE connection open through the Next proxy and renders the reply when it arrives.
- Clicking cancel closes the fetch and stops the typing indicator.
Source: lfnovo/open-notebook