Question: recovering when an LLM stream starts but never reaches a sentence end (slow-streaming window)
Question: documented way to recover when an LLM stream starts but dribbles (no sentence end for many seconds)
Setup: pipecat 1.10.0 (now 1.11.0), OpenAIResponsesLLMService (gpt-5.6-terra, streaming), Deepgram Flux STT, Telnyx transport, sentence-based TTS aggregation, retry_on_timeout=True / retry_timeout_secs=5 on the LLM.
What happened (real call, 16 Sep 2026 08:23-08:25Z): first tokens arrived normally (metrics.ttfb avg 1.37 s, max 2.93 s over 9 requests), but from one point on the streams never reached a sentence end: partial text such as "Sorry, did you mean which swim" sat for 3.7 / 5.5 / 3.5 / 8.9 s with no sentence boundary, so nothing went to TTS and the bot was silent. The caller said "Hello?" three times; each "Hello?" was an interruption that cancelled the in-flight reply (ABORTED_TURN before any sentence played), the next reply started and was cancelled the same way, and the caller hung up after ~22 s of silence. Two other calls in the same 40-minute window on different machines saw LLM TTFB maxima of 20-22 s, so this looks like a provider slow-streaming window rather than anything pipeline-side. The PipelineWorker heartbeat warning ("heartbeat frame not received for more than 10.0 seconds") fired during it.
What I found in the docs/source: retry_on_timeout covers a request that produces no output within retry_timeout_secs (a silent start). I could not find a documented lever for a stream that has started but is too slow to reach a sentence end, nor a documented LLM fallback service, nor a documented way to speak a holding line when the first sentence is late.
Questions:
- Is there a documented setting that bounds time-to-first-sentence (or time between tokens) and retries/cancels the stream, the way
retry_on_timeoutbounds time-to-first-token? - Is there a documented pattern for a fallback LLM (or a spoken holding line) when a stream dribbles, or is this intentionally left to application code?
- Is the behaviour above (each caller interruption cancelling a slow reply, so a slow window compounds into total silence) the intended interaction, and is there a recommended configuration for it?
No stock repro attached because the trigger was a provider-side slow window I cannot reproduce on demand; happy to run one if you can suggest a way to simulate a slow stream.
Source: pipecat-ai/pipecat