No user-facing signal while a response is generating on slower local TTS/LLM backends

Issue

When running local/self-hosted backends with significant latency (e.g. OmniVoice for TTS, or resource-constrained hardware), there is currently no signal to the user that a response is being generated. This makes the wait ambiguous, and users are likely to speak again before the first response has even started producing audio, cancelling it before it's ever heard.

Happy to work on this issue if there's enough interest for slower setup users!

Steps to Reproduce

1. Run speech-to-speech local with a slower TTS backend (e.g. --tts omnivoice) on limited hardware (e.g. Apple Silicon MPS).
2. Connect via the terminal client or browser demo and speak a turn.
3. Observe: after your speech ends, there is no indication a response is being generated — no event, log line visible to the client, or UI state, until audio actually starts arriving, which can take several seconds.

Evidence

OmniVoice TTS Backend

To note: 37 second gap between 2026-09-13 11:53:48,344 ASSISTANT text response and 2026-09-13 11:54:25,583 produced audio response with no acknowledgement that TTS is running successfully.

2026-09-13 11:53:42,707 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Speech started (confirmed, active=384ms, min=384ms, segment=384ms, turn=turn_2 rev=0) 2026-09-13 11:53:43,537 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Speech soft-ended (segment=1716ms, active=1120ms, turn=turn_2 rev=0) 2026-09-13 11:53:43,577 - [pipeline 0] speech_to_speech.STT.parakeet_tdt_handler - INFO - Parakeet progressive STT timing turn=turn_2 rev=0 audio=0.884s age=0.000s lock_scope=0.869s inference=0.869s chars=19 2026-09-13 11:53:43,578 - [pipeline 0] speech_to_speech.utils.mlx_lock - INFO - ParakeetSTT-Progressive: MLX lock released after holding 0.87s 2026-09-13 11:53:43,627 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Smart Turn: complete (p=0.970, 89.4ms); using 800ms speculative reopen grace 2026-09-13 11:53:43,660 - [pipeline 0] speech_to_speech.STT.parakeet_tdt_handler - INFO - Parakeet final STT start turn=turn_2 rev=0 audio=1.716s age=0.033s USER: I didn't2026-09-13 11:53:43,749 - [pipeline 0] speech_to_speech.STT.parakeet_tdt_handler - INFO - Parakeet final STT done turn=turn_2 rev=0 total=0.089s lock_scope=0.088s inference=0.088s chars=23 USER: I didn't hear anything. Language: en 2026-09-13 11:53:43,750 - [pipeline 0] speech_to_speech.baseHandler - INFO - ParakeetTDTSTTHandler: 0.090 s 2026-09-13 11:53:43,750 - [pipeline 0] speech_to_speech.STT.transcription_notifier - INFO - Transcription completed (language=en): chars=23 USER: I didn't hear anything. 2026-09-13 11:53:48,293 - [pipeline 0] speech_to_speech.utils.mlx_lock - INFO - MLX-LLM: MLX lock released after holding 4.53s 2026-09-13 11:53:48,344 - [pipeline 0] speech_to_speech.LLM.language_model - INFO - Tools: chars=2 ASSISTANT: Sorry about that! I'm here to help—what would you like to know or do? ASSISTANT: EVENT: response.content_part.added ASSISTANT: Sorry about that! I'm here to help—what would you like to know or do? 2026-09-13 11:54:25,583 - [pipeline 0] speech_to_speech.api.openai_realtime.service - INFO - Token usage (response): input=379, output=21 2026-09-13 11:54:25,585 - [pipeline 0] speech_to_speech.api.openai_realtime.handlers.response - INFO - Response done (status=completed) — this response: input_tokens=379, output_tokens=21, audio=1.72s | cumulative: input_tokens=727, output_tokens=38, audio=3.53s 2026-09-13 11:54:25,586 - [pipeline 0] speech_to_speech.api.openai_realtime.websocket_router - INFO - Pipeline 0: response complete, listening re-enabled ASSISTANT: EVENT: response.content_part.done ASSISTANT:

Qwen 3 TTS Backend

To note: Quicker backends result in a more seamless experience, though the architectural gap still exists in theory.

2026-09-13 12:25:18,089 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Speech started (confirmed, active=384ms, min=384ms, segment=384ms, turn=turn_2 rev=0) USER: What can you tell me about the2026-09-13 12:25:19,881 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Speech soft-ended (segment=2676ms, active=2080ms, turn=turn_2 rev=0) 2026-09-13 12:25:19,958 - [pipeline 0] speech_to_speech.VAD.vad_handler - INFO - Smart Turn: complete (p=0.987, 77.2ms); using 800ms speculative reopen grace 2026-09-13 12:25:19,959 - [pipeline 0] speech_to_speech.STT.parakeet_tdt_handler - INFO - Parakeet final STT start turn=turn_2 rev=0 audio=2.676s age=0.000s 2026-09-13 12:25:20,089 - [pipeline 0] speech_to_speech.STT.parakeet_tdt_handler - INFO - Parakeet final STT done turn=turn_2 rev=0 total=0.130s lock_scope=0.130s inference=0.130s chars=43 USER: What can you tell me about the Roman Forum? Language: en 2026-09-13 12:25:20,090 - [pipeline 0] speech_to_speech.baseHandler - INFO - ParakeetTDTSTTHandler: 0.132 s 2026-09-13 12:25:20,091 - [pipeline 0] speech_to_speech.STT.transcription_notifier - INFO - Transcription completed (language=en): chars=43 USER: What can you tell me about the Roman Forum? 2026-09-13 12:25:23,987 - [pipeline 0] speech_to_speech.utils.mlx_lock - INFO - MLX-LLM: MLX lock released after holding 3.89s 2026-09-13 12:25:23,993 - [pipeline 0] speech_to_speech.LLM.language_model - INFO - Tools: chars=2 ASSISTANT: The Roman Forum was the heart of ancient Rome—a bustling public space for politics, religion, and daily life. ASSISTANT: EVENT: response.content_part.added ASSISTANT: The Roman Forum was the heart of ancient Rome—a bustling public space for politics, religion, and daily life.2026-09-13 12:25:24,347 - [pipeline 0] speech_to_speech.TTS.qwen3_tts_handler - INFO - Qwen3-TTS TTFA: 0.35s (custom_voice_mlx) 2026-09-13 12:25:24,348 - [pipeline 0] speech_to_speech.TTS.qwen3_tts_handler - INFO - Last speech detected to first speech out: 4.389s (turn=turn_2 rev=0) 2026-09-13 12:25:32,034 - [pipeline 0] speech_to_speech.TTS.qwen3_tts_handler - INFO - Qwen3-TTS generated 7.94s audio in 8.04s (RTF: 0.99, custom_voice_mlx) 2026-09-13 12:25:32,035 - [pipeline 0] speech_to_speech.utils.mlx_lock - INFO - Qwen3TTS: MLX lock released after holding 8.04s 2026-09-13 12:25:32,056 - [pipeline 0] speech_to_speech.api.openai_realtime.service - INFO - Token usage (response): input=383, output=22 2026-09-13 12:25:32,058 - [pipeline 0] speech_to_speech.api.openai_realtime.handlers.response - INFO - Response done (status=completed) — this response: input_tokens=383, output_tokens=22, audio=2.68s | cumulative: input_tokens=731, output_tokens=39, audio=4.42s 2026-09-13 12:25:32,059 - [pipeline 0] speech_to_speech.api.openai_realtime.websocket_router - INFO - Pipeline 0: response complete, listening re-enabled

ASSISTANT: EVENT: response.content_part.done ASSISTANT:

Proposal

Add user-facing written confirmation in CLI that TTS process is running, most noticeable for slower pipeline setups.