Listener backchannels abort active response mid-playback

Author: xprilionCreated Sep 16, 2026Updated Sep 16, 2026

While testing the browser realtime demo, I noticed that uttering natural listener acknowledgements/backchannels (such as "mm-hmm", "uh-huh", "alright", or "go ahead") while the assistant is speaking immediately aborts the in-progress response, clears the audio playback, and triggers a brand-new reply answering the acknowledgement.

I might be missing an existing setting or flag for this, but I wanted to check if this is intended behavior or if there is interest in allowing short passive backchannels to pause and resume playback instead of triggering a destructive cancel.


Steps to Reproduce

  1. Start the server: speech-to-speech serve.
  2. Connect via the browser demo at http://localhost:8765 (or connect via talk).
  3. Ask a question that produces a long response (say "Explain how the solar system formed").
  4. While the assistant is actively speaking sentence 1, say "mm-hmm" or "right".

Observed Behavior

  1. The assistant abruptly cuts off mid-sentence.
  2. The queued audio for the rest of the explanation is cleared.
  3. The model receives "mm-hmm" as a new user prompt and begins answering it from scratch.

Expected Behavior

Brief listener feedback/backchannels should ideally not abort the active assistant turn. Instead, playback could briefly pause, and then resume the existing response once the short vocalization ends (saving full cancellations for substantive interruptions or queries).


Environment

  • Version: speech-to-speech v1.0.0 (main at 16d7f98)
  • Platform: macOS (Apple Silicon) / Linux

Screenshot

Image

Source: huggingface/speech-to-speech