Browser realtime demo can underrun TTS playback without a startup jitter buffer

Summary

The browser realtime demo can produce unnatural / glitchy speech when the TTS backend streams audio close to or slower than realtime.

The issue appears to be in the browser playback path rather than in TTS generation itself.

In my case, adding a small startup jitter buffer in demo/s2s-realtime-client.js fixed the issue.

Setup

• Hugging Face speech-to-speech
• Browser demo using WebSocket transport
• OpenAI-compatible local TTS backend
• VoxCPM2 as TTS
• TTS output: PCM s16le
• Pipeline output sample rate: 24 kHz
• Windows + Docker Desktop
• NVIDIA consumer GPU

Symptoms

Without buffering, speech sounds mostly correct, but some syllables / words become unnatural or glitchy during realtime playback.

The problem is not present in the generated audio itself.

Tests

I tested the same pipeline at several points:

1. VoxCPM native generate_streaming() output saved directly to WAV

   • sounds natural

2. Raw PCM returned by the local OpenAI-compatible TTS HTTP endpoint

   • sounds natural

3. PCM after the speech-to-speech backend resampling/output path

   • sounds natural when saved and played offline

4. Browser WebSocket realtime playback

   • some words / syllables sound unnatural

5. Browser playback changed to buffer the whole response before playback

   • sounds natural

6. Browser playback changed to use a startup buffer

   • 800 ms: much better, but occasional glitches remain
   • 1200 ms: playback becomes natural

This strongly suggests the browser playback queue is occasionally underrunning.

Current browser behavior

The browser client currently forwards incoming PCM chunks to the AudioWorklet immediately from _onAudio().

Conceptually:

this._playbackNode.port.postMessage({
  kind: "audio",
  samples
}, [samples.buffer]);

There does not appear to be enough startup buffering to absorb TTS generation jitter.

TTS realtime performance

Typical warm TTS performance in my setup is approximately:

• first chunk latency: ~0.28-0.35 s
• average RTF: ~1.1-1.2

So generation is close to realtime, but not always fast enough to continuously feed an immediately-started playback queue.

Even when average RTF is close to 1, individual chunk latency can vary enough to cause playback underruns.

Workaround

I added a 1200 ms startup buffer to demo/s2s-realtime-client.js.

The client first accumulates around 1.2 seconds of assistant PCM audio, then starts playback. After that, incoming chunks are forwarded normally.

This completely fixed the audible glitches in my setup.

Suggestion

Would it make sense for the browser demo to support a configurable playback jitter buffer, similar in purpose to --playback-buffer-ms used by the packaged local playback client?

Possible options:

• configurable startup playback buffer
• rebuffering when the AudioWorklet queue approaches underrun
• adaptive low-watermark jitter buffer

The browser demo currently seems to assume that the TTS backend can continuously provide audio at realtime speed or faster.

That assumption works well with fast TTS backends, but can cause audible artifacts with local TTS models running close to realtime on consumer GPUs.