Streaming `/v1/audio/speech` emits one complete container file per chunk, inserting audible silence at every seam (all container formats)

Summary

When /v1/audio/speech is called with stream: true, each streamed chunk is encoded as a complete, standalone container file and those files are concatenated on the wire. This affects every container format — mp3, ogg, opus, webm and wav alike. Only raw/pcm escapes it, because it has no container.

There are two distinct symptoms. Structurally, the response carries one container header per chunk where a well-formed stream has exactly one — true of every format including uncompressed WAV. Audibly, in the lossy formats each segment's encoder padding and the next segment's encoder delay stack up into a short gap at every seam. The inserted material is silence, not a click. Both are present in the bytes the server sends, before any client is involved — confirmed by playing the saved response body in VLC.

This is the same underlying behaviour #484 described when it added raw/pcm ("streaming in WAV format causes players to interpret wav headers as audio, resulting in audible clicks between streamed segments"). That PR sidestepped containers rather than making them streamable, so both the WAV case it describes and the compressed formats still carry the problem in 0.5.0.

raw/pcm is not a workaround for every deployment: sending uncompressed PCM is not viable over a bandwidth-constrained link, and browser MediaSource with addSourceBuffer('audio/mpeg') will not accept it.

Version

• mlx-audio 0.5.0 (installed from PyPI)
• Also read main at ebf44dfc (2026-08-20); the commits past the v0.5.0 tag (65fb3ccb) do not touch the server streaming or audio_io encode paths, so this should still apply.
• macOS, Apple Silicon, Python 3.12, ffmpeg via Homebrew

Reproduction

Reproduced on two different models, so it is not model-specific:

• mlx-community/pocket-tts (voice: "eponine", no ref_audio)
• mlx-community/chatterbox-turbo-fp16 (voice cloning via ref_audio)

Request: stream: true, response_format: "mp3", an explicit streaming_interval, and a ~35s input text (long enough to produce several seams). Requests were made directly against the server with curl, saving the raw response body to a file — no browser and no client-side audio pipeline in the path.

Container headers — one per chunk instead of one per stream.

Counting ID3 occurrences in the saved mp3 body returns one per emitted chunk. A correctly formed stream has exactly one. The same holds for OggS in ogg/opus output, and for RIFF in wav.

WAV shows half of this pattern. A streamed wav response of ~40s at streaming_interval: 4 contains 11 RIFF headers — one per chunk. Summing the segments gives 10 x 4.000s + 0.720s = 40.72s, with no per-seam inflation at all: WAV is uncompressed, so there is no encoder delay or end padding to accumulate. The per-chunk container is therefore present regardless of codec, while the duration inflation below is specific to the lossy formats.

Duration — streamed output runs longer than the same text rendered in one pass.

Summing the per-segment durations of the streamed response and comparing against a non-streamed (stream: false) render of the identical text:

Model	streaming_interval	Streamed	Non-streamed	Delta	Seams	Per seam
mlx-community/pocket-tts	4	34.66s	31.84s	+2.82s	8	~0.35s
mlx-community/chatterbox-turbo-fp16	4	42.05s	36.44s	+5.61s	10	~0.56s
mlx-community/pocket-tts	2	43.10s	40.32s	+2.78s	20	~0.14s

The per-seam figure is not a constant — it varies with streaming_interval and with the model. What is consistent is that the total added duration scales with the number of seams rather than with the length of the audio.

(Measuring this needs the per-segment durations summed rather than a plain ffprobe call, but how ffprobe misreads the file depends on the container. On a streamed wav it reports only the first segment — 4.00s for the 40.72s response above. On a streamed mp3 it has no container-level duration to trust and estimates across the whole file, landing near the true total, so the discrepancy is invisible there. Neither reading is a reliable total.)

Listening to the saved file confirms the audible result: a gap at each multiple of streaming_interval, absent from a non-streamed render of the same text. Setting streaming_interval low (2–4) makes the seams frequent and easy to locate. Note that sample-level discontinuity detection does not flag these — the artifact is inserted silence rather than a waveform discontinuity, so an automated click detector will report the file as clean.

Sample files for maintainers to verify the problem: mlx-audio-streaming-samples.zip

Prior art

I searched open and closed issues and PRs for streaming clicks/gaps/pops, response_format, the audio_io encode path, container headers, and opus/ogg/webm, and did not find this raised before. #484 documents the WAV-header symptom and resolves it with raw/pcm; #297 added the response_format option. Neither covers continuous compressed streaming. Happy to be pointed at anything I missed.

Why this looks like a chunk-production issue rather than a codec issue (Claude's hypothesis)

The encode path is batch-oriented by construction, and the streaming path calls it once per chunk:

• server.py:595 — _emit_audio() creates a fresh io.BytesIO() and calls audio_write(...) on it for every chunk, then emits the resulting bytes. Nothing is carried across calls, so every chunk is encoded with no knowledge that it is part of a longer stream.
• audio_io.py:599 — write() dispatches every lossy format to _encode_ffmpeg.
• audio_io.py:405 — _encode_ffmpeg() spawns ffmpeg, pipes the entire buffer via subprocess.run(input=pcm_bytes), waits for exit, and returns the bytes. One process per call, started and finalized around a single chunk — so each invocation necessarily writes a header and flushes a complete, finalized container.

So the artifact is produced by how chunks are handed to the encoder, not by the codec: the audio samples themselves are continuous across the boundary (the non-streamed render of the same text is clean), and the format determines only how the breakage manifests, not whether it happens. Every format that has a container is written one chunk at a time — WAV through miniaudio, the rest through the per-chunk ffmpeg spawn above — which is why mp3, ogg, opus, webm and wav all show one header per chunk, and raw/pcm alone does not.