STT handlers retain the detected language across sessions; only Parakeet resets it
Summary
In --language auto, four of the five bundled STT handlers keep self.last_language when a session ends. Pipeline units are reused across clients, so the next client inherits the previous client's detected language.
ParakeetTDTSTTHandler — the default backend — already resets it, which is what makes this look like an oversight rather than a policy:
# STT/parakeet_tdt_handler.py
def on_session_end(self) -> None:
super().on_session_end()
self.last_language = self.start_language if self.start_language else "en"BaseSTTHandler.on_session_end only clears revision bookkeeping:
def on_session_end(self) -> None:
if hasattr(self, "_completed_final_revision_keys"):
self._completed_final_revision_keys.clear()| handler | resets last_language |
|---|---|
parakeet_tdt_handler (default) |
yes |
whisper_stt_handler |
no |
mlx_audio_whisper_handler |
no |
lightning_whisper_mlx_handler |
no |
qwen3_asr_handler |
no |
on_session_end is the documented place for this — websocket_router._clean_unit describes SESSION_END as "the soft reset signal for stateful handlers", and flushes the queues specifically so work "cannot be picked up by handlers and leak into the next session that claims this unit."
Impact, per backend
I want to be precise rather than overstate this, because it varies:
Qwen3-ASR — user-visible. Progressive requests are forced into last_language:
# STT/qwen3_asr_handler.py:148
request_language = self.forced_language
if request_language is None and progressive:
request_language = self.last_languageSo with --enable_live_transcription, a new client's live partials are transcribed in the previous client's language until their first final turn re-detects. The handler's own docstring says progressive windows reuse "the language of the last final turn" — across a session boundary that's the wrong conversation's last final turn.
Whisper family — fallback only. last_language is consulted when detection yields nothing, so the leak surfaces on the new session's first turn only if detection is unavailable or returns an unsupported code. Milder. Note process() copies gen_kwargs (dict(self.gen_kwargs)), so the forced-language kwarg is not mutated at runtime — I checked, and only last_language persists.
Reproduction
No model needed; this is handler state, and it is what SESSION_END is supposed to clear:
from speech_to_speech.STT.whisper_stt_handler import WhisperSTTHandler
from speech_to_speech.STT.parakeet_tdt_handler import ParakeetTDTSTTHandler
w = object.__new__(WhisperSTTHandler)
w.start_language = "auto"; w.last_language = "de"; w.gen_kwargs = {}
w.on_session_end()
print(w.last_language) # 'de' <- survives into the next session
p = object.__new__(ParakeetTDTSTTHandler)
p.start_language = "auto"; p.last_language = "de"; p.enable_live_transcription = False
p.on_session_end()
print(p.last_language) # 'auto' <- resetEnd to end: run serve with --stt qwen3-asr --language auto --enable_live_transcription, connect a German speaker, disconnect, then connect a French speaker to the same pipeline unit. The second client's live partials come back German-forced until their first final turn.
Suggested fix
Reset the per-session language state where the convention already lives. Either:
- Give each handler an
on_session_endmatching Parakeet's, or - Do it once in
BaseSTTHandler.on_session_end—start_language/last_languageis a shared convention across all five handlers, so one reset covers every current backend and any future one, which is the part I'd value most given this was missed four times.
Happy to open a PR; I'd default to (2) with the contract documented on the base class, unless you'd rather keep it explicit per handler.
One adjacent thing I noticed while checking, not part of this: Parakeet resets to self.start_language, which is the literal string "auto" in auto mode rather than a language code. That's a separate question from whether the reset happens.
Environment
speech-to-speechv1.0.0,mainat16d7f98- Affects the realtime server, where
PipelineUnits are claimed and released across clients
Source: huggingface/speech-to-speech