#1141·MiniCPM-V

[MiniCPM-o-4.5] tts_bound selects a history turn's <|tts_eos|> when current speech is truncated, yielding an empty TTS span

Author: jaminmeiCreated Sep 17, 2026Updated Sep 17, 2026

Hi! I hit an edge case in modeling_minicpmo.py of openbmb/MiniCPM-o-4_5 while testing multi-turn speech chat.

chat() takes the last <|tts_bos|> but the globally last <|tts_eos|> over prompt + generated tokens:

python
tts_bos_idx = tts_bos_indices[-1] if tts_bos_indices else -1
tts_eos_idx = tts_eos_indices[-1] if tts_eos_indices else None

If the history contains a finished speech turn (prompt has an old <|tts_eos|>) and the current reply stops before its own marker (e.g. a small max_new_tokens), the end index lands before the start index, full_sequences[0][start:end] is empty, and the talker gets an empty condition — no audio, even though the turn had speakable tokens. The bare except around _generate_speech_non_streaming hides the failure, so you just get text with no wav. The None fallback only covers "no <|tts_eos|> anywhere"; it can't help when an old marker sits before the last <|tts_bos|>.

The text-side extraction in the same file already does this right (text.split("<|tts_bos|>")[-1].split("<|tts_eos|>")[0]), so maybe just scope the token path the same way:

python
start = tts_bos_indices[-1] + 1
eos_in_segment = [i for i in tts_eos_indices if i >= start]
tts_eos_idx = eos_in_segment[0] if eos_in_segment else None
tts_bound = (start, tts_eos_idx)

Single-turn and completed-turn behavior stay unchanged.

(One more thing, probably a separate issue: truncated turns also hit an off-by-one between the token span and the hidden states. In _generate_speech_non_streaming (hidden_text_merge path), llm_tokens is sliced from full_sequences, but hidden_embeds comes from the per-step hidden stack, which only covers positions consumed by a forward pass — the last generated token of a truncated turn is never fed back in, so it has no hidden row, and llm_embeds + hidden_embeds would raises RuntimeError (swallowed by the bare except). Completed turns don't hit this because the tokens generated after the span, like <|tts_eos|>, consume the last span token as input.)