Multilingual tokenizer silently deletes Arabic word-initial hamza alefs (أ إ آ ٱ) before synthesis

Environment

• chatterbox-tts 0.1.7 (ChatterboxMultilingualTTS.from_pretrained(device="cuda")), Linux, GPU L4.

Summary

The multilingual tokenizer silently strips word-initial hamza alefs (and madda) from Arabic text before the model sees it. The model then faithfully speaks the mangled text.

Repro (pure text — no GPU/audio needed)

from chatterbox.mtl_tts import ChatterboxMultilingualTTS
m = ChatterboxMultilingualTTS.from_pretrained(device="cpu")  # tokenizer only
enc = m.tokenizer.encode("أهلا بكم اليوم أحب أن أرحب بكم وأتمنى لكم التوفيق")
print(m.tokenizer.decode(enc.ids if hasattr(enc, "ids") else enc))
# -> هلا بكم اليوم حب ن رحب بكم وتمنى لكم التوفيق

أهلا→هلا, أحب أن أرحب→حب ن رحب (the word أن collapses to ن), وأتمنى→وتمنى. Every hamza-bearing alef is deleted rather than mapped.

Audio corroboration

Generated audio of that sentence, transcribed independently by two ASRs (Whisper large-v3-turbo and AssemblyAI), both heard the mangled words (e.g. "هلا… حب… رحب… وتمنى") — confirming the model speaks the stripped text, not just a lossy decode.

Impact

Any Arabic word beginning with أ / إ / آ / ٱ loses its initial letter: قرآن, إسلام, أحب etc. Since Arabic words cannot begin with a bare vowel, deleting the hamza effectively removes a pronounced glottal-stop onset from every such word.

Suggested fix

Map these codepoints to plain alef (ا) in the tokenizer's Arabic normalization — the standard CAMeL normalize_alef operation (CAMeL Tools, LREC 2020) — rather than dropping them. Happy to send a PR if maintainers point at the right normalization entry point.

Workaround we use

Pre-normalize أ إ ٱ (word-initial) and آ → ا before calling generate().