Piper VITS: missing pad token after BOS drops the first phoneme of every utterance

Piper VITS: missing pad token after BOS drops the first phoneme of every utterance

Summary

PiperPhonemesToIdsVits() in sherpa-onnx/csrc/piper-phonemize-lexicon.cc builds the phoneme-ID sequence as:

^  p1  _  p2  _  ...  pn  _  $

but piper-phonemize — which Piper VITS models were trained with — inserts a pad after BOS as well (see the explicit comment "Pad after bos (_)" in piper-phonemize/src/phoneme_ids.cpp):

^  _  p1  _  p2  _  ...  pn  _  $

With the pad missing, the model treats the first real phoneme as if it were the interleaved blank, so the first phoneme of every utterance is dropped or severely weakened. This affects all Piper VITS voices in all languages; it is easiest to hear on short utterances and word-initial glides/nasals/stops.

Reproduction

Voice: vits-piper-is_IS-salka-medium (Icelandic; also reproduced with bui/steinn/ugla, fp32 and int8 — so not a quantization issue).

import sherpa_onnx
# config: model/tokens/data_dir from vits-piper-is_IS-salka-medium
tts.generate("Já")           # spoken as "á"    (j dropped)
tts.generate("Nei")          # spoken as "ei"   (n dropped)
tts.generate("Takk fyrir")   # spoken as "hakk fyrir" (t reduced to h-like burst)

The same texts through the reference piper CLI with the original HuggingFace model are pronounced correctly.

Proof it is the missing pad

Driving the original is_IS-salka-medium.onnx directly with onnxruntime, same phonemes (jˈaʊː from espeak-ng), only the sequence construction changed:

seq_sherpa = [bos] + [i for p in ids for i in (p, pad)] + [eos]        # ^  j _ a _ ...
seq_piper  = [bos, pad] + [i for p in ids for i in (p, pad)] + [eos]   # ^ _ j _ a _ ...

• seq_sherpa → 0.49 s of audio, clearly "á" (j missing)
• seq_piper → 0.58 s of audio, correct "já"

Fix

One line in PiperPhonemesToIdsVits():

  ans.push_back(bos);
+ ans.push_back(pad);  // piper-phonemize pads after BOS as well
  for (auto p : phonemes) {

Patched and rebuilt locally (v1.13.4, iOS build): all previously broken word-initial phonemes are restored across the four Icelandic voices, with no regressions we could hear elsewhere.

Note: PiperPhonemesToIdsMatcha() in the same file also omits a pad after BOS — we have not tested whether Matcha models expect one, but it may deserve the same check.

Happy to open a PR if useful.