#371·Qwen3-TTS

Finetuning loss misaligned with generation (double shift + leaked sub-talker hidden) — please merge #278

Author: thewh1teagleCreated Sep 16, 2026Updated Sep 16, 2026

Three alignment bugs in finetuning/sft_12hz.py + modeling_qwen3_tts.py (transformers 4.57.3, main). Measured on a real batch; all three are also covered by #337 and the open PR #278 — opening this to add numbers and ask for a merge.

  1. Talker trained to predict frame j+2. sft_12hz.py L101-103 slices inputs_embeds[:, :-1] / labels[:, 1:], and ForCausalLMLoss shifts once more. My fine-tuned checkpoint: CE 1.87 vs label j+2, 6.29 vs j+1. Stock model is the opposite (6.05 vs 11.03), so fine-tuning actively unlearns the generation contract. Probably #179 as well (skipping a frame every step = faster speech).
  2. Sub-talker gets a leaked hidden. L108 hidden_states[codec_mask[:, :-1]] is the hidden at position p, which already consumed frame p's codes. Generation uses past_hidden = hidden_states[:, -1:] (L1740), i.e. position p-1. With that alignment my trained sub-talker's CE is 10.08 — worse than uniform (ln 2048 = 7.62).
  3. Sub-talker labels shifted inside forward_finetune (L1242): the 15 codebook labels go through ForCausalLMLoss, so head i is trained on codebook i+2 and codebook 15 never. Stock model: CE 7.24 as-is vs 10.03 shifted.

Fix = #278 plus prev_mask[:, :-1] = codec_mask[:, 1:]; hidden_states[prev_mask]. Loss curves look healthy with the shipped code, so this is invisible until you generate. It cost me ~8 GPU-hours for a model that could not stop generating. Please merge.