Finetuning loss misaligned with generation (double shift + leaked sub-talker hidden) — please merge #278
Author: thewh1teagleCreated Sep 16, 2026Updated Sep 16, 2026
Three alignment bugs in finetuning/sft_12hz.py + modeling_qwen3_tts.py (transformers 4.57.3, main). Measured on a real batch; all three are also covered by #337 and the open PR #278 — opening this to add numbers and ask for a merge.
- Talker trained to predict frame j+2.
sft_12hz.pyL101-103 slicesinputs_embeds[:, :-1]/labels[:, 1:], andForCausalLMLossshifts once more. My fine-tuned checkpoint: CE 1.87 vs label j+2, 6.29 vs j+1. Stock model is the opposite (6.05 vs 11.03), so fine-tuning actively unlearns the generation contract. Probably #179 as well (skipping a frame every step = faster speech). - Sub-talker gets a leaked hidden. L108
hidden_states[codec_mask[:, :-1]]is the hidden at position p, which already consumed frame p's codes. Generation usespast_hidden = hidden_states[:, -1:](L1740), i.e. position p-1. With that alignment my trained sub-talker's CE is 10.08 — worse than uniform (ln 2048 = 7.62). - Sub-talker labels shifted inside
forward_finetune(L1242): the 15 codebook labels go throughForCausalLMLoss, so head i is trained on codebook i+2 and codebook 15 never. Stock model: CE 7.24 as-is vs 10.03 shifted.
Fix = #278 plus prev_mask[:, :-1] = codec_mask[:, 1:]; hidden_states[prev_mask]. Loss curves look healthy with the shipped code, so this is invisible until you generate. It cost me ~8 GPU-hours for a model that could not stop generating. Please merge.
Source: QwenLM/Qwen3-TTS