The talker_hidden_states not obtained correctly
Author: howitryCreated Jul 6, 2026Updated Aug 18, 2026
Description
Error code location:
Reproduction
- Error code location: https://github.com/QwenLM/Qwen3-TTS/blob/main/finetuning/sft_12hz.py#L108
- Error Description:
The position where codec_mask is True corresponds to the actual codec. hidden_states[codec_mask[:, :-1]] retrieves the hidden_states of the actual codec. Note that the hidden state at time t is used to predict the codec at time t+1.
The above logic cannot be matched with the code: https://github.com/QwenLM/Qwen3-TTS/blob/main/qwen_tts/core/models/modeling_qwen3_tts.py#L1612
The hidden state at time t should not be concatenated with the codec embedding at time t (the hidden state at time t-1 is used to predict the codec at time t), but should be concatenated with the codec embedding at time t+1.
The correct value is
talker_hidden_states=hidden_states[codec_mask[:, 1:]], https://github.com/QwenLM/Qwen3-TTS/blob/main/finetuning/sft_12hz.py#L108
Logs
Environment Information
no Environment
Known Issue
- The issue hasn't been already addressed in Documentation, Issues, and Discussions.
Source: QwenLM/Qwen3-TTS