bistream下2.0模型的zero_shot推理结果会拼接参考音频的结尾到推理结果上，偶尔也会出现开头跳过内容的问题

Describe the bug 如题，使用main分支bistream下2.0模型的zero_shot推理结果会少许拼接参考音频的结尾到推理结果上（如 the manger，In the morning...）；偶尔也会出现开头跳过内容的问题，如In the morning开头，实际为the morning...

To Reproduce Steps to reproduce the behavior:

sys.path.append('third_party/Matcha-TTS')

cosyvoice = CosyVoice2('pretrained_models/CosyVoice2-0.5B', load_jit=False, load_trt=False, fp16=True)
audio_path = './en_man_6s.wav'
prompt_speech_16k = load_wav(audio_path, 16000)

def text_generator(text_list):
    for text in text_list:
        yield text
text_list = ["In the morning, I often eat scrambled eggs with toast and fresh orange juice."," Sometimes, I enjoy a bowl of"]

for i, j in enumerate(cosyvoice.inference_zero_shot(text_generator(text_list), 'The shepherds went in haste to Bethlehem and found Mary and Joseph, and the infant lying in the manger.', prompt_speech_16k, stream=True)):
    torchaudio.save('gen_audio/stream_zero_shot_25token{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)

Desktop (please complete the following information): LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.9.2009 (Core) Release: 7.9.2009 Codename: Core GPU：A100 CUDA Version: 12.2

Additional context

en_man_6s.wav.zip 带上参考音频结尾的推理结果.zip