parakeet-tdt-0.6b-v3 转录中 len(words) 和 len(word_confidence) 不匹配
I am running a transcription pipeline using the parakeet-tdt-0.6b-v3 model. In order to avoid running out of memory, I have implemented an algorithm to chunk longer files into shorter chunks, run the transcription on the chunks, and then reassemble the transcription of the chunks to obtain the transcription of the file.
The chunking is done by running the "pyannote/voice-activity-detection" VAD model on the audio, then aggregating the segments with active voice until a given maximum length (e.g. 10 min) is exceeded. The audio is then cut in the mid point between the end of one active segment and the start of the next active segment to prevent the audio from being cut in the middle of a word.
I have tried using a naive chunking (without checking for voice activity) before, which led to the same error occurring.
I have tested this on publicly available audio recordings of the German parliament.
This works well for most of the files I tested. But for some, I encounter the following RuntimeError in line 463 of .venv/lib/python3.12/site-packages/nemo/collections/asr/parts/utils/asr_confidence_utils.py:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[7], [line 60](vscode-notebook-cell:?execution_count=7&line=60)
58 # 4) ASR
59 with torch.inference_mode(), torch.amp.autocast("cuda", dtype=torch.float16):
---> [60](vscode-notebook-cell:?execution_count=7&line=60) out = asr_model.transcribe(
61 chunk_arrays,
62 batch_size=batch_size,
63 num_workers=4,
64 return_hypotheses=True
65 )
File ~/repos/nvidia-parakeet-testing/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py:120, in context_decorator.<locals>.decorate_context(*args, **kwargs)
117 @functools.wraps(func)
118 def decorate_context(*args, **kwargs):
119 with ctx_factory():
--> [120](https://vscode-remote+ssh-002dremote-002bpdeki01.vscode-resource.vscode-cdn.net/home/sfelix/repos/nvidia-parakeet-testing/~/repos/nvidia-parakeet-testing/.venv/lib/python3.12/site-packages/torch/utils/_contextlib.py:120) return func(*args, **kwargs)
File ~/repos/nvidia-parakeet-testing/.venv/lib/python3.12/site-packages/nemo/collections/asr/models/rnnt_models.py:308, in EncDecRNNTModel.transcribe(self, audio, batch_size, return_hypotheses, partial_hypothesis, num_workers, channel_selector, augmentor, verbose, timestamps, override_config)
304 self.cfg.decoding.preserve_alignments = False
306 self.change_decoding_strategy(self.cfg.decoding, verbose=False)
--> [308](https://vscode-remote+ssh-002dremote-002bpdeki01.vscode-resource.vscode-cdn.net/home/sfelix/repos/nvidia-parakeet-testing/~/repos/nvidia-parakeet-testing/.venv/lib/python3.12/site-packages/nemo/collections/asr/models/rnnt_models.py:308) return super().transcribe(
309 audio=audio,
310
…内容来源: NVIDIA-NeMo/Speech