whisperX

核心特点

• •⚡️ Batched inference for 70x realtime transcription using whisper large-v2
• •🪶 faster-whisper backend, requires <8GB gpu memory for large-v2 with beam_size=5
• •🎯 Accurate word-level timestamps using wav2vec2 alignment
• •👯‍♂️ Multispeaker ASR using speaker diarization from pyannote-audio (speaker ID labels)
• •🗣️ VAD preprocessing, reduces hallucination & batching with no WER degradation
• •1st place at Ego4d transcription challenge 🏆
• •_WhisperX_ accepted at INTERSPEECH 2023
• •v3 transcript segment-per-sentence: using nltk sent_tokenize for better subtitlting & better diarization
• •v3 released, 70x speed-up open-sourced. Using batched whisper with faster-whisper backend!
• •v2 released, code cleanup, imports whisper library VAD filtering is now turned on by default, as in the paper.