Why WhisperX's sentence segmentation not working for Chinese?
Author: WmryCreated Dec 10, 2025Updated Jul 18, 2026
After using WhisperX to transcribe Chinese audio, I employed the Whisper.align model to segment the transcription results, but found it ineffective. I then examined the whisperx.align method and discovered that it uses NLTK for segmentation (lines 195-200 in whisperx/alignment.py). However, NLTK does not work effectively for Chinese. Does this mean WhisperX does not support Chinese text segmentation?
#(lines 195-200 in whisperx/alignment.py)
try:
sentence_splitter = nltk_load('tokenizers/punkt/english.pickle')
except LookupError:
nltk.download('punkt_tab', quiet=True)
sentence_splitter = nltk_load('tokenizers/punkt/english.pickle')
sentence_spans = list(sentence_splitter.span_tokenize(text))
}Source: m-bain/whisperX