A "phantom language" can ruin the whole transcription

I am using WhisperX v2.0.1 with the option "detect language" (by omitting the "--language" option from the command). I use the "detect language" option for a video in which two languages are spoken, English and German. I want the subtitles to reflect the spoken language.

It turns out that when the speech in a video transitions from one language to another, some of the audio near such a transition is not subtitled. I think this is because only one language can be detected for a chunk of audio. And if a chunk of audio contains two languages, only one of them (the detected language) is usually subtitled. Because the missing subtitles can be added later, this is not a big problem.

However, there is a real big problem: WhisperX sometimes incorrectly detects that an audio chunk is in a particular language when that language is not present in that audio chunk. In addition, if that language is one for which WhisperX does not have a default alignment model, WhisperX stops shortly before the end of a transcription (which may have taken many hours) with the error No default align-model for language — and then produces no subtitles at all. So a detected "phantom language" can ruin the whole transcription.

"Phantom languages" are sometimes detected when the speech in a video changes from one language to another, or more randomly, for example, when there is just some background noise.

This is an example of the messages that appear when a transcription is aborted just before completion, because of a missing default alignment model:

New language found (jw)! Previous was (en), loading new alignment model for new language...
There is no default alignment model set for this language (jw).                Please find a wav2vec2.0 model finetuned
on this language in https://huggingface.co/models, then pass the model name in --align_model [MODEL_NAME]
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python310\Scripts\whisperx.exe\__main__.py", line 7, in <module>
  File "C:\Program Files\Python310\lib\site-packages\whisperx\transcribe.py", line 187, in cli
    align_model, align_metadata = load_align_model(result["language"], device)
  File "C:\Program Files\Python310\lib\site-packages\whisperx\alignment.py", line 53, in load_align_model
    raise ValueError(f"No default align-model for language: {language_code}")
ValueError: No default align-model for language: jw

So, while 99% of the transcription seems to go very well, and while there are default alignment models for the two languages in the video — the transcription stops near the end because of a detected "phantom language" without a default alignment model.

My suggestion is, that when a language without a default align-model is detected, instead of stopping near the end of a transcription with an error message, [1] to give a warning, [2] to ignore all audio chunks with a language without a default align-model (or to produce subtitles for them that are not aligned), and [3] to produce subtitles for all the other audio chunks that do have a default align-model.