Whisper invents dialogue on videos with no speech, and it is presented as a normal transcript

Summary

On a video with no speech, the Whisper fallback returns confident hallucinated text, and /watch presents it as a normal transcript. The model downstream then answers from dialogue that was never spoken.

The signal needed to catch this is already being fetched and then discarded — see "Why this is cheap to fix" below.

Reproduction

python3 scripts/watch.py "https://www.youtube.com/watch?v=cXE_ZOOtX4A"

CGI 3D Animated Short: "Distracted" (TheCGBros, 1:59). No subtitles, so the Whisper path runs. The film has no dialogue at all — its own credits list one person for "MUSIC COMPOSER/SFX".

Actual result

- **Transcript:** 16 segments (via whisper (groq))

## Transcript

_Source: whisper (groq)._

[00:00] Pancasili
[00:02] Cajadum
[00:05] Guava
[00:07] Ica
[00:10] Sirap
...
[00:30] Terima kasih telah menonton!
[01:00] Terima kasih telah menonton!
[01:30] Terima kasih telah menonton!

All of it is invented. Terima kasih telah menonton! is Indonesian for "thank you for watching" — one of Whisper's best-known hallucinations on silence and music. Note the timestamps: exactly 00:30, 01:00, 01:30, which is the giveaway.

Expected result

No transcript, or a transcript flagged as unreliable. The report currently labels this _Source: whisper (groq)._ with no qualifier, which reads as a genuine transcript.

Why it matters

Silence is not a rare edge case here — it is every music video, every no-dialogue short, every screen recording with no narration, and every clip whose audio is background music. In all of those the plugin currently invents dialogue rather than reporting that there is none.

It is worse than returning nothing. A missing transcript makes the model fall back on the frames, which is correct behaviour. A fabricated one makes it answer confidently from text that does not exist. I only caught it here because I had 20 frames to check against; a --detail transcript run returns the hallucination alone, with nothing to contradict it.

Why this is cheap to fix

_post_whisper() already sends "response_format": "verbose_json", so the response carries per-segment no_speech_prob and avg_logprob. But _segments_from_response() keeps only three fields:

out.append({
    "start": round(float(seg.get("start") or 0.0), 2),
    "end": round(float(seg.get("end") or 0.0), 2),
    "text": text,
})

The confidence signal is on the wire already and is being dropped. No extra API call, no extra cost.

Possible approaches

Not attached to any of these — flagging the problem, not prescribing the fix:

1. Drop segments with high no_speech_prob. The usual heuristic is no_speech_prob > 0.6 combined with a low avg_logprob. Worth measuring against a couple of real clips before picking numbers.
2. Detect repetition. Identical segment text recurring at regular intervals is a strong hallucination signature and needs no thresholds.
3. Report rather than filter. If most segments look like non-speech, keep the text but mark the source as unreliable, e.g. _Source: whisper (groq) — low confidence, this video may have no speech._ so the model discounts it.

(3) alone would remove most of the harm and is the smallest change.

Environment

Windows 11, Python 3.13, ffmpeg 9.0.1, yt-dlp 2026.08.19, plugin 0.2.0, Groq whisper-large-v3.

Unrelated to my open PRs (#216, #217, #221) — this reproduces with or without them.