whisper: numerical parity tests fail with MLX 0.32.0 on Apple Silicon

Author: sfeng01-techCreated Aug 18, 2026Updated Aug 18, 2026

Summary

The Whisper example's numerical-parity tests fail with mlx==0.32.0 on Apple Silicon. The same failures reproduce on a clean checkout of main, so they are independent of PR #1439, where this was discovered.

Environment

  • macOS on Apple Silicon
  • Python 3.14
  • MLX 0.32.0
  • Current main

Reproduction

bash
cd whisper
python -m unittest \
  test.TestWhisper.test_torch_mlx \
  test.TestWhisper.test_decode_lang \
  test.TestWhisper.test_decode_greedy

Observed failures

  • test_torch_mlx: Torch/MLX logits are no longer within the existing 1e-2 tolerance.
    • max absolute difference: ~0.254
    • mean absolute difference: ~0.052
  • test_decode_lang: English probability differs by ~1.1e-4.
  • test_decode_greedy: no-speech probability differs by ~1.7e-4.

The requirements currently allow any mlx>=0.11, so a current installation resolves to MLX 0.32.0.

Expected

Either the Whisper implementation should retain parity with current MLX, or the supported/tested MLX version should be pinned and documented.