whisper: numerical parity tests fail with MLX 0.32.0 on Apple Silicon
Author: sfeng01-techCreated Aug 18, 2026Updated Aug 18, 2026
Summary
The Whisper example's numerical-parity tests fail with mlx==0.32.0 on Apple Silicon. The same failures reproduce on a clean checkout of main, so they are independent of PR #1439, where this was discovered.
Environment
- macOS on Apple Silicon
- Python 3.14
- MLX 0.32.0
- Current
main
Reproduction
cd whisper
python -m unittest \
test.TestWhisper.test_torch_mlx \
test.TestWhisper.test_decode_lang \
test.TestWhisper.test_decode_greedyObserved failures
test_torch_mlx: Torch/MLX logits are no longer within the existing1e-2tolerance.- max absolute difference: ~0.254
- mean absolute difference: ~0.052
test_decode_lang: English probability differs by ~1.1e-4.test_decode_greedy: no-speech probability differs by ~1.7e-4.
The requirements currently allow any mlx>=0.11, so a current installation resolves to MLX 0.32.0.
Expected
Either the Whisper implementation should retain parity with current MLX, or the supported/tested MLX version should be pinned and documented.
Source: ml-explore/mlx-examples