Measuring diffusion video performance on a MacBook: one speedup and a large gap

2026年8月8日3 次浏览来源:Dev.to阅读原文

Last month, I published a benchmark showing a 1.125× speedup from block-residual caching on 4-bit FLUX.

The main lesson was not the multiplier.

It was that my original quality metrics had been measuring the wrong thing, and that acceleration claims often combine speed, trajectory preservation, and perceptual quality into one number.

For the follow-up, I chose a stricter target: real-time autoregressive diffusion video on an Apple M5 Max, with the definition of "real time" frozen before results were visible.

The tested configuration did not meet that target.

The fastest claim-eligible result was 1.418 native generated frames per second, compared with a 16 FPS target.

That is an 11.28× gap.

I am publishing the result because the measured bottleneck, one systems improvement, and two rejected hypotheses are useful even without a real-time result.

The evidence can be checked from a repository checkout: The setup LiveFrame evaluates Wan2.1-T2V-1.3B-based causal video models across NVIDIA H100 CUDA and Apple M5 Max MLX/Metal.

The experiments include: Causal Forcing++ for the clean M5 performance fixture Rolling Forcing for the CUDA-to-MLX portability study Frame-wise Causal Forcing++ for the H100 cache-reuse experiment The clean M5 fixture produces 81 pixel frames at 480×832, corresponding to 5.06 seconds at the model's native 16 FPS.

Before holdout results were visible, the relevant protocols froze their prompts, seeds, content strata, horizons, thresholds, aggregation rules, and stop rules.

For the cross-runtime experiment, stochastic inputs were serialized once as BF16 tensors.

CUDA and MLX consumed byte-identical tensors rather than relying on nominally matching random seeds.

LiveFrame separates four claim layers: Numerical trajectory: Does the candidate follow the reference latents?

Intrinsic quality: Is the candidate video acceptable independently of the reference trajectory?

Same-seed identity: Is it recognizably the same generated video?

Complete-wall performance: How long does the measured generation pipeline take, including mandatory refresh, VAE decoding, materialization, synchronization, and encoding?

The clean M5 row excludes model loading and prompt encoding, so it is a complete measured generation wall rather than application startup time.

Result 1: A systems speedup with matching RGB digest records Increasing MLX's bounded free-buffer cache from 1 GiB to 4 GiB reduced complete measured wall time on the frozen fixture from 69.167 to 57.127 seconds.

That is a 1.2108× speedup, or a 17.41% wall-time reduction.

All measured trials: Recorded the same SHA-256 digest for the 97,044,480-byte pre-encode RGB output Passed the pinned latent anchor Added zero swap The raw RGB payload itself was not retained.

The exact-output evidence therefore consists of repeated immutable trial records containing the same digest, not an independently redistributable copy of the raw pixels.

The retained H.264 output and its decoded RGB have separate hashes.

The profile was more informative than the multiplier.

In this M5 configuration, causal VAE decoding consumed 72.33% to 78.69% of complete measured wall time.

The transformer had already been distilled to one denoising step.

For this configuration, decoder execution and memory reuse were the larger optimization targets.

Result 2: The measured gap to real time With the allocator improvement included, the fixture reached 1.418 native generated FPS.

LiveFrame defines real time as at least 16 native generated FPS for a continuous 60-second stream, with time to first frame, p95 chunk latency, memory, thermal state, quality, identity, and drift disclosed.

Interpolated or duplicated presentation frames do not count toward native generation rate.

The measured gap is: 16 ÷ 1.4179056 = 11.2842× This result has two important boundaries: It is an 81-frame clean fixture, not a sustained 60-second performance result.

It is not an Apple Silicon impossibility result.

Quantization, lower-resolution tiers, diffe

分享