macOS 27: long-context inference 10x slower than macOS 26.6 (even after HOST_VM_INFO64 fix)

Environment

omlx: 0.4.4rc1 (also tested on 0.4.3) macOS: 27.0 (upgraded from 26.6) Hardware: M3 Max (16-core CPU, 40-core GPU), 48GB RAM (also reproduced on M2 Ultra 48GB) Model: mlx-community/Qwen3.6-35B-A3B-4bit Background

Issue #1749 (HOST_VM_INFO64 crash) was fixed in 0.4.3 and the crash no longer occurs. However, long-context inference performance is still severely degraded compared to macOS 26.6 with the exact same omlx version, model, and settings.

Benchmark results

Context macOS 26.6 macOS 27 (0.4.4rc1) Short (~1K) ~100+ tok/s ~40-50 tok/s Long (~12K) ~80+ tok/s ~4-8 tok/s Long (~20K) N/A ~4.7 tok/s (50 tok generation) Observations

Short-context benchmark via omlx admin dashboard reaches 100+ tok/s on macOS 27, so GPU frequency appears normal under benchmark workloads. Real API chat completions with large context (10K-20K prompt tokens) are 10-15x slower than the same workload on macOS 26.6. Per-token generation time with 12K context is ~213ms/token (vs expected ~12ms/token on macOS 26.6). process_memory_enforcer logs show no errors after 0.4.3 fix. Prefill memory guard is disabled. burst_decode_mode is set to aggressive. iogpu.wired_limit_mb has been set to 43008 (Metal cap issue worked around). Suspected cause

The HOST_VM_INFO64 crash is fixed, but the KV cache / paged cache management may still not be functioning correctly on macOS 27. The gap between benchmark speed (short context, 100+ tok/s) and real API speed (long context, 4-8 tok/s) suggests the SSD cache tiering or hot/cold cache eviction is not working as on macOS 26.6.

Temporary workaround

Setting prefill_memory_guard: false, burst_decode_mode: aggressive, enable_thinking: false helped short-context speed but didn't fix long-context degradation. The only reliable fix currently is to use a cloud API provider instead of local inference. Request Could you please investigate the long-context KV cache performance regression on macOS 27? Happy to run any diagnostics or provide logs.