SSD prefix cache never stores on qwen4_exp hybrid: available_boundaries=0 while boundary snapshots are enabled

Summary

On a qwen4_exp hybrid checkpoint, the paged SSD prefix cache never stores anything. Every store is rejected with reason=boundary_snapshot_unavailable and available_boundaries=0, even though the scheduler logs that boundary snapshots are enabled and the sequences are many multiples of block_size.

Net effect: the SSD prefix cache is inert. It retains blocks written previously but gains no new ones, so prefix reuse never survives a restart.

Environment

• omlx 0.7.0.dev2, tap at ddc47184
• mlx 0.31.2
• macOS 26.6.2, arm64 (M3 Ultra)
• Served via VLMBatchedEngine (text-only requests)

Model geometry

• model_type: qwen4_exp
• 48 layers — {'linear_attention': 36, 'full_attention': 12}
• num_key_value_heads: 2, num_attention_heads: 24, head_dim: 256
• max_position_embeddings: 262144
• Lightning MTP enabled; PLE SSD offload enabled

Observed

At startup the scheduler enlarges the block size for this cache type:

omlx.scheduler - INFO - Enlarging paged cache block_size=256 to 4096 for ArraysCache hybrid model (reduces boundary snapshot overhead)
omlx.scheduler - INFO - paged SSD cache enabled: block_size=4096, max_blocks=100000

Then, on the first request, it reports snapshots as enabled — and immediately rejects the store:

omlx.scheduler - INFO - Enabled boundary cache snapshots for stateful non-sliceable cache layers
omlx.scheduler - INFO - Skipping cache store for <id>: reason=boundary_snapshot_unavailable tokens=34886 block_size=4096 available_boundaries=0; storing live non-sliceable state would corrupt later prefix hits

This repeats for every request, at every length:

tokens=34886 block_size=4096 available_boundaries=0
tokens=32638 block_size=4096 available_boundaries=0
tokens=11374 block_size=4096 available_boundaries=0
tokens=6718  block_size=4096 available_boundaries=0

A 34,886-token sequence crosses eight 4096-token boundaries, so available_boundaries should not be 0.

Expected

With boundary snapshots enabled and sequences well over block_size, at least one boundary snapshot should be captured, allowing the store to proceed.

Impact

• du -sh on the cache directory is unchanged for 24h+ under continuous load; zero files written.
• The in-memory prefix cache still works — an identical 6,714-token prompt repeated in the same session goes 9.8s → 3.5s (2.8×) — so this is specifically the persistent path. Cold start after any restart re-prefills from scratch.

Pointers

• Rejection raised at scheduler.py _BoundaryStoreUnavailable → reason="boundary_snapshot_unavailable" (~L11924).
• _boundary_cache_snapshots populated around L6743 and L7172.
• _emit_prefill_boundary_snapshot (~L6131) carries a docstring describing a previously fixed variant of this failure, where routing through request_id_to_uid.get(request_id, -1) returned None and silently dropped every snapshot for "ArraysCache / GDN / hybrid models". The symptom here looks like a remaining path to the same outcome.

Not yet tested

Whether disabling Lightning MTP, or forcing block_size back to 256, changes the behaviour. Happy to run either if useful.