Photon `md.photon()` fails on Apple Silicon: `ValueError: preallocated k_scale_tensor must be a device FP32 scalar`

Environment

• Mac Studio, Apple M4 Max, 128GB unified memory
• macOS (Sequoia), arm64
• Python 3.14.7
• moondream package: 2.3.0
• kestrel 0.7.2, kestrel-kernels 0.6.2, kestrel-mps-torch-ext 0.1.2, kestrel-native 0.1.8
• Tried both torch==2.14.0 and torch==2.11.0 — identical failure on both, so this isn't a torch-version issue.

Repro

​python import moondream as md model = md.photon("moondream3.1-9B-A2B") ​

Same failure with md.photon("moondream2") too — not specific to the 3.1 MoE architecture, so this looks like a general Photon/MPS engine bug, not a model-specific one.

Traceback

​ File ".../kestrel/engine/core.py", line 419, in _build_configured_runtimes self._runtimes[model_id] = self._build_runtime(model_id, max_lora_rank) File ".../kestrel/engine/core.py", line 441, in _build_runtime return spec.runtime(...) File ".../kestrel/models/moondream/runtime.py", line 715, in __init__ cache = _LayerPagedCache(page_table=self.page_table, ...) File ".../kestrel/models/moondream/runtime.py", line 451, in __init__ self.cache = PagedKVCache(page_table, ...) File ".../kestrel/kv_cache.py", line 222, in __init__ raise ValueError(f"preallocated {name} must be a device FP32 scalar") ValueError: preallocated k_scale_tensor must be a device FP32 scalar ​

What I found tracing it

In kestrel/models/moondream/runtime.py, get_device_capability() (in kestrel/device.py) correctly returns (0, 0) on MPS devices, so device_sm = 0, which is correctly excluded from _FP8_KV_SUPPORTED_SMS — so the code does take the non-quantized fallback path (confirmed by the accompanying UserWarning: ... falling back to standard KV cache).

The actual failure is in kv_cache.py's PagedKVCache.__init__, in the branch where storage is passed in (not None) — it takes k_scale_tensor = storage.k_scale_tensor from a preallocated storage object, then validates it with: ​python if ( tensor.shape != torch.Size([]) or tensor.dtype is not torch.float32 or tensor.device != pool.device ): raise ValueError(f"preallocated {name} must be a device FP32 scalar") ​ This fails, which means the storage.k_scale_tensor passed in from the caller is not on pool.device (or not the right shape/dtype) at the point this check runs — a device-placement mismatch in how the preallocated scale tensor storage is constructed/passed for MPS specifically.

Ruled out

• Not a RAM/memory issue (128GB unified memory, nowhere close to a limit for a 9B model)
• Not a Hugging Face cache/download issue (weights download and cache successfully both times; failure happens after that, during engine init)
• Not a torch version issue (identical failure on 2.11.0 and 2.14.0)
• Not model-specific (moondream3.1-9B-A2B and moondream2 both fail identically)
• Not the Station binary (using the documented pip install moondream + md.photon() path directly, not moondream-station)

No existing open/closed issue found matching this error message as of 2026-09-18.

Happy to provide more diagnostic info if useful.