[BUG] Qwen3.8-27B Q8/BF16 processor initialization and sustained-decoding failures

Describe the bug

Onboarding mlx-community/Qwen3.8-27B-8bit and mlx-community/Qwen3.8-27B-bf16 exposed two distinct problems in EXO's MLX path:

1. The real VisionEncoder loader can fail to resolve an image processor even though Transformers has a compatible mapping for the model configuration. Processor metadata names Qwen3VLImageProcessor, while Transformers 5.6.2 maps Qwen3_5Config to Qwen2VLImageProcessor. Passing the explicitly loaded model configuration to the fallback resolves this failure.
2. Sustained Q8 text generation failed on two of four workers approximately 264 seconds into a retained run with [metal::malloc] Resource limit (499000) exceeded, surfaced at mx.async_eval while EXO evaluated token and logprob outputs. A focused reproduction shows lazy ArraysCache length/padding metadata graphs growing when callers evaluate a populated cache tensor without evaluating the complete cache state.

Short text generation already worked despite the caught processor error. The processor failure is not the cause of the separate sustained-decoding failure. No equivalent historical BF16 crash was captured.

Proposed fix and public reproduction artifacts: https://github.com/exo-explore/exo/pull/2301. The PR also supplies bundled cards for both exact model IDs. It remains a draft; this report does not claim the fixes are merged or released.

To Reproduce

Exact model artifacts

Model	Hugging Face revision
mlx-community/Qwen3.8-27B-8bit	815b83c0df8ffd1d1b5244cf75fd6ef14fca9ef9
mlx-community/Qwen3.8-27B-bf16	6f265714824f3c38d4452baa1628aef3d9b9aae9

Both select qwen3_5 / Qwen3_5ForConditionalGeneration: 48 recurrent layers using ArraysCache(size=2) and 16 full-attention layers. This is not a new architecture implementation.

Processor component

1. Obtain a complete model repository at the corresponding revision above and make it discoverable to EXO.
2. In the uncorrected EXO/Transformers environment, exercise detect_vision_from_config followed by the real VisionEncoder.ensure_loaded() on the Q8 model. The retained baseline loaded vision parameters and then failed with an unrecognized-image-processor error.
3. Repeat with explicit AutoConfig.from_pretrained(...) configuration passed to the existing AutoImageProcessor.from_pretrained(...) fallback, keeping assets and dependencies unchanged. The loader succeeds. Both model variants passed the loader check on the public PR branch.

The executable regression and isolated-environment commands are in the public reproduction guide. This component check loads real vision weights; it does not require a running EXO service or prove end-to-end multimodal inference.

Cache component and sustained generation

The standalone cache regression uses real MLX/MLX-LM and no model weights. It initializes both state tensors, advances length/padding metadata 256 times, and evaluates cache[0], not complete cache.state. Against the tested unpatched MLX-LM pin, the graph check fails at 1,280 edges; with the dependency correction it passes the at-most-eight-edge bound and final-value assertions.

For the historical distributed scenario, use four Ready Tensor/JACCL ranks, then run sustained streaming text generation with thinking and top-five logprobs enabled. The public guide includes deterministic request generation and SSE capture/check instructions. An HTTP 200 alone is not success: inspect stream errors, finish reason, completion, and runner health. A short answer does not exercise the retained failure.

Expected behavior

• Resolve the compatible image processor from the installed library's model-configuration mapping.
• Keep decoding metadata graph growth bounded when the consumed recurrent cache state advances.
• Complete the requested sustained generation without worker resource-limit failures.

Actual behavior

• Uncorrected real loader: processor-resolution exception after loading vision parameters.
• Retained sustained Q8 run: two workers failed with [metal::malloc] Resource limit (499000) exceeded around 264 seconds.
• Focused unpatched cache reproduction: metadata graph grows with decode steps.

The resource error is a Metal object limit, not necessarily physical RAM exhaustion. The source path, graph reproduction and corrected-run results support the cache explanation; a trace attributing every retained Metal object to these arrays was not captured. Four-node execution was the integration environment, not a demonstrated prerequisite for the cache defect.

Environment

Historical distributed testing on September 9, 2026:

• Hardware: four Mac Studio M3 Ultra nodes, four-way Tensor/JACCL over Thunderbolt 5/RDMA.
• EXO reported version: 0.3.70.
• MLX reported version: 0.32.0.dev20260522.
• MLX-LM reported version: 0.31.3, actually fork pin rltakashige/mlx-lm@6a3df6cd6b00a347ee40f12d97a182aaf86ea599, not an unmodified PyPI release.
• MLX-VLM: 0.4.4; Transformers: 5.6.2.
• Fresh public-branch component/build checks: macOS 26.6.1 (25G76), public EXO base 21a54c5ea0230a3bec1e1a786d200126c7e34ec6. These checks are separate from the historical four-node runs.

Additional context

The combined development candidate completed sustained generation of 17,992 Q8 output tokens and 19,211 BF16 output tokens. Both completed 210,668-token cold recall and a cached extension with 243,192 prompt tokens; Gemma controls also passed. Production was restored afterward.

PR #2301 has fresh public-source build/component coverage, including both processor loaders, cache regression and model-card registration. Four-node sustained generation has not been rerun on that public upstream port. No fresh full-262k prefill, concurrency, overnight stability, 4-bit, end-to-end multimodal or generated-code correctness claim is made.

The cache correction belongs to MLX-LM. The PR temporarily carries it through EXO's Nix dependency override without changing pins; uv run exo does not apply that Nix-only patch. A companion MLX-LM PR is not yet published. The adapted advance() mechanism comes from Pierre Lamy's bb615eb; related work includes MLX-LM #1632 and #1845. No endorsement by those contributors is implied.

This report was prepared with GitHub Copilot under Mark Smeltzer's direction. The retained hardware checks were agent-executed against real software and hardware using deterministic scripts. Mark completed human code review of the EXO PR.