在 Apple Silicon 上, LLM 解码速度最高可提高 3 倍, 无损。 DeepSeek 的 DSpark 和 z-lab 的 DFlash 推测性解码的原生 MLX 端口 – Gemma-4、Qwen3、Ornith-1.0
在 Apple Silicon 上, LLM 解码速度最高可提高 3 倍, 无损。 DeepSeek 的 DSpark 和 z-lab 的 DFlash 推测性解码的原生 MLX 端口 – Gemma-4、Qwen3、Ornith-1.0
DeepSeek's DSpark and z-lab's DFlash speculative decoding — native on Apple Silicon via MLX.
Lossless drafters (same output, just faster) for Gemma-4, Qwen3, LFM2.5, Muse-Glimmer, Ornith-1.0, Qwen3.6, Qwen3.8, Nemotron, and Bonsai targets —
plus any matched DSpark / DFlash checkpoint. Run them at the CLI, from Python, serve an OpenAI-compatible API to LM Studio / any local tool,
or drive Claude Code with a model on your own Mac.
mlx-dspark runs two EAGLE-family speculative-decoding drafters natively on Apple Silicon: DeepSeek's DSpark (semi-autoregressive, from the DeepSpec codebase, used to accelerate DeepSeek-V4) and z-lab's DFlash (block diffusion). Both are lossless — the target verifies every token, so output is identical to normal decoding — and run under one verify loop, so you can serve them, script them, or benchmark them head-to-head.
What this is not: DeepSeek-V4 inference. The targets are consumer-size models (Gemma-4, Qwen3, Meta's Muse-Glimmer, NVIDIA's Nemotron, PrismML's ternary Bonsai-27B, …) with published DSpark drafters — so this runs the real drafter method on a Mac, but the model producing tokens is one of those, not V4. V4 Flash/Pro (MoE, batched serving) is DSpark's own headline use case.
Every row auto-resolves its drafter from --model (any quant of the target matches). Measured warm on
an M4 Pro, medians of 3 — most rows with mlx-dspark benchmark --trials 3 (three prompts:
chat/code/math), the Muse row per-content best (footnoted); full tables, baselines, and method in
Results at a glance. Sorted by best measured speedup:
Qwen3.8-27B's rows are its DFlash 2 drafter (incoai/Qwen3.8-27B-DFlash2),
which --mode auto (the default, and what the Mac app uses) resolves for both quants; the DSpark
heads for this target remain available with --mode dspark — the paired numbers are in
DFlash 2 on Qwen3.8-27B.
The speed column is the measured range across the three benchmark contents at the row's best configuration — chat at the low end, code/math at the high end (decoding speed depends on what is being generated: copy- and structure-heavy content accepts longer drafts). Baselines and per-content splits are in Results at a glance.
This table is the set of pairs we have measured and vouch for, which is also exactly the
auto-resolve registry — that is the only thing the registry is for. It is not the set of models
that work: any DeepSpec-native drafter runs against any compatible target via --drafter, and any
target at all gets drafter-free speculation via --mode auto. See
Bring your own drafter. Per-model caveats and
methodology live in the numbered footnotes at the end of this page — click a marker to jump.
Target precision: the quants shown are each model's measured best — ratios are non-monotone in bits and peak at 8-bit on current MLX (full Ornith sweep: 4-bit 1.38× · 8-bit 2.17× · bf16 1.54× on code; bf16 loses in both ratio and absolute speed because MLX's unquantized matmul pays a ~2× cost cliff at verify width 2). Details in Results at a glance.
Nothing in this table is hand-tuned per model, and none of it is pinned to this M4 Pro:
with no --max-draft, mlx-dspark measures your machine's verify/drafter cost curves once
(~5 s, cached per model + quant + mlx version) and derives the draft cap from them — an M1 or
an M5 gets its own optimum, not the one these rows were measured at. --max-draft auto
additionally adapts the cap per round while generating. See Tuning.
Copy-heavy code editing goes further: when the model re-emits or refactors code already in its
context (the daily agent/assistant workload), match-scaled lookup drafts reach 4.5× on Gemma-12B
(75 tok/s) and 3.6× on Ornith-9B (93 tok/s). Any model not listed still gets drafter-free
lookup speculation via --mode auto.
Everything below is also available as a native Mac app — chat with saved sessions, a model manager that answers "will this fit my Mac?" before you download, live speculative-decoding telemetry (per-round acceptance, this machine's measured cost curves), a "This Mac" roofline view (your measured memory bandwidth, the plain-decode ceiling for the loaded model and how far above it speculation runs, macOS memory pressure and swap), a decoder Race with a checked lossless verdict, one-click coding-agent setup, and a menu-bar gauge with live tok/s and model memory.
brew tap ARahim3/mlx-dspark https://github.com/ARahim3/mlx-dspark
brew trust arahim3/mlx-dspark # Homebrew 6+: third-party taps need explicit trust
brew install --cask mlx-dspark
xattr -dr com.apple.quarantine /Applications/mlx-dspark.app # not notarized yet: clear
# quarantine once, or use System Settings › Privacy &
# Security › "Open Anyway" after the first launch(On Homebrew ≤ 5, brew install --cask --no-quarantine mlx-dspark still works and replaces
the trust/xattr steps — Homebrew 6 removed that flag.)
Or download the DMG from Releases (app-v*
tags) and drag it to Applications. First launch sets up its own private engine runtime (no
Homebrew Python, no venv of yours touched, ~2–4 min once) and keeps the engine on the latest
release automatically; the app itself tells you when a newer app version exists
(brew upgrade --cask mlx-dspark). pip install mlx-dspark stays engine-only — the app is
not in the wheel, and the app never touches a pip-installed engine.
pip install mlx-dspark # or: uv pip install mlx-dsparkApple Silicon + Python ≥ 3.10; installs mlx ≥ 0.32.0 automatically (0.32's quantized-matmul kernels are what current speedup numbers are measured on). Model weights download from the Hugging Face cache on first use (none bundled). No server framework is pulled in — the API server is built on the standard library.
Known upstream incompatibilities (both handled): mlx-vlm 0.6.5 moved an internal rope-utils module, which crashed
import mlx_dsparkon fresh installs of mlx-dspark ≤ 0.4.2 — fixed in 0.4.3 (both module layouts supported), so upgrade mlx-dspark rather than pinning mlx-vlm. Separately, mlx-vlm 0.6.4 × transformers ≥ 5.12 breaks loading the gemma4 target with a misleadingOSError: Can't load video processor …(#4, upstream Blaizzy/mlx-vlm#1578 — fixed in mlx-vlm 0.6.5). mlx-dspark ≥ 0.3.2 shims that one at load time, so any mlx-vlm ≥ 0.6.3 works; the shim self-retires on fixed releases, andmlx-dspark doctorreports when it is active.
mlx-dspark generate --model mlx-community/Qwen3-8B-8bit --prompt "Explain rainbows."
mlx-dspark serve --model mlx-community/Qwen3-8B-8bit # OpenAI + Anthropic API on :8080That's the whole setup — swap in any target from the table above. Three things worth knowing, then you can stop reading:
--model takes any HF repo or local path (exactly like
mlx-lm); the matching drafter and that pair's measured-best settings resolve automatically.
A model that isn't in the table still gets drafter-free speculation via --mode auto, or pass
--drafter yourself.--max-draft auto additionally adapts per round while generating (the safest choice if you
only remember one flag); --max-draft N pins a value only if you've measured a better one.mlx-dspark benchmark --model --trials 3 is the same
reproducible sweep this README's tables come from (the Mac app's Race shows it live, with
a token-by-token identical-output verdict).Prefer clicking to typing? The Mac app wraps all of this — including the calibration and the model picker with "will it fit my Mac?" answered up front.
…--mode auto picks the best available speculation for any target (a known DSpark drafter → else
DFlash → else drafter-free n-gram lookup), so any repo serves with some speedup and no extra flags.
Then point any OpenAI client at it — the speculative speedup is transparent:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")
print(client.chat.completions.create(
model="Qwen3-8B-8bit",
messages=[{"role": "user", "content": "Explain rainbows briefly."}],
).choices[0].message.content)Works with any OpenAI-compatible client — the same frontends people point at llama.cpp's
llama-server or Ollama work here by switching one setting: set the OpenAI base URL to
http://127.0.0.1:8080/v1 (API key: anything). That covers Open WebUI, SillyTavern,
Continue/Cline, LibreChat, Raycast AI, and the rest of that ecosystem. (There is no GGUF
interop — mlx-dspark runs MLX weights natively — but the HTTP surface is shared, which is the
part those tools actually talk to.)
The server speaks the OpenAI API: POST /v1/chat/completions (streaming and non-streaming,
multi-turn), POST /v1/completions, GET /v1/models, GET /health (incl. a warnings list —
memory pressure, context-window RAM notes), GET /metrics, and GET /machine (this Mac's measured
bandwidth, the loaded model's bytes per token, the plain-decode ceiling and a verdict). It supports
temperature, top_p, top_k, max_tokens, stop, seed, presence_penalty / frequency_penalty,
logprobs / top_logprobs, tool calling (tools / tool_calls), a per-request thinking toggle
(enable_thinking), and per-request reasoning_effort on models whose template supports it (Qwen3.8-class;
GET /health reports supports_reasoning_effort). Each response carries an x_mlx_dspark block — accept length, decode and
end-to-end tok/s, time to first token, prompt tokens served from the prefix cache, and the decode rate as a multiple of this
Mac's single-stream roofline — so the spec-decode gain is visible and "why was this turn fast/slow" has an answer. Continuous batching (--max-batch N) serves concurrent requests in one
batched forward for ~2.5× aggregate throughput (see Concurrent throughput);
prefix caching (on by default) reuses the conversation prefix so multi-turn chat and agents don't
re-prefill each turn — measured on an ~8k-token context: first token in ~62 s cold vs 0.2–1 s on
cached turns, hybrid targets included (see Prefix caching).
mlx-dspark serve is a local, single-user server. It binds 127.0.0.1 by default; if you expose
it further, use --api-key — then every route except GET /health requires the key
(Authorization: Bearer … or x-api-key). A checkpoint that ships its own Python
(config.json: model_file, auto_map custom classes) is refused by default, because
mlx-lm/transformers would otherwise import and run it; --trust-remote-code (or
MLX_DSPARK_TRUST_REMOTE_CODE=1) opts the
暂无开放 Issues,或尚未同步最近议题。