[Bug] b10995-mix-3e83366: Qwen3.8-Flash-Next MTP aborts at load (nextn.hc_head_norm still [hc_dim] after rebase)
Environment
Where are you using Unsloth?
- Unsloth desktop application
- Unsloth web UI (
unsloth studio) - Unsloth CLI
- Python package or notebook
- Colab or Kaggle
(unsloth run, model loaded through POST /api/inference/load)
Operating system and version: macOS 27.0 (26A428)
GPU model(s) and accelerator backend: Apple M4 Max, 128 GB unified memory, Metal
Versions:
- Unsloth package 2026.9.5 (Studio venv Python 3.13.13)
- llama.cpp
b10995-mix-3e83366(fails) /b10909-mix-bea84f7(works, same model files and settings)
What happened?
Steps to reproduce:
unsloth studio updateto 2026.9.5, which installs llama.cppb10995-mix-3e83366.- Load
unsloth/Qwen3.8-Flash-Next-GGUFUD-Q4_K_XL with its MTP drafterMTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.ggufnext to it. Studio auto-detects the drafter and launches with--model-draft … --spec-type ngram-mod,draft-mtp --spec-draft-n-max 3. - llama-server aborts while creating the draft context.
Expected behavior: loads with MTP speculative decoding, as it does on b10909-mix-bea84f7.
Actual behavior: llama-server exits with code -6. Studio retries with --flash-attn off (same abort), then silently loads without speculative decoding. The load API returns 200 and the server reports healthy, so the only visible symptom is slower decode.
An explicit -md <head> --spec-type draft-mtp via llama_extra_args aborts the same way. Model files verified: the MTP head's sha256 matches the Hub (5ff54097…96e6), and all four Q4_K_XL shard sizes match.
Model and operation involved: GGUF inference, unsloth/Qwen3.8-Flash-Next-GGUF UD-Q4_K_XL + MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf
Diagnostics and logs
Paths shortened (<models> = the model folder).
logs/llama-server/*.log:
0.14.337.527 W Setting 'enable_thinking' via --chat-template-kwargs is deprecated. Use --reasoning on / --reasoning off instead.
0.14.337.579 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.14.340.745 W srv llama_server: -----------------
0.14.342.889 I srv load_model: loading model '<models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf'
0.14.808.763 E llama_model_load: error loading model: borrow_shared_tensor: this model is a draft head without its own 'token_embd.weight'; load it as a draft of its target model, not on its own
0.14.808.889 E llama_model_load_from_file_impl: failed to load model
0.14.809.347 W operator(): failed to measure the memory of the extra model, fitting without it: failed to load model
0.18.830.344 W llama_context: n_ctx_seq (393216) > n_ctx_train (262144) -- possible training context overflow
0.19.489.381 I cmn init: llama threadpool init, n_threads = 12
0.19.563.146 I common_speculative_init_result: loading draft model '<models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf'
0.19.820.505 W llama_context: n_ctx_seq (393216) > n_ctx_train (262144) -- possible training context overflow
/Users/runner/work/llama.cpp/llama.cpp/src/ggml/src/ggml.c:2293: GGML_ASSERT(ggml_can_repeat(b, a)) failed
WARNING: Using native backtrace. Set GGML_BACKTRACE_LLDB for more info.
WARNING: GGML_BACKTRACE_LLDB may cause native MacOS Terminal.app to crash.
See: https://github.com/ggml-org/llama.cpp/pull/17869
0 libggml-base.0.dylib 0x0000000102b9d3f0 ggml_print_backtrace + 276
1 libggml-base.0.dylib 0x0000000102c10e3c ggml_abort + 156
2 libggml-base.0.dylib 0x0000000102c112c8 ggml_div_impl.cold.1 + 0
3 libggml-base.0.dylib 0x0000000102b9fdec ggml_mul_inplace + 0
4 libllama.0.dylib 0x0000000102ed213c _ZN20llama_model_qwen4exp5graph12build_hc_mixEP11ggml_tensorS2_S2_S2_S2_PS2_i + 116
5 libllama.0.dylib 0x0000000102ed4b58 _ZN20llama_model_qwen4exp9graph_mtpC2ERK11llama_modelRK16llm_graph_params + 2388
6 libllama.0.dylib 0x0000000102ed2074 _ZNK20llama_model_qwen4exp16build_arch_graphERK16llm_graph_params + 76
7 libllama.0.dylib 0x000000010303d924 _ZNK11llama_model11build_graphERK16llm_graph_params + 40
8 libllama.0.dylib 0x0000000102f9369c _ZN13llama_context13graph_reserveEjjjPK22llama_memory_context_ibPm + 1096
9 libllama.0.dylib 0x0000000102f92df0 _ZZN13llama_context17resolve_fused_opsEPK22llama_memory_context_ijENK3$_0clERK18llm_fused_op_probeRb + 96
10 libllama.0.dylib 0x0000000102f92d54 _ZN13llama_context17resolve_fused_opsEPK22llama_memory_context_ij + 344
11 libllama.0.dylib 0x0000000102f92110 _ZN13llama_context13sched_reserveEv + 428
12 libllama.0.dylib 0x0000000102f90d4c _ZN13llama_contextC2ERK11llama_model20llama_context_params + 4644
13 libllama.0.dylib 0x0000000102f9b824 llama_init_from_model + 892
14 libllama-common.0.dylib 0x0000000103e5073c _ZN30common_speculative_init_resultC2ER13common_paramsP11llama_modelP13llama_context + 624
15 libllama-common.0.dylib 0x0000000103e50a78 _Z35common_speculative_init_from_paramsR13common_paramsP11llama_modelP13llama_context + 64
16 libllama-server-impl.dylib 0x00000001034d72a4 _ZN19server_context_impl10load_modelER13common_params + 2124
17 libllama-server-impl.dylib 0x0000000103439d24 _Z12llama_serverR13common_paramsiPPc + 20412
18 libllama-server-impl.dylib 0x0000000103434d04 _Z12llama_serveriPPc + 164
19 dyld 0x000000018ceefe80 start + 6688
(The borrow_shared_tensor error near the top also appears on b10909 and is harmless: it is the memory-fit probe of the shared head on its own. The abort is new in b10995.)
logs/server/server-*.log (first attempt, Studio's own launch arguments):
2026-09-16T19:56:56.435963Z INFO Detected MTP drafter: <models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf
2026-09-16T19:57:12.409988Z INFO GGUF size: 103.7 GB, mmproj: 0.8 GB, est. KV cache: 9.2 GB, MTP reserve: 12.81 GB (draft KV @ 393216 x 2 slots, ubatch 512, n_max 3), context: 393216, GPUs free: [], selected: None, --fit: on
2026-09-16T19:57:12.421005Z INFO Using separate MTP drafter: <models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf
2026-09-16T19:57:12.421052Z INFO Spec decoding: draft-mtp (chained ngram-mod)
2026-09-16T19:57:12.422403Z INFO Capping llama-server context checkpoints at 29 per slot (this build's default 32): each one snapshots this model's whole recurrent state (112.2 MiB), so the default would hold 7.0 GiB of host RAM across 2 slot(s).
2026-09-16T19:57:12.422582Z INFO Starting llama-server: ~/.unsloth/llama.cpp/build/bin/llama-server -m <models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --port 50438 --parallel 2 --flash-attn on --no-context-shift -c 393216 --video-fps 1 --alias Qwen3.8-Flash-Next-GGUF --fit on --metrics --slot-save-path ~/.unsloth/studio/cache/llama-slots --kv-unified --fit-ctx 393216 --jinja --model-draft <models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf --spec-type ngram-mod,draft-mtp --spec-draft-n-max 3 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 --chat-template-kwargs {"enable_thinking": true, "preserve_thinking": true} --mmproj <models>/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/mmproj-F16.gguf --ctx-checkpoints 29
2026-09-16T19:57:32.331777Z ERROR llama-server exited with code -6. Output (tail): [same backtrace as above]
2026-09-16T19:57:32.332251Z WARNING llama-server hard-crashed at startup (exit -6) with flash attention on; retrying once with --flash-attn off (keeps vision and MTP).
2026-09-16T19:57:50.699634Z ERROR llama-server exited with code -6. Output (tail): [same backtrace as above]
2026-09-16T19:57:50.702631Z WARNING llama-server failed to start with speculative drafter (mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf); retrying without speculative decoding in case the drafter is the cause.
Additional context
The likely cause is the re-merge of https://github.com/unslothai/llama.cpp/pull/144 (MTP for Qwen3.8-Flash-Next) onto upstream ggml-org/llama.cpp PR 28896 ("qwen4exp: enable rms_norm + mul fusion") in f40f64a81 "Fix merge conflicts". That PR's pinned commit is the only patch that changed between b10909-mix-bea84f7 (a9e9c3c) and b10995-mix-3e83366 (f40f64a).
That upstream change made the trunk hyper-connection norm gammas to load as { n_embd, hc } with TENSOR_ALLOW_RESHAPE, and build_hc_mix now does ggml_mul(ggml_rms_norm(x), w_norm) directly on the [n_embd, hc, n_tokens] stream, with no reshape to hc_dim first. The MTP head's gamma was not converted:
// src/models/qwen4exp.cpp:280 at f40f64a81
layer.nextn.hc_head_norm = create_tensor(tn(LLM_TENSOR_NEXTN_HC_HEAD_NORM, "weight", il), { hc_dim }, flags);
graph_mtp passes it to build_hc_mix at :644, so the multiply sees a [10240] weight against a [2560, 4, n_tokens] input, and ggml_can_repeat fails, matching the backtrace. The GGUF stores it flat, like the trunk gammas: the head has blk.48.nextn.hc_head_norm.weight = [10240], and blk.48.hc_attn_norm / hc_ffn_norm are also [10240] but are declared { n_embd, hc } + TENSOR_ALLOW_RESHAPE. Presumably the fix is:
layer.nextn.hc_head_norm = create_tensor(tn(LLM_TENSOR_NEXTN_HC_HEAD_NORM, "weight", il), { n_embd, hc }, flags | TENSOR_ALLOW_RESHAPE);
nextn.hnorm (:277) is still { hc_dim } too, but graph_mtp reshapes to 2D before multiplying (:540), so it looks unaffected. I have not built or tested this. It comes from reading the source, the GGUF header and the backtrace.
Workaround: UNSLOTH_LLAMA_RELEASE_TAG=b10909-mix-bea84f7 UNSLOTH_LLAMA_TAG=b10909 unsloth studio update.
Separately, it would help if the MTP fallback were reported more loudly. The load API returns 200, and the only sign of the fallback is a warning in the server log.
Source: unslothai/unsloth