[RFC] Evaluate PrismML Bonsai 2 support in ds4
Author: 2001YCreated Sep 18, 2026Updated Sep 18, 2026
Summary
I would like to discuss whether ds4 should support PrismML's Bonsai 2 27B ternary GGUF model.
This is a design and scope discussion, not a claim that I already have an implementation or a ready-to-submit PR.
Bonsai 2 is based on Qwen3.8-27B, but it uses a rotated ternary weight representation with custom PTQ1_0/PQ2_0 packing and an activation-side Hadamard/FWHT transform. It is distinct from the Qwen3.8 Flash Next support discussed in #867.
References:
- https://github.com/PrismML-Eng/Bonsai-demo
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf
- https://github.com/antirez/ds4/issues/867
Questions
- Is Bonsai 2 in scope for ds4?
- Would a Metal-first investigation on Apple Silicon be useful?
- Should PTQ1_0 and PQ2_0 be supported natively, or should the work first target a canonical model format?
- Should this be tracked separately from #867?
- What minimum validation would maintainers expect before opening a draft PR?
Possible investigation scope
- Validate the GGUF metadata and tensor layouts.
- Compare the PrismML llama.cpp fork against a reference implementation.
- Determine whether the existing ds4 model/session abstractions can support the Qwen3.8-27B hybrid-attention graph.
- Estimate the required Metal kernels, vision integration, and speculative-decoding support.
I am opening this issue to get direction before attempting any implementation.
Source: antirez/ds4