[Bug]: Qwen 3.8 Flash Next is too slow
Commit or version
git clone --branch dev --single-branch https://github.com/JustVugg/colibri.git
Environment
Debian 13, AMD 7950X, 64GB DDR5, 2TB Gen 4 Samsung Pro 990.
Reproduction steps
COLI_FORCE_CPU=1 Q38_MAXT=190000 Q38_NATIVE_FP8=1 Q38_NATIVE_BF16=1 Q38_PREFILL_BATCH=1 PROF=1 CTX=190000 URING=1 PILOT=1 SPEC_PIN=1 TOPP=1.0 COLI_TOOL_SALVAGE=1 COLI_DEBUG=1 COLI_TIMERS=1 COLI_API_KEY=local-secret ./coli web --model /media/user/6f1e9b86-52a9-4ef6-a5fd-8965ef1db31f/LLM/Qwen3.8-Flash-Next-FP8 --ram 43 --temp 1.0 --topp 1.0 --host 127.0.0.1 --port 8000 --model-id Qwen3.8-Flash-Next-FP8
Expected behavior
I watched this Video here: https://www.youtube.com/watch?v=_TCRH725hAM and was wondering why he is getting 20 tok/s with llama.cpp by loading only the experts to the ram but the integrated lookup table is not loaded to the ram, it is acceded directly from ssd.
Actual behavior and logs
Qwen 3.8 Flash Next have 0.3 tok/s and Kimi K3 have 0.5 tok/s how is this possible? Qwen have only 6B Active parameter and it barely using the SSD to get there Experts. Using only 200 to 500 MB/s instead of the maximal available 3000 MB/s.
Source: JustVugg/colibri