[Bug]: GPU doesn't engage

Commit or version

f028d26b422144ed4a69ad9aeaee2553ce0f9572

Environment

OS: Windows 11, CUDA: 13.3, V13.3.73 Storage: SSD WD_BLACK SN7100 2TB compiler: MSYS2 I'm reporting now while I try to run deepseek-v41-flash, but I had the same problem with v4(it was quicker, but GPU didn't engage as well)

Reproduction steps

EDIT: I managed to run everything from powershell and x64 native tools command prompts, so imagine that now I follow the newest instructions from the docs, the problem still remains though

(from powershell) python -m pip install -U "huggingface_hub[hf_transfer]"
$env:HF_HUB_ENABLE_HF_TRANSFER = "1"
hf download deepseek-ai/DeepSeek-V4.1-Flash --local-dir C:\msys64\home\Poly\deepseek git clone https://github.com/JustVugg/colibri.git cd c make colibri.exe ARCH=native

gcc -D_FILE_OFFSET_BITS=64 -O3 -march=native -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function colibri.c -o colibri.exe -lm -fopenmp -static -lpsapi process_begin: CreateProcess(NULL, gcc -D_FILE_OFFSET_BITS=64 -O3 -march=native -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function colibri.c -o colibri.exe -lm -fopenmp -static -lpsapi, ...) failed. make (e=2): The system cannot find the file specified. make: *** [colibri.exe] Error 2

I've seen in some issues to prompt UCRT64 from x64 Native Tools Command Prompt for VS, so I do that: from this prompt: C:\msys64\\ucrt64.exe

from ucrt64 cd colibri/c make colibri.exe ARCH=native

gcc -D_FILE_OFFSET_BITS=64 -O3 -march=native -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function colibri.c -o colibri.exe -lm -fopenmp -static -lpsapi make iobench.exe gcc -D_FILE_OFFSET_BITS=64 -O3 -march=x86-64-v3 -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function iobench.c -o iobench.exe -lm -fopenmp -static -lpsapi

but now I cannot just build cuda. I close the ucrt64 shell, and from 64x Native Tools Command Prompt I run set MSYS2_PATH_TYPE=inherit C:\msys64\msys2_shell.cmd -ucrt64

now, from the new ucrt64 shell, I can run make cuda-dll CUDA_ARCH=native

"nvcc" -O3 -std=c++17 -ftz=false -arch=native -Xcompiler=-W3 -shared -DCOLI_CUDA_BUILDING_DLL
-L"C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.3/lib/x64" -lcudart
backend_cuda.cu -o coli_cuda.dll backend_cuda.cu tmpxft_00002068_00000000-8_backend_cuda.cudafe1.cpp Creating library coli_cuda.lib and object coli_cuda.exp

make colibri.exe CUDA_DLL=1 ARCH=native

gcc -D_FILE_OFFSET_BITS=64 -O3 -march=native -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function -DCOLI_CUDA -c backend_loader.c -o backend_loader.o gcc -D_FILE_OFFSET_BITS=64 -O3 -march=native -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function -DCOLI_CUDA colibri.c backend_loader.o -o colibri.exe -lm -fopenmp -static -lpsapi colibri.c:658:12: warning: 'g_cuda_raw_experts' defined but not used [-Wunused-variable] 658 | static int g_cuda_raw_experts=-1; /* experimental ANS tier: keep this global hot prefix raw */ | ^~~~~~~~~~~~~~~~~~

now I can't run prepare_dsv41.py because I haven't tried to create a new python environment and install transformers, but from powershell I can: python .\c\tools\prepare_dsv41.py --model C:\msys64\home\Poly\deepseek

[transformers] You are using a model of type deepseek_v41 to instantiate a model of type ``. This may be expected if you are loading a checkpoint that shares a subset of the architecture (e.g., loading a sam2_video checkpoint into Sam2Model), but is otherwise not supported and can yield errors. Please verify that the checkpoint is compatible with the model you are instantiating. C:\msys64\home\Poly\deepseek\dsv41_engram.json (0.88 MB) 2 tables, 99092 compressed ids, [384006168, 384016682] rows

make deepseek_v41.exe

gcc -D_FILE_OFFSET_BITS=64 -O3 -march=x86-64-v3 -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function deepseek_v41.c -o deepseek_v41.exe -lm -fopenmp -static -lpsapi cd .. && cd web npm install added 81 packages, and audited 82 packages in 5s

20 packages are looking for funding run npm fund for details

4 vulnerabilities (2 moderate, 2 high)

To address all issues, run: npm audit fix

Run npm audit for details

npm run build

✓ 1795 modules transformed. computing gzip size... dist/index.html 0.56 kB │ gzip: 0.33 kB dist/assets/index-8GrhJFo0.css 30.46 kB │ gzip: 7.22 kB dist/assets/index-Ca2ven0A.js 242.13 kB │ gzip: 79.24 kB

✓ built in 211ms

and then from powershell $env:COLI_CUDA="1"; $env:COLI_GPU="0"; $env:CUDA_DENSE="1"; $env:CUDA_EXPERT_GB="10"

.\coli web --model C:\msys64\home\Poly\deepseek --ngen 64 --ctx 32000

dashboard: http://127.0.0.1:8000/ (opens automatically when the engine is ready) [v41] n_mtp_layers is absent from config.json; the checkpoint carries 3 DSpark stages [v41] DSpark on: 3 stages, 128 experts (3 routed), 5 tokens per draft [v41] 40 layers, 384 experts/layer (6 routed), dim 5120, hc 4, window 128, engram on ΓÇö loaded in 5.63s OpenAI-compatible API listening on http://127.0.0.1:8000/v1

I open opencode and send a simple question like 'what's the purpose of an AI harness?'

Expected behavior

I'd expect GPU to engage, thus to be faster, and in the dashboard to see something like this

but I see

Actual behavior and logs

Slow, no GPU load and picture above related

I'll download deepseek v4 again and show the logs from there (they were completely different)