coli run: SNAP env var never set for non-glm engines (olmoe confirmed) — always fails with 'started without a model'
Summary
coli run (and any one-shot invocation of openai_server-adjacent gateway) never sets the SNAP environment variable for non-glm engine families. The engines themselves (olmoe.c confirmed, likely inkling.c/kimi_k3.c/deepseek_v4.c too since they share the same env_for_engine() launcher path) read SNAP to locate the model directory, so coli run always fails with "started without a model" even when the exact same --model directory passes coli doctor cleanly.
Root cause
In c/coli:
env_for_engine(a, arch)(around line 426) only special-casesarch == "glm", delegating toenv_for(a), which is the only function that setsSNAP(e = dict(os.environ, SNAP=a.model), around line 679).- For every other arch,
env_for_enginebuilds its ownenvdict fromos.environ.copy()and never setsSNAP. cmd_run'solmoebranch (around line 1300) calls the engine directly:
with noe = env_for_engine(a, arch) result = subprocess.run([engine, str(cap_for_launch(a.cap, e, 16)), "8"], input=prompt + "\n", text=True, env=e, check=False)SNAPineand no model path passed as an argument either.olmoe.creads the model path exclusively viagetenv("SNAP")(line ~1585).
Contrast with coli chat/coli serve for the same non-glm archs: those spawn openai_server.py as a subprocess, and openai_server.py sets SNAP itself when it in turn launches the engine (child_env = dict(env or os.environ, SNAP=str(model), SERVE="1", ...), around line 2938). That's why chat/serve work while run does not.
Repro
cd c
make olmoe
python3 tools/convert_olmoe_merged.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ./olmoe_merged
python3 ./coli doctor --model ./olmoe_merged # -> result ok
python3 ./coli run --model ./olmoe_merged "hi" # -> fails
Output of the failing run:
colibri: this is the OLMoE engine, and it was started without a model.
The engine is not the program you run directly -- the launcher is:
./coli chat --model <model directory> interactive chat
...
immediately followed by the run banner (OLMoE · 7B · 7.4 GB on disk / run) and a non-zero exit — the launcher believes it found and validated the model (via need_model/resolve_model), then hands the engine an environment missing SNAP.
Manually adding SNAP confirms the engine and model are otherwise fine:
SNAP="$(pwd)/olmoe_merged" CHAT=1 MAX_NEW=100 ./olmoe 16 8 <<< "Explain what a mixture-of-experts model is."
# -> loads, generates a coherent answer in ~9s
Environment
- macOS 26.6.2, Apple M3 Max, colibri v1.11.0, built from source (
make olmoe, no METAL — this family has no Metal path). - Confirmed on the
olmoearch; worth checking whetherinkling/kimi_k3/deepseek_v4'scmd_runbranches have the same gap, since they shareenv_for_engine().
Suggested fix
Either have env_for_engine() set SNAP=a.model unconditionally (mirroring what env_for() does for glm, and what openai_server.py already does when it spawns these same engines), or have each non-glm branch of cmd_run pass SNAP explicitly before invoking the engine directly.
Source: JustVugg/colibri