OpenAI API: auto-switch does not cold-load a downloaded GGUF, so `/v1/chat/completions` returns 400 "No model loaded" when nothing is loaded

Summary

With openai_api_auto_switch_model enabled, an OpenAI-compatible request that names a fully downloaded local GGUF is served correctly when some model is already loaded, but returns 400 No model loaded. Call POST /inference/load first. when nothing is loaded at all. The setting's own documentation says it cold-loads:

openai_api_auto_switch_model: when on, a /v1 request whose model names a downloaded local model different from the loaded one transparently loads it before serving (llama-swap-style).

and _no_model_loaded_detail describes it as:

Auto-switch (default off) cold-loads a requested downloaded GGUF

In practice a cold server never loads, so every external agent that talks to Studio over /v1 fails on its first request after Studio starts, until the user loads a model by hand in the UI.

Environment

• unsloth 2026.9.4 (PyPI, install_source: pypi), Unsloth.app 0.1.808-beta
• macOS 26 (Darwin 27.0.0), Apple M4 Pro, 24 GB
• Studio bound to 127.0.0.1:8888, keyless_api_access_scope: "full"

Reproduction

1. Quit Unsloth Studio, relaunch it, and do not load a model. Confirm nothing is loaded:

$ curl -s localhost:8888/v1/models
{"object":"list","data":[{"id":"unsloth/Qwen3.8-27B-GGUF","object":"model","loaded":false,"quant":"UD-Q4_K_S",...}]}

1. Confirm auto-switch is on, as the running server sees it:

$ curl -s localhost:8888/api/settings/openai-auto-switch
{"enabled":true,"auto_unload_idle_seconds":0,"default_enabled":false,"idle_unload_active":false,
 "auto_unload_keep_kv":true,"auto_download_model":false,"auto_unload_api_only":false,...}

1. Request that model over the OpenAI-compatible route:

$ curl -s localhost:8888/v1/chat/completions \
    -H 'Content-Type: application/json' \
    -d '{"model":"unsloth/Qwen3.8-27B-GGUF","messages":[{"role":"user","content":"hi"}],"max_tokens":8}'
{"error":{"message":"No model loaded. Call POST /inference/load first.",
          "type":"invalid_request_error","param":null,"code":null}}

HTTP 400, returned in ~150 ms. No load is attempted.

Expected

Auto-switch loads unsloth/Qwen3.8-27B-GGUF and serves the request, as it does when a different model is already loaded.

Actual

400 No model loaded. Call POST /inference/load first. The hint suffix ("Or enable Model auto-switch...") is correctly omitted, which confirms the server sees auto-switch as enabled. The switch simply did not run.

Additional detail

• Reproduces identically with the quant suffix: unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_S.
• Not a cold-index effect: warming the resolver with GET /v1/models (which logs Cached model scan: inspected=7 ... returned=2) and retrying gives the same 400.
• The model resolves fine out of band. In Studio's own venv:

>>> from core.inference.local_model_resolver import resolve_local_gguf, index_is_built
>>> index_is_built()
False
>>> resolve_local_gguf('unsloth/Qwen3.8-27B-GGUF')
('/Users/…/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca7207…',
 'UD-Q4_K_S', 'unsloth/Qwen3.8-27B-GGUF')

• Once any model is loaded, switching between downloaded models over /v1 works as documented. The failure is specific to the empty-slot case.
• Backend log shows only request_completed … /v1/chat/completions status_code=400 process_time_ms=149.32; no switch or load attempt is logged.

Impact

Any external agent driving Studio over the OpenAI API (this came up with Hermes Agent) fails on the first request after every Studio restart, with an error that tells the user to call an endpoint the auto-switch feature exists to avoid.

Related

#9171 (OpenCode) and #10306 (PocketPal) both report Studio not auto-loading a model for an API request from an external client, so this may be one underlying problem and they are probably worth consolidating.

One distinction worth preserving if they are: the error text quoted in #10306 still ends with "Or enable Model auto-switch (settings > API)", which means auto-switch was off in that report. This issue is the case where it is verifiably on. GET /api/settings/openai-auto-switch returns enabled: true, and that same hint suffix is absent from the 400 above, which is the server confirming it reads the setting as enabled. The switch runs and does nothing, rather than being disabled.

Also related on the client-visible side: #6380 and #10911, where the same external-agent setups hit long silent waits once a model is loaded.