/api/ps and /api/tags report size and size_vram as a hardcoded 0
LocalAI version:
v4.9.0 (f7ad3f70eb5d8a0ddf80e08557f0d7df28cf032e), image localai/localai:v4.9.0-gpu-nvidia-cuda-12
Environment, CPU architecture, OS, and Version:
x86_64, RTX 3060 12GB, Docker on WSL2Describe the bug
GET /api/ps reports "size": 0 and "size_vram": 0 for every loaded model. The
values are literals rather than unpopulated fields:
https://github.com/mudler/LocalAI/blob/v4.9.0/core/http/endpoints/ollama/models.go#L106-L110
GET /api/tags has the same literal Size: 0 at line 40, and ExpiresAt on the ps
entry is synthesised as time.Now().Add(24 * time.Hour).
This is worse than omitting the fields. /api/ps is an Ollama-compatibility surface,
and Ollama reports real byte counts there, so a client written against Ollama reads
0 as authoritative and concludes the models cost nothing. Anything scheduling GPU
work from that will over-commit the card. An absent field or a null is safe, because
a consumer can distinguish "unknown" from "nothing"; a 0 cannot be distinguished.
To Reproduce
- Load any model (
granite-4.1-8b, llama-cpp backend) so it is resident. curl localhost:8080/api/ps- Compare against
nvidia-smi.
Expected behavior Either the real resident sizes, or the fields omitted/null when the backend cannot report them. Not 0.
Logs
{"models":[{"name":"granite-4.1-8b:latest","size":0,"size_vram":0,
"details":{"format":"gguf","family":"llama-cpp","parameter_size":"8B",
"quantization_level":"Q4_K_M"}}]}
Card at the same moment: 10849 / 12288 MiB used, two models resident (second row elided, also all-zero).
Additional context
I gave up on /api/ps for this engine entirely and treat the size as unknown unconditionally, because I can't tell its zeros from real ones.
Filed separately as a feature request: there is no endpoint that reports per-model memory at all.
I'm not in a position to take the PR, but I'm happy to test a fix against this setup.
Written by my beloved Claude Code
Source: mudler/LocalAI