Failed Ollama model pulls are reported as successful: in-band NDJSON errors are silently dropped

Summary

OllamaService._doDownloadModel reports success: true for pulls that transferred nothing. Ollama's /api/pull returns HTTP 200 and reports failures in-band, as a JSON line inside the NDJSON stream. Our stream handler reads only the progress fields off each line and silently drops everything else, including {"error": ...}. The stream then ends normally, on('end') resolves unconditionally, and we log the model as downloaded successfully.

Every failed model pull, for any reason, is reported to the user as a success.

Evidence

Against a live Ollama:

$ curl -s -o body.txt -w "HTTP_STATUS=%{http_code}\n" -X POST http://localhost:11434/api/pull \
    -d '{"model":"llama3.1:405b-instruct-fp16-nope","stream":true}'
HTTP_STATUS=200

$ cat body.txt
{"status":"pulling manifest"}
{"error":"pull model manifest: file does not exist"}

Traced through admin/app/services/ollama_service.ts:

pullResponse.data.on('error', ...) only fires for transport errors (a destroyed socket). It never sees an in-band error, because at the HTTP layer the request succeeded.

Reported instance

#1071. The reporter's log shows the whole "download" of an ~800 GB fp16 model completing in two seconds:

1783536247289  [OllamaService] Dispatching model download for llama3.1:405b-instruct-fp16 via job queue
1783536248895  [OllamaService] Download progress for model "llama3.1:405b-instruct-fp16": 100%
1783536249322  [OllamaService] Model "llama3.1:405b-instruct-fp16" downloaded successfully.

The 100% is consistent with this too: the aggregate is computed only across digests that reported progress, so if the small config/template blobs completed and the large layer errored before emitting a progress line, aggCompleted / aggTotal is legitimately 1.0.

Why this matters beyond one report

This is almost certainly inflating the "download says complete but nothing is installed" bucket that gets triaged as disk space or as a stale/partial blob. Those explanations produce workarounds that cannot work, because the pull is not reaching the disk at all. It is also why the remediation steps from #690 stopped making any difference for the reporter.

Distinct from #1214/#1047 (retained terminal job dedupes the dispatch, so no worker runs and there are no [DownloadModelJob] lines) and from #1194 (orphaned job 500s the queue endpoint). Here the worker runs to completion and misreports the outcome.

Proposed fix

Two changes in _doDownloadModel:

1. Check parsed.error in the data handler and reject the promise with it, so the existing catch reports a real failure to the user.
2. Verify the model is actually installed via getModels() before returning success. This is the load-bearing half: it catches silent failure modes we have not anticipated, not just the ones Ollama labels. The function already calls getModels() at the top to skip already-installed models, so the call is established.

A bad model name is not retryable and should not spin through attempts, consistent with how DownloadModelJob retry storms have been handled previously.