Chat answers are silently truncated at 1024 output tokens
Problem
Every chat response in v1.35.0-rc.1 is capped at 1024 output tokens. The cap does not scale with the context window, and nothing indicates to the user that the answer was cut short rather than finished.
This appears to be an unintended side effect of the prompt-budgeting work in #1253; it is not described in any release note.
Steps to reproduce
Ask the assistant for anything long-form, e.g.
Write an extremely detailed manual on building a raised bed vegetable garden. Cover site selection, materials, dimensions, soil mix ratios, irrigation, crop rotation, pest management, and month-by-month maintenance. Write at least 4000 words.
Observe where the answer stops.
Expected vs actual
Expected: a long answer, or a clear signal that the output was truncated.
Actual: roughly 700 words, ending mid-sentence, with no indication. Measured on three servers with two different models:
| Server | Model | Words returned | Ends on |
|---|---|---|---|
| A | llama3.1:8b |
699 | ...Alternate Between Annuals |
| B | llama3.1:8b |
657 | ...tomatoes, peppers, egg (mid-word) |
| C | qwen2.5vl:3b |
715 | ...help deter pests and |
All three returned done: true with no done_reason, so neither the UI nor an API client can distinguish truncation from completion.
Root cause
admin/app/utils/context_budget.ts:
export const MAX_RESPONSE_RESERVE = 1024;
...
const responseReserve = inputs.responseReserve ??
Math.max(256, Math.min(MAX_RESPONSE_RESERVE,
Math.floor(contextWindow * RESPONSE_RESERVE_FRACTION)));
...
numPredict: responseReserve,planPrompt() returns the reserve as numPredict, which ollama_controller.ts passes to Ollama as the generation cap. Because of the Math.min, a 65,536-token window still yields min(1024, 16384) = 1024.
Confirmed the window itself is being set correctly — the admin log shows [ContextWindow] qwen2.5vl:3b: 65536 tokens and Ollama /api/ps reports context_length: 65536. Only the output reserve is pinned.
Suggested fix
Let the reserve scale with the window (drop or substantially raise the ceiling), and surface done_reason: "length" to the client so the UI can show a truncation notice or a continue affordance.
Long guides, complete procedures and document summaries are a core use case for an offline knowledge server, and they are what this affects.
Files involved
admin/app/utils/context_budget.ts—MAX_RESPONSE_RESERVE,planPromptadmin/app/controllers/ollama_controller.ts— passesnumPredictupstream
Source: Crosstalk-Solutions/project-nomad