Chat answers are silently truncated at 1024 output tokens

Author: chriscrosstalkCreated Sep 14, 2026Updated Sep 14, 2026

Problem

Every chat response in v1.35.0-rc.1 is capped at 1024 output tokens. The cap does not scale with the context window, and nothing indicates to the user that the answer was cut short rather than finished.

This appears to be an unintended side effect of the prompt-budgeting work in #1253; it is not described in any release note.

Steps to reproduce

  1. Ask the assistant for anything long-form, e.g.

    Write an extremely detailed manual on building a raised bed vegetable garden. Cover site selection, materials, dimensions, soil mix ratios, irrigation, crop rotation, pest management, and month-by-month maintenance. Write at least 4000 words.

  2. Observe where the answer stops.

Expected vs actual

Expected: a long answer, or a clear signal that the output was truncated.

Actual: roughly 700 words, ending mid-sentence, with no indication. Measured on three servers with two different models:

Server Model Words returned Ends on
A llama3.1:8b 699 ...Alternate Between Annuals
B llama3.1:8b 657 ...tomatoes, peppers, egg (mid-word)
C qwen2.5vl:3b 715 ...help deter pests and

All three returned done: true with no done_reason, so neither the UI nor an API client can distinguish truncation from completion.

Root cause

admin/app/utils/context_budget.ts:

typescript
export const MAX_RESPONSE_RESERVE = 1024;
...
const responseReserve = inputs.responseReserve ??
    Math.max(256, Math.min(MAX_RESPONSE_RESERVE,
             Math.floor(contextWindow * RESPONSE_RESERVE_FRACTION)));
...
numPredict: responseReserve,

planPrompt() returns the reserve as numPredict, which ollama_controller.ts passes to Ollama as the generation cap. Because of the Math.min, a 65,536-token window still yields min(1024, 16384) = 1024.

Confirmed the window itself is being set correctly — the admin log shows [ContextWindow] qwen2.5vl:3b: 65536 tokens and Ollama /api/ps reports context_length: 65536. Only the output reserve is pinned.

Suggested fix

Let the reserve scale with the window (drop or substantially raise the ceiling), and surface done_reason: "length" to the client so the UI can show a truncation notice or a continue affordance.

Long guides, complete procedures and document summaries are a core use case for an offline knowledge server, and they are what this affects.

Files involved

  • admin/app/utils/context_budget.tsMAX_RESPONSE_RESERVE, planPrompt
  • admin/app/controllers/ollama_controller.ts — passes numPredict upstream

Source: Crosstalk-Solutions/project-nomad