#4653·firecrawl

[Self-Host] JSON/query extraction sends the whole page to the model; with local models the instructions are silently truncated away

Author: LurigeLarsCreated Sep 16, 2026Updated Sep 16, 2026

Describe the Issue For formats: ["json"] (and query / summary), generateCompletions in apps/api/src/scraper/scrapeURL/transformers/llmExtract.ts builds the prompt from the full page markdown without trimming it to the model's input limit. On the hosted service the default model has a large context, so this rarely matters. Self-hosted users with a local model (Ollama via OLLAMA_BASE_URL / OPENAI_BASE_URL + MODEL_NAME) typically have 8k-32k tokens of context.

When the prompt is too long, Ollama truncates it by keeping only the first few tokens and the tail. The extraction instructions and schema (at the start of the prompt) and the top of the page (for example a Wikipedia infobox) are dropped, and the model returns confident but wrong values. No error or warning reaches the caller.

trimToTokenLimit already exists in the same file, and model limits are looked up via getModelLimits, but that lookup only knows hosted model names, and the JSON path does not trim.

To Reproduce

  1. Self-host v2.11.343 with OLLAMA_BASE_URL/OPENAI_BASE_URL pointing to Ollama and MODEL_NAME=qwen2.5:7b (Modelfile with num_ctx 16384).
  2. POST /v2/scrape for https://en.wikipedia.org/wiki/Ericsson with formats: [{ "type": "json", "prompt": "Company facts", "schema": { name, founded, founder, headquarters } }].
  3. Ollama logs: truncating input prompt limit=8194 prompt=45757 keep=4 new=8194.
  4. The result is partly invented, for example "founder": "Ericsson Family" and a made-up list of headquarters cities (the infobox says Lars Magnus Ericsson and Kista, Stockholm).

Expected Behavior The page content is trimmed from the end to fit the configured model's input budget before the call, so the instructions, schema, and beginning of the page always reach the model, plus a warning when content was trimmed.

Evidence / workaround A small proxy between Firecrawl and Ollama that cuts the longest text field from the end to about 7,000 tokens fixed it without other changes. Same model, 4 pages (Klarna, Spotify, Ericsson, iana.org), 18 field checks against the pages' own content: all 18 correct afterwards, and the Ericsson example above returned the correct values.

Proposal

  • Trim markdown in generateCompletions with trimToTokenLimit, keeping the head, and add a warning when trimmed.
  • Make the budget configurable for self-hosting, for example MODEL_MAX_INPUT_TOKENS (fall back to getModelLimits for known hosted models).

Environment

  • OS: Windows 11 host, Docker Desktop 29.7.2 (Linux containers)
  • Firecrawl Version: v2.11.343, built from source with the root docker-compose.yaml
  • LLM: Ollama 0.34.1, qwen2.5:7b (16k context), Intel Arc iGPU

Related: #4083 (misconfigured OpenAI settings failing silently); this issue is about correctly configured local models. I'm happy to open a PR if this approach works for you.