[Self-Host] JSON/query extraction sends the whole page to the model; with local models the instructions are silently truncated away
Describe the Issue
For formats: ["json"] (and query / summary), generateCompletions in
apps/api/src/scraper/scrapeURL/transformers/llmExtract.ts builds the prompt from the full page markdown without
trimming it to the model's input limit. On the hosted service the default model has a large context, so this rarely
matters. Self-hosted users with a local model (Ollama via OLLAMA_BASE_URL / OPENAI_BASE_URL + MODEL_NAME)
typically have 8k-32k tokens of context.
When the prompt is too long, Ollama truncates it by keeping only the first few tokens and the tail. The extraction instructions and schema (at the start of the prompt) and the top of the page (for example a Wikipedia infobox) are dropped, and the model returns confident but wrong values. No error or warning reaches the caller.
trimToTokenLimit already exists in the same file, and model limits are looked up via getModelLimits, but that
lookup only knows hosted model names, and the JSON path does not trim.
To Reproduce
- Self-host v2.11.343 with
OLLAMA_BASE_URL/OPENAI_BASE_URLpointing to Ollama andMODEL_NAME=qwen2.5:7b(Modelfile withnum_ctx 16384). POST /v2/scrapeforhttps://en.wikipedia.org/wiki/Ericssonwithformats: [{ "type": "json", "prompt": "Company facts", "schema": { name, founded, founder, headquarters } }].- Ollama logs:
truncating input prompt limit=8194 prompt=45757 keep=4 new=8194. - The result is partly invented, for example
"founder": "Ericsson Family"and a made-up list of headquarters cities (the infobox says Lars Magnus Ericsson and Kista, Stockholm).
Expected Behavior
The page content is trimmed from the end to fit the configured model's input budget before the call, so the
instructions, schema, and beginning of the page always reach the model, plus a warning when content was trimmed.
Evidence / workaround A small proxy between Firecrawl and Ollama that cuts the longest text field from the end to about 7,000 tokens fixed it without other changes. Same model, 4 pages (Klarna, Spotify, Ericsson, iana.org), 18 field checks against the pages' own content: all 18 correct afterwards, and the Ericsson example above returned the correct values.
Proposal
- Trim
markdowningenerateCompletionswithtrimToTokenLimit, keeping the head, and add a warning when trimmed. - Make the budget configurable for self-hosting, for example
MODEL_MAX_INPUT_TOKENS(fall back togetModelLimitsfor known hosted models).
Environment
- OS: Windows 11 host, Docker Desktop 29.7.2 (Linux containers)
- Firecrawl Version: v2.11.343, built from source with the root
docker-compose.yaml - LLM: Ollama 0.34.1,
qwen2.5:7b(16k context), Intel Arc iGPU
Related: #4083 (misconfigured OpenAI settings failing silently); this issue is about correctly configured local models. I'm happy to open a PR if this approach works for you.
Source: firecrawl/firecrawl