json format silently returns empty via Responses API on OpenAI-compatible backends (o3-mini escape hatch is too narrow)

Summary

When OPENAI_BASE_URL points at an OpenAI-compatible backend that does not fully implement the Responses API, formats: ["json"] / jsonOptions silently returns success: true with no json field. The same request succeeds when the model is routed through Chat Completions.

getModel() in apps/api/src/lib/generic-ai.ts already has an escape hatch for exactly this failure mode, but it is keyed on a model-name prefix:

// o3-mini returns empty text via the Responses API — force Chat Completions
if (provider === "openai" && modelName.startsWith("o3-mini")) {
  return providerList.openai.chat(modelName);
}
return providerList[provider](modelName);

Any self-hosted setup using MODEL_NAME with a different name falls through to the Responses API and hits the empty-output path. There is no configuration to opt out.

Environment

• Firecrawl v2.9.0 (apps/api), self-hosted. Verified unchanged on v2.11.187.
• OPENAI_BASE_URL -> LiteLLM proxy -> llama.cpp (OpenAI-compatible).
• MODEL_NAME set to a local model.

What happens

Scraping 10 documentation pages with formats: ["json"] and a small schema: 5 returned HTTP 200 with the json field absent, 5 timed out at the default 60s scrape timeout. 0/10 usable. The same 10 pages with formats: ["markdown"] succeeded in 0.3-1.9s each.

Log sequence for the silent case:

Wrapping original schema with SmartScrape fields
Generating object...
AI SDK Warning (openai.responses / <model>): "specificationVersion" compatibility mode
Repairing text
Repaired text with string manipulation
warn [executeTransformers/coerceFieldsToFormats]: Request had format json,
     but there was no json field in the result

Root cause

I isolated this with direct calls to the same backend, same input, same strict: true, varying only the endpoint and the schema shape:

schema	/v1/responses	/v1/chat/completions
flat (title, summary)	```json fenced block	valid JSON
SmartScrape-wrapped	markdown prose (**Title:** ...)	valid JSON

Tested across three different models — identical behaviour in every cell, so this is not model quality.

On the Responses API the json_schema constraint stops being enforced once the schema contains union types (["string", "null"]) and nested additionalProperties: false; the backend falls back to unconstrained generation. prepareSmartScrapeSchema() in apps/api/src/scraper/scrapeURL/lib/extractSmartScrape.ts adds exactly those: the user schema is nested under extractedData alongside shouldUseSmartscrape (boolean) and two ["string", "null"] fields.

experimental_repairText then looks for a ```json fence, finds prose, the parse fails, extractedDataArray ends up empty and document.json is undefined — while the API still reports success: true.

Two suggestions

1. Make the Chat Completions fallback configurable rather than keyed on the o3-mini prefix — e.g. an env var to force providerList.openai.chat(). Every OpenAI-compatible backend without complete Responses API support hits this, and the current condition can only ever match one model family.

2. Consider skipping the SmartScrape wrapping when the feature cannot run. In v2.9.0 and v2.11.187, prepareSmartScrapeSchema and processSmartScrapeResult are both commented out at the call site (llmExtract.ts, around the "Prepare the schema, potentially wrapping it" block), so the schema is made harder to satisfy for a code path that is currently inert.

Separately: success: true with a missing requested format is easy to miss. Surfacing the existing coerceFieldsToFormats warning in the API response (it is already logged) would make this fail loudly instead of silently.

Workaround for others hitting this

Alias the model behind a name starting with o3-mini so getModel() picks the Chat Completions path.