StructuredOutputMode is never sent by any SDK; validationOnly docs are stale
Follow-up from #840 by @ayaangazali -- thanks again for that PR!
What
bindings/flutter/packages/runanywhere/lib/public/api/namespaces/llm.dart:118 documents the default validationOnly mode as "generate freely, then validate," but no SDK (Flutter, Swift Options.swift:101, Kotlin MappingOptions.kt:122-126) ever sends StructuredOutputOptions.mode. Commons treats an unset mode as constrained generation: core/src/features/llm/llm_module.cpp:1644-1647 sets constrain = mode != VALIDATION_ONLY when mode is unspecified, then compiles the schema to GBNF (llm_module.cpp:1651-1653) and engines/llamacpp/rac_llm_llamacpp.cpp:99-100 forwards it into llama_sampler_init_grammar. A stale comment in structured_output.cpp:598-609 still claims grammar isn't wired up.
Why it matters
The documented default behavior (unconstrained generation, validated after the fact) doesn't match what actually happens (constrained generation via grammar) for every SDK today, which is confusing for anyone reading the docs or debugging why output is more constrained than expected.
Suggested approach
Either map StructuredOutputMode from each SDK's public options into StructuredOutputOptions.mode (validationOnly -> VALIDATION_ONLY, repair -> REPAIR), or correct the docs to describe the real default, and remove the stale "grammar not wired" comment in structured_output.cpp.
Done when
- Docs and code agree on what the default structured-output mode actually does.
- Either mode is mapped from at least one SDK's public API, or the mode-related docs/comments are corrected to match current behavior.
Not blocking #840. @ayaangazali, you know this code well now -- you're welcome to take this one if you're interested.
Opened with help from Claude Code and Codex.
Reviewed with help from Claude Code and Codex.
Source: RunanywhereAI/runanywhere-sdks