Enable `catalog.prompt()`-generated system prompts support for smaller context window sizes

Author: howlowckCreated Aug 19, 2026Updated Aug 19, 2026

The system prompt generated by catalog.prompt() is approximately 4,300 tokens for our catalog of 18 components.

This creates problems for small language models and models running locally on browsers, mobile devices, and other edge environments. Many of these models support relatively small context windows (at most ~4k tokens). A 4,300-token system prompt, alone, cannot fit inside this context window.

For now, we manually reduced the system prompt to approximately 2,000 tokens, so json-render can work on a SLM.

Desired outcome

Provide a way in PromptOptions to enable the caller of catalog.prompt() to generate a much smaller system prompt.

Implementation Ideas:

  • size option: a rough t-shirt size (sm, md, lg) for high level prompt length control, (where sm targets the prompt to ~1k tokens, md 2k tokens, and lg the current size)
  • sections options + size option: grandular control over the specificity at each section
  • template option (with sections + size hooks): allow user to add a handlebars-like template with preset helpers for sections. This is the most robust and flexible but could potentially introduce a new dependency.