Add optional web_search built-in tool to chat (opt-in)

Author: pamelafoxCreated Jun 26, 2026Updated Jun 26, 2026

Summary

Add an opt-in setting that attaches the OpenAI Responses web_search built-in tool to chat answer calls, so the model can ground answers with real-time public web results when the retrieved documents aren't enough.

Docs: Web search with the Responses API

Motivation

Some questions need fresh, public-web information (recent events, current facts) that isn't in the indexed corpus. Wiring web_search as an optional, off-by-default tool adds capability without changing how answers are normally generated — so existing answer-quality evals remain valid.

Proposed implementation

Follow the existing optional-override pattern used by use_web_source (read an override flag, branch on it, surface the effect in the thought process). This change is self-contained and does not depend on any other in-flight work.

  • Backend (app/backend/approaches/chatreadretrieveread.py):
    • Read use_web_search = bool(overrides.get("use_web_search")) in run_until_final_call (near where the other override flags are read).
    • Build a tools list for the final answer create_response() call and, when use_web_search is true, append {"type": "web_search"}. Pass tools=tools if tools else None to create_response().
    • create_response() in approach.py already forwards a tools argument to responses.create — no change needed there. (If it does not, add an optional tools parameter that is forwarded.)
  • Thought process visibility (required):
    • After the answer call, inspect the response output for web_search_call items (per the docs: any(item.type == "web_search_call" for item in response.output)) and surface them as a ThoughtStep so each web search the model performs appears in the Thought Process panel of the frontend (queries issued and/or sources used). This makes the tool's behavior observable.
  • System prompt update (app/backend/approaches/prompts/chat_answer.system.jinja2):
    • When use_web_search is enabled, conditionally replace the strict "Answer ONLY with the facts listed in the list of sources below" instruction with a softer version that tells the model to prefer sources but use web_search when sources are insufficient. Without this, the model will follow the "ONLY" directive and never invoke the tool.
  • Frontend:
    • Add use_web_search?: boolean to the overrides in api/models.ts.
    • Add a toggle in components/Settings/Settings.tsx, wired through pages/chat/Chat.tsx like the other override checkboxes (mirror how use_web_source is wired).
    • Add labels to locales/*/translation.json.
  • Docs: note the setting in the relevant docs/ customization page.

Tool declaration

Per the docs, web search is enabled by declaring the tool in the request:

python
response = openai.responses.create(
    model="gpt-5.5",
    tools=[{"type": "web_search"}],
    input="...",
)

Use web_search (not the deprecated web_search_preview). Works with GPT-4 models and later.

Scope / out of scope

  • In scope: opt-in toggle, backend tool wiring, thought-process visibility of web_search calls.
  • Out of scope: changing default answer-generation prompts; enabling by default; deep-research / agentic multi-step modes.

Acceptance criteria

  • Toggle in Settings; off by default.
  • When on, web_search is attached to the answer call and the model can search the public web.
  • Each web_search_call is surfaced as a ThoughtStep and visible in the Thought Process panel.
  • When off, request payload is unchanged (no eval impact); existing tests pass.

Testing tips

After starting the app locally (./app/start.sh or via the Development task), you can verify the feature end-to-end with curl:

bash
# With web_search enabled — model should use web search and return a real answer
curl -s -X POST http://localhost:50505/chat \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"content": "What is the current population of Tokyo?", "role": "user"}], "context": {"overrides": {"retrieval_mode": "text", "use_web_search": true}}}' \
  | python -c "
import sys, json
r = json.load(sys.stdin)
thoughts = r.get('context', {}).get('thoughts', [])
print('OUTPUT:', r.get('output_text', '')[:300])
print()
print('THOUGHT TITLES:', [t['title'] for t in thoughts])
for t in thoughts:
    if t['title'] == 'Web search':
        print('WEB SEARCH:', t['description'])
        print('PROPS:', t.get('props'))
"

Expected: OUTPUT contains a factual answer with a web URL citation; THOUGHT TITLES includes 'Web search'.

bash
# Without web_search (default) — model should say "I don't know"
curl -s -X POST http://localhost:50505/chat \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"content": "What is the current population of Tokyo?", "role": "user"}], "context": {"overrides": {"retrieval_mode": "text"}}}' \
  | python -c "import sys,json; r=json.load(sys.stdin); print(r.get('output_text','')[:200])"

Expected: I don't know. (no web search fallback, existing behavior preserved).

Source: Azure-Samples/azure-search-openai-demo