Add optional web_search built-in tool to chat (opt-in)
Summary
Add an opt-in setting that attaches the OpenAI Responses web_search built-in tool to chat answer calls, so the model can ground answers with real-time public web results when the retrieved documents aren't enough.
Docs: Web search with the Responses API
Motivation
Some questions need fresh, public-web information (recent events, current facts) that isn't in the indexed corpus. Wiring web_search as an optional, off-by-default tool adds capability without changing how answers are normally generated — so existing answer-quality evals remain valid.
Proposed implementation
Follow the existing optional-override pattern used by use_web_source (read an override flag, branch on it, surface the effect in the thought process). This change is self-contained and does not depend on any other in-flight work.
- Backend (
app/backend/approaches/chatreadretrieveread.py):- Read
use_web_search = bool(overrides.get("use_web_search"))inrun_until_final_call(near where the other override flags are read). - Build a
toolslist for the final answercreate_response()call and, whenuse_web_searchis true, append{"type": "web_search"}. Passtools=tools if tools else Nonetocreate_response(). create_response()inapproach.pyalready forwards atoolsargument toresponses.create— no change needed there. (If it does not, add an optionaltoolsparameter that is forwarded.)
- Read
- Thought process visibility (required):
- After the answer call, inspect the response output for
web_search_callitems (per the docs:any(item.type == "web_search_call" for item in response.output)) and surface them as aThoughtStepso each web search the model performs appears in the Thought Process panel of the frontend (queries issued and/or sources used). This makes the tool's behavior observable.
- After the answer call, inspect the response output for
- System prompt update (
app/backend/approaches/prompts/chat_answer.system.jinja2):- When
use_web_searchis enabled, conditionally replace the strict "Answer ONLY with the facts listed in the list of sources below" instruction with a softer version that tells the model to prefer sources but useweb_searchwhen sources are insufficient. Without this, the model will follow the "ONLY" directive and never invoke the tool.
- When
- Frontend:
- Add
use_web_search?: booleanto the overrides inapi/models.ts. - Add a toggle in
components/Settings/Settings.tsx, wired throughpages/chat/Chat.tsxlike the other override checkboxes (mirror howuse_web_sourceis wired). - Add labels to
locales/*/translation.json.
- Add
- Docs: note the setting in the relevant
docs/customization page.
Tool declaration
Per the docs, web search is enabled by declaring the tool in the request:
response = openai.responses.create(
model="gpt-5.5",
tools=[{"type": "web_search"}],
input="...",
)Use web_search (not the deprecated web_search_preview). Works with GPT-4 models and later.
Scope / out of scope
- In scope: opt-in toggle, backend tool wiring, thought-process visibility of web_search calls.
- Out of scope: changing default answer-generation prompts; enabling by default; deep-research / agentic multi-step modes.
Acceptance criteria
- Toggle in Settings; off by default.
- When on,
web_searchis attached to the answer call and the model can search the public web. - Each
web_search_callis surfaced as a ThoughtStep and visible in the Thought Process panel. - When off, request payload is unchanged (no eval impact); existing tests pass.
Testing tips
After starting the app locally (./app/start.sh or via the Development task), you can verify the feature end-to-end with curl:
# With web_search enabled — model should use web search and return a real answer
curl -s -X POST http://localhost:50505/chat \
-H "Content-Type: application/json" \
-d '{"messages": [{"content": "What is the current population of Tokyo?", "role": "user"}], "context": {"overrides": {"retrieval_mode": "text", "use_web_search": true}}}' \
| python -c "
import sys, json
r = json.load(sys.stdin)
thoughts = r.get('context', {}).get('thoughts', [])
print('OUTPUT:', r.get('output_text', '')[:300])
print()
print('THOUGHT TITLES:', [t['title'] for t in thoughts])
for t in thoughts:
if t['title'] == 'Web search':
print('WEB SEARCH:', t['description'])
print('PROPS:', t.get('props'))
"Expected: OUTPUT contains a factual answer with a web URL citation; THOUGHT TITLES includes 'Web search'.
# Without web_search (default) — model should say "I don't know"
curl -s -X POST http://localhost:50505/chat \
-H "Content-Type: application/json" \
-d '{"messages": [{"content": "What is the current population of Tokyo?", "role": "user"}], "context": {"overrides": {"retrieval_mode": "text"}}}' \
| python -c "import sys,json; r=json.load(sys.stdin); print(r.get('output_text','')[:200])"Expected: I don't know. (no web search fallback, existing behavior preserved).
Source: Azure-Samples/azure-search-openai-demo