Bug: Incorrect CORRECTNESS_PROMPT — missing format placeholders (user_question, report, answer)
Author: n-tateiwaCreated Nov 13, 2025Updated Feb 12, 2026
Hi team! Thank you for the great project.
I found that CORRECTNESS_PROMPT does not use user_question, report, or answer in tests/prompts.py:
However, the eval_correctness function in tests/evaluators.py formats the prompt with these fields:
As a result, format() has no effect and the evaluator receives a prompt that lacks the actual question, the generated report, and the authoritative answer.
Could you please confirm whether this is a bug?
Source: langchain-ai/open_deep_research