Add Guardrail Evaluation Harness for Tool-Calling Agents
Summary
Add an evaluation harness that tests tool-calling agents against guardrail, safety, and regression scenarios.
Proposed location
advance_ai_agents/guardrail_eval_harness/
What to build
Create a runnable evaluation workflow with a small tool-calling agent, mock tools, and adversarial test cases. The harness should evaluate unsafe tool calls, prompt injection, invalid structured outputs, refusal failures, and incorrect tool selection.
Use tools such as DeepEval, Guardrails AI, and pytest where appropriate. Keep offline tests runnable with mocks, and document any optional live model evaluation separately using the repo's existing model-provider configuration pattern.
Acceptance criteria
- Includes a
README.mdbased on.github/README_TEMPLATE.md. - Includes 2-3 safe mock tools for the agent to choose from.
- Includes adversarial and normal evaluation cases.
- Produces clear pass/fail output for tool choice, refusal behavior, prompt injection resistance, and structured output validity.
- Offline tests run without requiring external paid API calls.
- Documents optional live evaluation configuration separately.
- Adds one catalog entry to the root
README.mdunder Advanced AI Agents.
Contribution notes
Please keep this to one self-contained project and one pull request. Link the pull request with Closes #<issue-number>.
Source: Arindam200/awesome-ai-apps