Add Guardrail Evaluation Harness for Tool-Calling Agents

Author: AstrodevilCreated Sep 7, 2026Updated Sep 16, 2026

Summary

Add an evaluation harness that tests tool-calling agents against guardrail, safety, and regression scenarios.

Proposed location

advance_ai_agents/guardrail_eval_harness/

What to build

Create a runnable evaluation workflow with a small tool-calling agent, mock tools, and adversarial test cases. The harness should evaluate unsafe tool calls, prompt injection, invalid structured outputs, refusal failures, and incorrect tool selection.

Use tools such as DeepEval, Guardrails AI, and pytest where appropriate. Keep offline tests runnable with mocks, and document any optional live model evaluation separately using the repo's existing model-provider configuration pattern.

Acceptance criteria

  • Includes a README.md based on .github/README_TEMPLATE.md.
  • Includes 2-3 safe mock tools for the agent to choose from.
  • Includes adversarial and normal evaluation cases.
  • Produces clear pass/fail output for tool choice, refusal behavior, prompt injection resistance, and structured output validity.
  • Offline tests run without requiring external paid API calls.
  • Documents optional live evaluation configuration separately.
  • Adds one catalog entry to the root README.md under Advanced AI Agents.

Contribution notes

Please keep this to one self-contained project and one pull request. Link the pull request with Closes #<issue-number>.

Source: Arindam200/awesome-ai-apps