#2748·deepeval

Add retail support evaluation example using synthetic customer support cases

Author: hienhtphanCreated Jun 13, 2026Updated Sep 14, 2026

Hi DeepEval team,

I am posting from hienhtphan, admin of the SData-Pro GitHub organisation, and would submit any PR from an SData-Pro/deepeval fork.

I would like to contribute a small retail support evaluation example using synthetic customer support cases. The example would show how to use DeepEval to evaluate support-style LLM responses with LLMTestCase, GEval, FaithfulnessMetric, and AnswerRelevancyMetric.

The sample would use a tiny inline synthetic retail support dataset with cases such as order status, returns, refunds, delivery exceptions, substitutions, loyalty points, invoice requests, and address changes. It would reuse the same 12-row synthetic retail support sample already prepared for our Langfuse and OpenEvals contribution planning, adapted only to DeepEval's API shape.

The goal is to demonstrate a copy-pasteable DeepEval workflow for support-style LLM applications:

  • policy-aligned correctness with GEval
  • grounding against provided retail context with FaithfulnessMetric
  • answer relevance with AnswerRelevancyMetric
  • deterministic checks for forbidden commitments
  • optional CI-style execution with deepeval test run

The example would avoid external/private data files, customer data, warehouse-specific setup, platform login requirements, and vendor-specific product placement. The sample data would be synthetic and public-safe; it would not be presented as a benchmark or production evaluator.

Would this be a useful contribution?

If so, where would you prefer it to live: a Python example file, docs page, README section, notebook, or another examples location?

Do you prefer examples to use deepeval test run/pytest style, or the programmatic evaluate(...) APII?