Add retail support evaluation example using synthetic customer support cases
Hi DeepEval team,
I am posting from hienhtphan, admin of the SData-Pro GitHub organisation, and would submit any PR from an SData-Pro/deepeval fork.
I would like to contribute a small retail support evaluation example using synthetic customer support cases. The example would show how to use DeepEval to evaluate support-style LLM responses with LLMTestCase, GEval, FaithfulnessMetric, and AnswerRelevancyMetric.
The sample would use a tiny inline synthetic retail support dataset with cases such as order status, returns, refunds, delivery exceptions, substitutions, loyalty points, invoice requests, and address changes. It would reuse the same 12-row synthetic retail support sample already prepared for our Langfuse and OpenEvals contribution planning, adapted only to DeepEval's API shape.
The goal is to demonstrate a copy-pasteable DeepEval workflow for support-style LLM applications:
- policy-aligned correctness with
GEval - grounding against provided retail context with
FaithfulnessMetric - answer relevance with
AnswerRelevancyMetric - deterministic checks for forbidden commitments
- optional CI-style execution with
deepeval test run
The example would avoid external/private data files, customer data, warehouse-specific setup, platform login requirements, and vendor-specific product placement. The sample data would be synthetic and public-safe; it would not be presented as a benchmark or production evaluator.
Would this be a useful contribution?
If so, where would you prefer it to live: a Python example file, docs page, README section, notebook, or another examples location?
Do you prefer examples to use deepeval test run/pytest style, or the programmatic evaluate(...) APII?
Source: confident-ai/deepeval