deepeval · Issues· 614 open

[TS] Published 0.9.13 ships a stale dist/telemetry.js that require()s the removed @sentry/node, every CLI command crashes

LangChain CallbackHandler implements on_llm_error/on_tool_error/on_retriever_error but not on_chain_error, so an errored chain span is never exited and its trace is never posted

EvaluationDataset.save_as('csv'/'jsonl') splits context items that contain '|' when reloaded

Add JudgeLM as a way to evaluate and compare historical test runs

KimiModel.__init__ raises TypeError float(None) whenever the selected model has no registry pricing

Verbose judge verdicts are scored as pass in role_violation, prompt_alignment, conversation_completeness

OpenAIModel rejects unknown catalog ids; LocalModel base_url is the Chat Completions path

HumanEval benchmark validates n/k with a bare assert and accepts invalid values

Add retail support evaluation example using synthetic customer support cases

fix(metrics): whitespace-only actual_output/expected_output pass the empty-param guard

Question: stable cache format for reading per-test-case metric scores

OpenInference TOOL spans drop tool record on non-dict tool args and leave tools_called output empty (Strands parity)

AgentCore TOOL spans drop gen_ai.tool.call.result and crash on non-dict tool args (Strands parity)

fix(scorer): string scorers treat whitespace-only predictions as non-empty

[First-time contributor] Looking for beginner-friendly issues to contribute