[Bug] keploy diff ignores newly added and deleted test cases when comparing test runs[Bug] keploy diff ignores newly added and deleted test cases when comparing test runs
Author: yush-1018Created Sep 13, 2026Updated Sep 18, 2026
Labelsbugkeploy
Is there an existing issue for this?
- I have searched and didn't find similar issue
Current behavior
When comparing two test runs using keploy diff <run1> <run2>, the comparison logic in pkg/service/diff/diff.go (ComputeDiff()) only considers test case IDs that exist in both test reports (commonIDs).
If a test case was added in run2 (present in run2 but absent in run1) or removed in run2 (present in run1 but absent in run2), it is completely omitted from the comparison result. It is neither reported under regressions, fixes, status transitions, nor unchanged test cases.
What I expect is:
keploy diff should detect and report test cases that were added or removed between test runs (e.g., under status transitions or added/deleted categories).
Steps to Replicate
- Record a test run
test-run-0with test casestc-1andtc-2. - Add a new test case
tc-3and recordtest-run-1(sotest-run-1hastc-1,tc-2,tc-3). - Run
keploy diff test-run-0 test-run-1. - Observe that
tc-3is completely ignored in the diff summary.
Version
v3-dev (main branch)
Repository
keploy
Source: keploy/keploy