[Feature]: Automated Performance Regression Detection and Latency Analytics

Is there an existing feature request for this?

• I have searched the existing issues

Enhancement description

This enhancement introduces a robust Automated Performance Tracking and Regression Detection system into Keploy. It extends Keploy's capabilities beyond functional testing to include performance reliability by monitoring latency, identifying bottlenecks, and enforcing performance budgets during the test lifecycle.

---

Purpose (The Problem)

Currently, Keploy excels at validating functional correctness but lacks visibility into performance characteristics. This leads to several critical issues:

1. Silent Slowdowns: API endpoints can become significantly slower due to code changes (e.g., inefficient algorithms, N+1 queries) without failing functional tests.
2. Lack of Historical Context: Developers cannot easily compare the performance of the current PR against a stable baseline or previous releases.
3. Late Detection: Performance issues are often only discovered in staging or production environments, where they are more expensive to fix.
4. Opaque Latency: Without per-dependency timing, it's difficult to distinguish if a slowdown is in the application logic, the database, or an external gRPC service.

---

Proposed Solution

The solution transforms Keploy into a performance-aware testing tool through four core pillars:

1. Precision Multi-Layer Timing

Keploy will instrument the proxy and hooks to capture highly granular timing data:

• Total Duration: Wall-clock time for the entire request-response cycle.
• Handler Time: Time spent within the application code.
• Dependency Time: Breakdown of time spent in mocked dependencies (SQL, Mongo, Redis, gRPC).

2. Statistical Baseline Engine

Instead of simple threshold matching, Keploy will use a statistical approach:

• Historical Baselines: Maintains a moving window of performance data (P50, P95, P99) for every test case.
• Version-Aware Comparison: Automatically compares results against the main branch or the last "known good" version.
• Performance Budgets: Allows users to define hard limits (e.g., maxP95: 200ms) in the keploy.yaml config.

3. Analytics & Visualization

New CLI capabilities to expose performance insights:

• keploy performance report: Generates a detailed breakdown of latency per endpoint.
• keploy performance trends: Shows how performance has evolved over the last N days/commits.
• keploy performance compare: Direct A/B comparison between two branches or environments.

---

Implementation Plan (4-Phases )

Phase 1: Foundation & Data Capture

• Timing Instrumentation: Update pkg/models and pkg/service/replay to capture and propagate execution timings.
• Storage Layer: Implement a lightweight time-series storage (JSON/YAML based initially) to persist metrics in the .keploy directory.

Phase 2: Logic & Detection

• Baseline Engine: Develop the statistical analyzer to calculate P50/P95/P99 from historical data.
• Regression Algorithm: Implement the detection logic with configurable sensitivity (e.g., alert if current > baseline * 1.2).
• Alerting System: Add console-based alerting during test execution.

Phase 3: Tooling & Reporting

• CLI Integration: Implement the performance sub-command and its associated actions (report, trends).
• Formatters: Create beautiful table-based and summary outputs for the terminal.
• Comparison Logic: Add logic to pull baselines from different branches for PR-checks.

Phase 4: Integration & UX

• Config Support: Add performance-specific settings to keploy.yaml.
• CI/CD Optimization: Optimize the runner to fail or warn in CI based on performance budgets.
• Documentation: Comprehensive guides and example workflows.

---

Why should this be worked on?

1. Stop "Death by a Thousand Cuts": Prevents the gradual degradation of API performance that accumulates over many small PRs.
2. Drastic Reduction in MTTR: Breakdown timing allows developers to immediately pinpoint where the slowdown is (Code vs. DB).
3. Shift-Left Strategy: Brings performance testing into the daily development loop, making it a "day zero" concern rather than a "day ninety" task.
4. Zero-Configuration Value: Unlike JMeter or k6 which require separate test scripts, Keploy provides performance insights for existing functional tests automatically.
5. Enterprise Readiness: Provides the technical oversight required for high-scale microservices where performance stability is as critical as functional correctness.

Repository

keploy