Monte Carlo (trades): calmar_ratio uses total return while the 'original' column uses CAGR, making the two incomparable

Summary

In the trades-based Monte Carlo, the calmar_ratio shown in the original column and the one shown in the worst_5 / median / best_5 columns are computed with two different formulas. They end up side by side in the same summary row, so the table reads as if reshuffling the trade order improves Calmar by an order of magnitude.

Version: jesse 2.5.0.

Where

The backtest metric is annualized (CAGR / MaxDD) — jesse/services/metrics.py:120:

def calmar_ratio(returns):
    """
    Calculates the calmar ratio (CAGR% / MaxDD%)
    """

The Monte Carlo scenario metric uses cumulative total return instead — jesse/research/monte_carlo/monte_carlo_trades.py:317:

calmar_ratio = total_return / abs(max_drawdown) if max_drawdown < 0 else 0

The original column does not go through that function. At monte_carlo_trades.py:398, calmar_ratio falls into the else branch and is taken straight from the backtest metrics:

original_value = original_metrics.get(metric_name, 0)

So original is CAGR-based while every percentile is total-return-based.

Reproduction

Strategy on Binance Perpetual Futures BTC-USDT 1D, 2020-06-01 → 2026-07-29, 500 trades scenarios. Reported summary:

The percentile values are exactly total_return / abs(max_drawdown):

• 1272.17 / 21.18 = 60.06 → reported median 60.07
• 1272.17 / 68.07 = 18.69 → reported worst_5 18.69
• 1272.17 / 7.88 = 161.44 → reported best_5 161.39

Over this ~6-year window the inflation is roughly 24× relative to the original value.

Why it matters

1. The summary table is misleading. A reader compares original 2.51 against median 60.07 and concludes the historical ordering was extraordinarily unlucky. The two numbers are not comparable at all.

2. The Calmar p-value is always 1.0. At monte_carlo_trades.py:414:

if metric_name in ['total_return', 'sharpe_ratio', 'calmar_ratio']:
    p_value = np.sum(values_array >= original_value) / len(values_array)

Every scenario Calmar is on the inflated scale, so it always exceeds the CAGR-based original. The test can never reject, for any strategy.

1. The metric carries no information in this mode. Reshuffling trade order does not change the compounded final equity, so total_return is constant across scenarios (visible in the table above). The numerator is fixed, which makes the scenario Calmar a pure restatement of drawdown rather than an independent metric.

Suggested fix

Compute the scenario Calmar the same way metrics.py does — annualize the return over the scenario's own time span before dividing by max drawdown — so original and the percentiles share a definition. Alternatively, derive the original value with the scenario formula so both columns are at least internally consistent.

Possibly related: sharpe_ratio in the same mode

_calculate_volatility_metrics (monte_carlo_trades.py:360) annualizes with ANNUALIZATION_FACTOR = 365 (common.py:19, commented "Trading days per year"):

annualized_volatility = daily_std * np.sqrt(ANNUALIZATION_FACTOR)

But the equity curve it consumes is rebuilt from trades (_reconstruct_equity_curve_from_trades), so each consecutive pair is one trade, not one day. In the run above that is 64 points across ~6 years — average spacing ~35 days — while the factor assumes daily spacing. The resulting Sharpe is not on the same scale as the backtest's original either (median 0.76 vs original 1.71 here).

I have not verified this one as thoroughly as the Calmar issue, so I am flagging it rather than asserting it. Happy to open it separately if you prefer.