#14284·hyperswitch

[FEATURE]

Author: rohanm13Created Sep 17, 2026Updated Sep 17, 2026
LabelsC-featureS-awaiting-triage

Feature Description

Feature Description

Run competing revenue-recovery retry implementations side by side on live traffic, with a per-invoice sticky assignment, so we can measure which one recovers more revenue.

Today the implementation used inside the Smart arm is chosen by the revenue_recovery.adaptive_retry_enabled Superposition flag, re-read on every retry (crates/router/src/workflows/revenue_recovery.rs, Smart arm). Two consequences:

  1. No traffic split. Every invoice under a merchant gets the same implementation, so the two cannot be compared against each other.
  2. No stickiness. If the flag flips mid-recovery, an invoice switches implementation part-way through its retry chain — and because the flag is also read independently in EXECUTE, a single retry can be scheduled by one implementation and tokenised by the other.

What we want

A per-invoice assignment that is resolved once from Superposition (which owns the bucketing), persisted on the invoice, and replayed on every subsequent retry.

Why stickiness has to be explicit

Superposition's bucketing is hash((targeting_key, group_id)) % 100, and for user-created groups the toss is rescaled by the experiment's traffic percentage (superposition_core::experiment::get_applicable_buckets_from_group). So a ramp change re-buckets invoices that are already in flight. Recovery spans the grace window (default 30 days), which makes a ramp change during an invoice's life likely rather than exceptional. An invoice that flips arms mid-recovery contaminates both arms, so the assignment must be stored and replayed rather than re-derived.

Secondary reason: the bucket hash uses std::hash::DefaultHasher, whose algorithm std explicitly does not guarantee across releases, and it is computed in-process by the router — so a toolchain bump (or a rolling deploy across one) can also re-bucket.

Possible Implementation

Possible Implementation

1. Superposition config

Two keys in crates/router/src/consts.rs (mod superposition) with config! blocks in crates/router/src/core/configs/dimension_config.rs:

Key Type Default Targeting key
revenue_recovery.ab_enabled bool false none (plain gate)
revenue_recovery.ab_algorithm string enum GlobalPaymentId

Two keys rather than one, for two reasons:

  • A bool cannot encode three states (off / hybrid / decider) — false already means "not in A/B", so it cannot also mean "control arm".
  • The gate must not be overridable by an experiment variant. Variant overrides are themselves contexts keyed on variantIds (dimension position 0), and get_overrides merges matching contexts in server-delivered order — so a gate living inside the experiment-driven value could not be reliably forced off for a specific merchant.

Gate-off must be byte-identical to today's behaviour, so it doubles as the kill switch.

2. TargetingKey for GlobalPaymentId

Required in crates/common_utils/src/id_type/global_id/payment.rs. GlobalPaymentId is built by the global_id_type! macro, which does not generate this impl (unlike impl_id_type_methods!, which does — see GlobalCustomerId). Without it the config will not compile, and a blank identifier makes get_applicable_buckets_from_group return empty, so no experiment would ever apply.

3. Storage of the assignment

feature_metadata.payment_revenue_recovery_metadata.recovery_routing on the payment intent (Option<String>, #[serde(default)], in both api_models and diesel_models). No migration — feature_metadata is JSONB.

Intent rather than process_tracker.tracking_data, because:

  • PT rows are keyed one per (runner, task, invoice) and are finished on RetriesExhausted / HardDecline, then recreated by a later webhook — the assignment would reset.
  • tracking_data.revenue_recovery_retry is already re-seeded from the caller on every task upsert rather than carried forward, so anything added there inherits that fragility.
  • tracking_data is not joinable to payment_intent.status for the measurement query.

The convertors (ApiModelToDieselModelConvertor for PaymentRevenueRecoveryMetadata) must pass the field through in both directions, and PaymentIntent::get_updated_feature_metadata must carry the existing value forward — otherwise each new failed attempt rebuilds the metadata and wipes the assignment.

4. Flow inside the Smart arm

Fetch both keys at the top, then an if / else if / else chain:

rust
let adaptive_retry_enabled = dimensions.get_adaptive_retry_enabled(..).await;
let ab_enabled = dimensions.get_revenue_recovery_ab_enabled(..).await;

if ab_enabled {
    // replay the stored assignment; resolve + record only when absent or unrecognised
    let algorithm = match stored_algorithm {
        Some(a) => a,
        None => {
            let resolved = dimensions
                .get_revenue_recovery_ab_algorithm(.., Some(&payment_intent.id))
                .await;
            // record on the intent's feature metadata
            resolved
        }
    };
    // dispatch: hybrid or decider
} else if adaptive_retry_enabled {
    // existing hybrid path, unchanged
} else {
    // existing decider path, unchanged
}

A stored value that is absent or does not name a known implementation both fall through to the resolve — one parse().ok() covers both. The unrecognised case should log a warning, since that is the only way a silent stickiness violation (schema drift, hand-edited row) becomes observable.

Dispatching from the A/B branch requires the existing adaptive body to be callable from two places, so it needs extracting into a helper returning (PaymentProcessorTokenResponse, Option<StaticLadderProgress>). The decider side needs no change — it is already a single call to get_best_psp_token_available_for_smart_retry.

5. Persisting the assignment

payment_intent is threaded as &mut PaymentIntent through perform_calculate_workflow and get_token_with_schedule_time_based_on_retry_algorithm_type; the A/B block records the assignment on the in-memory feature metadata, and reset_connector_transmission_and_active_attempt_id_before_pushing_to_execute_workflow carries it to the database, because that function builds its update request from the same object.

Note this needs one borrow adjustment: active_payment_attempt_id is read both before and after the mutable call, so the later read must be re-derived rather than reusing the binding (otherwise the shared borrow spans the &mut call — E0502).

6. Superposition setup

  • POST /default-config for both keys
  • an experiment on revenue_recovery.ab_algorithm with two variants overriding it with "hybrid" and "decider"
  • ramp (the value is per-variant: 50 with 2 variants enrols 100% at a 50/50 split)
  • a USER_CREATED experiment group — the demo server returns experiment groups from the context-less /experiment-config fetch the provider polls only if at least one exists; without it A/B routing silently reverts to the default. Re-verify against the currently pinned superposition-demo image, as this was confirmed on an older version.

Have you spent some time checking if this feature request has been raised before?

  • I checked and didn't find a similar issue

Have you read the Contributing Guidelines?

Are you willing to submit a PR?

Yes, I am willing to submit a PR!