[FEATURE]
Feature Description
Feature Description
Run competing revenue-recovery retry implementations side by side on live traffic, with a per-invoice sticky assignment, so we can measure which one recovers more revenue.
Today the implementation used inside the Smart arm is chosen by the
revenue_recovery.adaptive_retry_enabled Superposition flag, re-read on every retry
(crates/router/src/workflows/revenue_recovery.rs, Smart arm). Two consequences:
- No traffic split. Every invoice under a merchant gets the same implementation, so the two cannot be compared against each other.
- No stickiness. If the flag flips mid-recovery, an invoice switches implementation part-way through its retry chain — and because the flag is also read independently in EXECUTE, a single retry can be scheduled by one implementation and tokenised by the other.
What we want
A per-invoice assignment that is resolved once from Superposition (which owns the bucketing), persisted on the invoice, and replayed on every subsequent retry.
Why stickiness has to be explicit
Superposition's bucketing is hash((targeting_key, group_id)) % 100, and for user-created
groups the toss is rescaled by the experiment's traffic percentage
(superposition_core::experiment::get_applicable_buckets_from_group). So a ramp change
re-buckets invoices that are already in flight. Recovery spans the grace window (default 30
days), which makes a ramp change during an invoice's life likely rather than exceptional. An
invoice that flips arms mid-recovery contaminates both arms, so the assignment must be stored
and replayed rather than re-derived.
Secondary reason: the bucket hash uses std::hash::DefaultHasher, whose algorithm std
explicitly does not guarantee across releases, and it is computed in-process by the router — so
a toolchain bump (or a rolling deploy across one) can also re-bucket.
Possible Implementation
Possible Implementation
1. Superposition config
Two keys in crates/router/src/consts.rs (mod superposition) with config! blocks in
crates/router/src/core/configs/dimension_config.rs:
| Key | Type | Default | Targeting key |
|---|---|---|---|
revenue_recovery.ab_enabled |
bool |
false |
none (plain gate) |
revenue_recovery.ab_algorithm |
string enum | — | GlobalPaymentId |
Two keys rather than one, for two reasons:
- A bool cannot encode three states (
off/hybrid/decider) —falsealready means "not in A/B", so it cannot also mean "control arm". - The gate must not be overridable by an experiment variant. Variant overrides are themselves
contexts keyed on
variantIds(dimension position 0), andget_overridesmerges matching contexts in server-delivered order — so a gate living inside the experiment-driven value could not be reliably forced off for a specific merchant.
Gate-off must be byte-identical to today's behaviour, so it doubles as the kill switch.
2. TargetingKey for GlobalPaymentId
Required in crates/common_utils/src/id_type/global_id/payment.rs. GlobalPaymentId is built
by the global_id_type! macro, which does not generate this impl (unlike
impl_id_type_methods!, which does — see GlobalCustomerId). Without it the config will not
compile, and a blank identifier makes get_applicable_buckets_from_group return empty, so no
experiment would ever apply.
3. Storage of the assignment
feature_metadata.payment_revenue_recovery_metadata.recovery_routing on the payment intent
(Option<String>, #[serde(default)], in both api_models and diesel_models). No migration
— feature_metadata is JSONB.
Intent rather than process_tracker.tracking_data, because:
- PT rows are keyed one per
(runner, task, invoice)and are finished onRetriesExhausted/HardDecline, then recreated by a later webhook — the assignment would reset. tracking_data.revenue_recovery_retryis already re-seeded from the caller on every task upsert rather than carried forward, so anything added there inherits that fragility.tracking_datais not joinable topayment_intent.statusfor the measurement query.
The convertors (ApiModelToDieselModelConvertor for PaymentRevenueRecoveryMetadata) must
pass the field through in both directions, and PaymentIntent::get_updated_feature_metadata
must carry the existing value forward — otherwise each new failed attempt rebuilds the metadata
and wipes the assignment.
4. Flow inside the Smart arm
Fetch both keys at the top, then an if / else if / else chain:
let adaptive_retry_enabled = dimensions.get_adaptive_retry_enabled(..).await;
let ab_enabled = dimensions.get_revenue_recovery_ab_enabled(..).await;
if ab_enabled {
// replay the stored assignment; resolve + record only when absent or unrecognised
let algorithm = match stored_algorithm {
Some(a) => a,
None => {
let resolved = dimensions
.get_revenue_recovery_ab_algorithm(.., Some(&payment_intent.id))
.await;
// record on the intent's feature metadata
resolved
}
};
// dispatch: hybrid or decider
} else if adaptive_retry_enabled {
// existing hybrid path, unchanged
} else {
// existing decider path, unchanged
}A stored value that is absent or does not name a known implementation both fall through to
the resolve — one parse().ok() covers both. The unrecognised case should log a warning, since
that is the only way a silent stickiness violation (schema drift, hand-edited row) becomes
observable.
Dispatching from the A/B branch requires the existing adaptive body to be callable from two
places, so it needs extracting into a helper returning
(PaymentProcessorTokenResponse, Option<StaticLadderProgress>). The decider side needs no
change — it is already a single call to get_best_psp_token_available_for_smart_retry.
5. Persisting the assignment
payment_intent is threaded as &mut PaymentIntent through perform_calculate_workflow and
get_token_with_schedule_time_based_on_retry_algorithm_type; the A/B block records the
assignment on the in-memory feature metadata, and
reset_connector_transmission_and_active_attempt_id_before_pushing_to_execute_workflow carries
it to the database, because that function builds its update request from the same object.
Note this needs one borrow adjustment: active_payment_attempt_id is read both before and
after the mutable call, so the later read must be re-derived rather than reusing the binding
(otherwise the shared borrow spans the &mut call — E0502).
6. Superposition setup
POST /default-configfor both keys- an experiment on
revenue_recovery.ab_algorithmwith two variants overriding it with"hybrid"and"decider" - ramp (the value is per-variant: 50 with 2 variants enrols 100% at a 50/50 split)
- a
USER_CREATEDexperiment group — the demo server returns experiment groups from the context-less/experiment-configfetch the provider polls only if at least one exists; without it A/B routing silently reverts to the default. Re-verify against the currently pinnedsuperposition-demoimage, as this was confirmed on an older version.
Have you spent some time checking if this feature request has been raised before?
- I checked and didn't find a similar issue
Have you read the Contributing Guidelines?
- I have read the Contributing Guidelines
Are you willing to submit a PR?
Yes, I am willing to submit a PR!
Source: juspay/hyperswitch