watchTradesForSymbols over-delivers repeated trades on multi-symbol subscriptions

Operating System

Windows 11 x64

Programming Language

C#

CCXT Version

4.5.77

Description

Affected file(s)

• ts/src/base/ws/Cache.ts (ArrayCache.getLimit)
• Every exchange's watchTradesForSymbols implementation that resolves limit via trades.getLimit(<symbol of trades[0]>, limit), for example ts/src/pro/binance.ts, ts/src/pro/okx.ts, ts/src/pro/bybit.ts, ts/src/pro/bitmex.ts, ts/src/pro/aster.ts, ts/src/pro/bitget.ts, ts/src/pro/upbit.ts, ts/src/pro/coinex.ts, and others implementing the same pattern.

Steps to reproduce

1. Call watchTradesForSymbols([...manySymbols]) on any exchange that subscribes many symbols onto one shared ArrayCache trades cache.
2. Let trades arrive continuously for all subscribed symbols for several minutes.
3. Compare the number of trades returned across repeated resolutions of the same watchTradesForSymbols call against the number of genuinely new trades observed on the exchange for that period.

Expected behavior

Each call to watchTradesForSymbols should return only the trades that are new since the caller's previous resolution, across all subscribed symbols, with no repeats.

Root cause

watchTradesForSymbols resolves its return-size limit as:

if (this.newUpdates) {
    const first = this.safeValue (trades, 0);
    const tradeSymbol = this.safeString (first, 'symbol');
    limit = trades.getLimit (tradeSymbol, limit);
}
return this.filterBySinceLimit (trades, since, limit, 'timestamp', true);

trades is one ArrayCache shared by every symbol the caller subscribed to. ArrayCache.getLimit(symbol, limit) reads and clears the new-update counter for only the symbol of trades[0] (an arbitrary, typically stale row — whatever currently happens to sit at index 0 of the shared cache). Every other subscribed symbol's newUpdatesBySymbol[symbol] counter keeps incrementing on every append() and is never cleared, because nothing ever calls getLimit with those symbols. The counter for those symbols grows without bound across many resolutions, so the limit used to tail-slice the shared array keeps growing, and previously-returned trades for those symbols are included again on the next call. The amplification factor tracks how unevenly getLimit happens to land on one symbol versus the rest of the subscribed set.

Suggested fix

Resolve the multi-symbol case against the cache's global/all-symbols scope instead of a single symbol's scope, e.g.:

limit = trades.getLimit (undefined, limit);

ArrayCache already tracks a global allNewUpdates counter (incremented on every append() regardless of symbol, cleared only when getLimit(undefined, ...) is called) that is independent of any single symbol's counter, so this correctly returns "every trade new since this caller's last resolution" across the whole subscribed set with no unbounded growth. This is a one-line change per affected watchTradesForSymbols implementation.

Related

Already reported this under https://github.com/ccxt/ccxt/issues/29667 but that issue covers only Lighter exchange so decided to open a new issue as this affects multiple exchanges.