[BUG] Sessions go "deaf" after 30+ minutes — connection alive, messages.upsert stops firing

Sessions go "deaf" after 30+ minutes — connection alive, messages.upsert stops firing Environment

Baileys version: 7.0.0-rc.9 Node.js: v20.x Auth store: Redis (via makeCacheableSignalKeyStore) Scale: 30+ concurrent sessions across 8 Node.js processes (distributed mode) History sync: disabled (syncFullHistory: false)

Description After running for 30+ minutes (sometimes up to a few hours), sessions silently stop receiving messages. The WebSocket connection remains open, keepAlive pings succeed, connection.update never fires close, and the session appears perfectly healthy — but messages.upsert stops being emitted entirely. We call this "deaf sessions." The session can still send messages (when defaultQueryTimeoutMs is set properly), but no incoming messages arrive. Restarting the process or forcing ws.close() and reconnecting fixes it temporarily, but it happens again after some time. What we've tried so far

1. Patched missing sendMessageAck in handleMessage else-branch We found that in messages-recv.ts, inside handleMessage, there was a code path where sendMessageAck(node) was not being called. We patched this in messages-recv.js (compiled output) at line ~1032. This was the else branch of the main decryption result check. Result: Reduced frequency of deaf sessions but did NOT eliminate them.
2. Fixed defaultQueryTimeoutMs: undefined We had defaultQueryTimeoutMs: undefined in our config. We traced through generics.ts:154 and found that promiseTimeout(undefined, ...) skips timeout enforcement entirely (if (!ms) { return new Promise(promise) }). This caused assertSessions() queries (called inside authState.keys.transaction() during relayMessage) to hang forever, which locked the transaction mutex and blocked all sends. Changed to defaultQueryTimeoutMs: 60000. Result: Fixed the "can receive but cannot send" issue. Did NOT fix deaf sessions.
3. Changed fireInitQueries: false → true We had fireInitQueries: false. This skipped initial blocklist/privacy/app-state queries after connection. WhatsApp apparently expects these and was terminating connections after ~45 seconds (status 428). Result: Fixed the 428 reconnection loop. Did NOT fix deaf sessions.
4. Changed keepAliveIntervalMs: 45000 → 25000 WhatsApp expects pings every ~30s. At 45s, the server would sometimes terminate the connection before the next ping. Result: Reduced disconnections. Did NOT fix deaf sessions.
5. Changed markOnlineOnConnect: false → true Result: No observable change for deaf sessions.
6. Reduced transactionOpts from {maxCommitRetries: 10, delayBetweenTriesMs: 3000} to {3, 500} The original config could hold the transaction mutex for up to 30 seconds on commit failures. Reduced to max 1.5s. Result: Improved send latency. Did NOT fix deaf sessions. Root cause analysis After reading the Baileys source code in depth, we believe the root cause is the messageMutex + ACK timing pattern in handleMessage (messages-recv.ts:1153-1383): The problem

Every incoming message enters messageMutex.mutex() at line 1194 Inside the mutex: decrypt() → keys.transaction() → cacheMutex (global, single mutex in makeCacheableSignalKeyStore) → store.get/set() (Redis in our case) The ACK (sendReceipt or sendMessageAck) is sent inside the mutex, at lines 1356/1366 If any step inside the mutex is slow or hangs (Redis latency spike, cacheMutex contention, stuck transaction), the ACK is delayed WhatsApp has server-side flow control: if it doesn't receive ACKs within a certain window, it stops delivering messages to that client But keepAlive pings still work (they don't go through the mutex) → connection appears alive → deaf session

Additional risk: retryMutex inside messageMutex When decryption fails (CIPHERTEXT stub), the code enters retryMutex.mutex() at line 1290 — while still holding messageMutex. Inside retryMutex:

uploadPreKeys(5) — network call, can be slow delay(1000) — hardcoded 1 second wait sendRetryRequest(node, ...) — network call delay(retryRequestDelayMs) — another wait (2000ms in our config)

If any of these hang, both retryMutex AND messageMutex are blocked. No incoming messages can be processed. No ACKs are sent. Session goes deaf. The mutex has no timeout make-mutex.ts uses async-mutex's runExclusive() with no timeout parameter: typescript// make-mutex.ts mutex(code: () => Promise | T): Promise { return mutex.runExclusive(code) // no timeout } If the task inside never resolves, the mutex is locked forever. Proposed solutions Option A: Send ACK before entering messageMutex Move sendMessageAck(node) to execute immediately when handleMessage is called, before messageMutex.mutex(). This way WhatsApp always gets its ACK regardless of how long message processing takes. typescriptconst handleMessage = async (node: BinaryNode) => { // ... shouldIgnoreJid check ...

// Send ACK immediately, before any heavy processing
await sendMessageAck(node)

try {
    const { fullMessage: msg, decrypt } = decryptMessageNode(...)
    await messageMutex.mutex(async () => {
        await decrypt()
        // ... process message, send receipt for read status separately ...
    })
} catch (error) {
    // ACK already sent, just log
    logger.error({ error }, 'error in handling message')
}

} Trade-off: If processing fails after the ACK, the message won't be retransmitted by WhatsApp. But this is better than losing ALL messages when the session goes deaf. Option B: Add timeout to messageMutex typescript// make-mutex.ts mutex(code: () => Promise | T, timeoutMs = 30000): Promise { return Promise.race([ mutex.runExclusive(code), new Promise((_, reject) => setTimeout(() => reject(new Error('Mutex timeout')), timeoutMs) ) ]) } Option C: Separate ACK mutex from processing mutex Use two different code paths:

A fast path that sends the ACK/receipt immediately (no mutex needed) A slow path with messageMutex that does decryption, upsert, and event emission

Questions for maintainers

Is the ACK-before-processing approach (Option A) safe? Does WhatsApp expect the ACK to mean "message was decrypted successfully" or just "message was received"? Is there a reason sendMessageAck is inside the messageMutex instead of outside? Is there a protocol-level dependency we're not seeing? Has anyone else experienced deaf sessions at scale (30+ concurrent connections)? Would you consider adding a timeout parameter to makeMutex()? The cacheMutex in makeCacheableSignalKeyStore is a single global mutex for ALL key operations. With external stores like Redis, this becomes a bottleneck. Would you consider per-type mutexes or removing the mutex in favor of the transaction system?

Reproduction Hard to reproduce in dev because it requires:

30+ concurrent sessions Running for 30+ minutes under real message load External key store (Redis) with occasional latency spikes

The clearest indicator is: messages.upsert stops firing while connection.update never emits close and keepAlive pings succeed. Logs When a session goes deaf, there are no error logs. The session simply stops receiving. This is what makes it so hard to debug — nothing fails, nothing throws, the mutex just silently blocks and ACKs stop flowing. Current workaround We implemented a health monitor that tracks the last messages.upsert timestamp per session and forces ws.close() if no messages arrive for 5+ minutes while the connection is still open. This triggers reconnection and temporarily fixes the issue, but it's not a real solution.