[Bug] Persisted batch ACK indexes are not restored from MetadataStore
Search before reporting
- I searched in the issues and found nothing similar.
Read release policy
- I understand that unsupported versions don't get bug fixes. I reproduced the issue on the current
masterbranch.
User environment
- Broker version: current
master(67205b946b763d43ba254601514f2f42ee5c520e) - Broker operating system: macOS / Darwin 25.6.0, x86_64
- Broker Java version: Eclipse Temurin 25.0.2
- Client library: Java client built from the same
masterrevision - Reproduced with both a focused Managed Ledger test and a broker-level
Sharedsubscription test
Issue Description
When a cursor is cleanly closed through the MetadataStore persistence path, its partial batch acknowledgment indexes are written to ManagedCursorInfo.batchedEntryDeletionIndexInfos. However, reopening the cursor from that metadata does not restore those indexes.
For example, given a batch containing A, B, C, and D, if A and B are acknowledged while C and D remain unacknowledged, unloading and reloading the topic can cause A and B to be delivered again.
Expected behavior:
- The reopened cursor restores the persisted batch ACK bitmap.
- Only
CandDare delivered after recovery.
Actual behavior:
- The persisted batch ACK bitmap is absent from the reopened cursor.
- Already acknowledged messages such as
AandBcan be redelivered.
This is a bug because the write path persists the state, and the equivalent BookKeeper cursor-ledger recovery path restores it correctly. The MetadataStore recovery path restores the mark-delete position, individually deleted entry ranges, and properties, but skips batchedEntryDeletionIndexInfos before completing cursor initialization.
Error messages
No exception or log error is emitted. The persisted batch ACK state is silently omitted during cursor recovery.Reproducing the issue
- Enable batch-index-level acknowledgment persistence.
- Create a
Sharedsubscription and publish one batch containing four messages:A,B,C, andD. - Consume the batch and acknowledge only
AandB; wait for the acknowledgments to complete. - Cleanly unload the topic so that the cursor position is persisted through MetadataStore (
cursorsLedgerId == -1). - Verify before reopening that
ManagedCursorInfo.batchedEntryDeletionIndexInfoscontains the partial batch ACK record. - Create a new client and reopen the subscription, ensuring that a new cursor instance is constructed.
- Inspect the recovered cursor or consume the remaining messages.
The recovered cursor returns null for the batch position ACK set even though the metadata contained it before reopening. At the broker level, already acknowledged messages can consequently be delivered again.
The issue does not require dispatcherPauseOnAckStatePersistentEnabled and does not require exceeding the batch ACK persistence limit. A normal consumer reconnect can reuse the in-memory cursor and therefore may not exercise the affected recovery path.
Additional information
The asymmetry is in ManagedCursorImpl:
persistPositionMetaStore(...)includes partial batch ACK records throughbuildBatchEntryDeletionIndexInfoList().- The BookKeeper cursor-ledger recovery branch calls
recoverBatchDeletedIndexes(...). - The
cursorsLedgerId == -1MetadataStore recovery branch does not call it.
A fix and regression tests are available in PR #26474.
Are you willing to submit a PR?
- I'm willing to submit a PR!
Source: apache/pulsar