refactor: clarify `callback` responsibility and remove redundant topic cleanup logic
1. Background: The Role of asyncApplyDelta and its callback
In BrokerMetadataPublisher#handleTopicsDelta, core partition state changes are asynchronously applied via ElasticReplicaManager#asyncApplyDelta.
This method includes a callback mechanism designed for a fast response to single-partition state changes. For instance, after a partition becomes a leader (makeLeader) on the local broker, the callback is immediately invoked. This allows modules like GroupCoordinator to react quickly to the state change for that specific partition without waiting for all other operations in the Delta to complete, which is crucial for timely state synchronization.
2. Evolution of the Issue and Logic Redundancy
The logic for cleaning up consumer group offsets has evolved through two main iterations, leading to the current redundancy.
Phase 1 (PR #1264): The initial fix extended the
callback's role to address cleanup after topic deletion. It added logic to iterate over all partitions of a deleted topic and invoke thecallbackfor them if any replicas were found inlocalChanges.deletes.Phase 2 (PR #2626): It was later found that the fix in PR #1264 was incomplete. If a broker held no replicas for the deleted topic,
localChanges.deleteswould be empty, and the cleanup logic would never trigger on that node. To resolve this, PR #2626 introduced a new approach: the cleanup logic was removed from thecallbackand moved to a new method,notifyGroupCoordinatorOfDeletedPartitions. This method is called after theasyncApplyDeltatask completes (whenCompletestage) and operates on the globaltopicsDelta.deletedTopicIds(), ensuring reliable execution on all brokers.
3. Current State and Refactoring Proposal
The callback in asyncApplyDelta lacks a clear definition of responsibility at the interface level. This has led to it being used in two different scenarios: one for fast, single-partition state changes, and another for bulk, topic-level cleanup.
With the merge of PR #2626, the old cleanup logic (based on the callback) has become redundant, as its functionality is now superseded by the new implementation.
We propose removing the redundant logic to eliminate code redundancy and standardize the callback's responsibility.
Source: AutoMQ/automq