[Bug][Connector-V2][RocketMQ] Broker-side consumer offset never becomes visible after commitSync, breaking RocketMqIT across all container legs
Search before asking
- I searched in the issues and found nothing similar.
What happened
RocketMqIT failed 43 invocations (14 failures + 29 errors) in a single CI run — every container leg of testSinkRocketMq (and siblings) errored at the same place: after the test consumer successfully polled messages and called commitSync(), the broker-side consumer offset for MessageQueue [topic=test_topic, brokerName=broker-a, queueId=0] was never visible — not just lagging, but absent from RocketMqAdminUtil.currentOffsets — for the full 60s window.
Evidence (fork CI run on branch fix-12353-cancel-state-priority, 2026-09-17, jobs rocketmq-connector-it (8/11, ubuntu-latest))
ConditionTimeoutException: Consume offset should be visible for MessageQueue
[topic=test_topic, brokerName=broker-a, queueId=0] ==> expected: not <null> within 1 minutes.
at RocketMqIT.waitConsumedOffsetsSynced(RocketMqIT.java:577)
at RocketMqIT.getRocketMqConsumerData(RocketMqIT.java:558)- Message consumption itself worked (test logs show
Consumer test_topic data total 10..70). updateConsumeOffsetToBroker+commitSync()were called (the while loop completed —data totalis logged after it).- The admin-side offset query then never returned an entry for broker-a queue 0 within 60s.
Note this window was already widened from 30s → 60s for a previous occurrence, so "slow broker" alone does not explain an offset that never appears at all.
Expected behavior
After commitSync() returns, queryConsumerOffset/admin offset lookups for the consumed queues should observe the committed offset (within the retry window).
Suspect areas
- Offset commit vs. consumer group rebalance: the test consumer shuts down right after
commitSync(); if a rebalance/instance deregistration races the commit, the broker may drop or never persist the offset for that queue. RocketMqAdminUtil.currentOffsetssilently omitting queues whosequeryConsumerOffsetreturnsOFFSET_NOT_FOUNDor throws, which turns a broker-side miss into a permanent map miss instead of a retryable lag.- Topic route propagation timing for queues on
broker-awhen multiple queues/brokers exist.
Are you willing to submit PR?
- Yes, I am willing to investigate and submit a PR, but would first like maintainer input on the suspect areas above.
Source: apache/seatunnel