#12383·seatunnel

[Bug][Connector-V2][RocketMQ] Broker-side consumer offset never becomes visible after commitSync, breaking RocketMqIT across all container legs

Author: zhangshenghangCreated Sep 18, 2026Updated Sep 18, 2026

Search before asking

  • I searched in the issues and found nothing similar.

What happened

RocketMqIT failed 43 invocations (14 failures + 29 errors) in a single CI run — every container leg of testSinkRocketMq (and siblings) errored at the same place: after the test consumer successfully polled messages and called commitSync(), the broker-side consumer offset for MessageQueue [topic=test_topic, brokerName=broker-a, queueId=0] was never visible — not just lagging, but absent from RocketMqAdminUtil.currentOffsets — for the full 60s window.

Evidence (fork CI run on branch fix-12353-cancel-state-priority, 2026-09-17, jobs rocketmq-connector-it (8/11, ubuntu-latest))

ConditionTimeoutException: Consume offset should be visible for MessageQueue
[topic=test_topic, brokerName=broker-a, queueId=0] ==> expected: not <null> within 1 minutes.
    at RocketMqIT.waitConsumedOffsetsSynced(RocketMqIT.java:577)
    at RocketMqIT.getRocketMqConsumerData(RocketMqIT.java:558)
  • Message consumption itself worked (test logs show Consumer test_topic data total 10..70).
  • updateConsumeOffsetToBroker + commitSync() were called (the while loop completed — data total is logged after it).
  • The admin-side offset query then never returned an entry for broker-a queue 0 within 60s.

Note this window was already widened from 30s → 60s for a previous occurrence, so "slow broker" alone does not explain an offset that never appears at all.

Expected behavior

After commitSync() returns, queryConsumerOffset/admin offset lookups for the consumed queues should observe the committed offset (within the retry window).

Suspect areas

  • Offset commit vs. consumer group rebalance: the test consumer shuts down right after commitSync(); if a rebalance/instance deregistration races the commit, the broker may drop or never persist the offset for that queue.
  • RocketMqAdminUtil.currentOffsets silently omitting queues whose queryConsumerOffset returns OFFSET_NOT_FOUND or throws, which turns a broker-side miss into a permanent map miss instead of a retryable lag.
  • Topic route propagation timing for queues on broker-a when multiple queues/brokers exist.

Are you willing to submit PR?

  • Yes, I am willing to investigate and submit a PR, but would first like maintainer input on the suspect areas above.