Feature request: Support KIP-848 (Next-Gen Consumer Group Protocol: ConsumerGroupHeartbeat / ConsumerGroupDescribe)

Who is this for and what problem do they have today?

We run Redpanda as our Kafka-compatible streaming backbone and use Java/Spring Kafka consumers with consumer groups. On the latest Redpanda, API negotiation shows classic group APIs are supported (JoinGroup/SyncGroup/Heartbeat), but KIP-848 APIs are UNSUPPORTED, e.g.:

• ConsumerGroupHeartbeat (API key 68): UNSUPPORTED
• ConsumerGroupDescribe (API key 69): UNSUPPORTED

This forces us to stay on the legacy consumer group protocol and limits our ability to adopt Kafka 4.x’s next-gen group rebalance improvements, especially as our consumer groups grow.

What are the success criteria?

• Redpanda supports KIP-848 “consumer” group protocol so Kafka clients can use it (e.g. via group.protocol=consumer where applicable).
• ConsumerGroupHeartbeat (68) and ConsumerGroupDescribe (69) are implemented and usable with common Kafka clients.
• Rolling migration from classic protocol is possible (no full-cluster downtime), with clear documentation of any limitations.

Out of scope: Share Groups / queue semantics are not required for this request unless they are hard dependencies.

Why is solving this problem impactful?

KIP-848 is important for large consumer groups because it reduces rebalance disruption and improves scalability. Supporting it would help teams like ours modernize consumer group behavior and maintain compatibility with newer Kafka client capabilities.

Additional notes

• We can provide full ApiVersions negotiation output and help test pre-releases.
• Please share whether this is on the roadmap, and if there is an existing tracking issue or milestone.