Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Vote thread:  here

JIRA

PR:  

Motivation


Consumer group rebalance is one of the most critical lifecycle events for Kafka consumers, because it directly affects users' message consume behavior. At the same time, Kafka is recommending and adopting server-side rebalance.

Kafka clients already provide rebalance-related callbacks, but relying on client-side behavior is operationally fragile. In practice, callback availability and behavior depend on SDK adoption and upgrade cycles, which are difficult to control sometimes. Even when teams implemented the callback with log only. users may reduce or disable verbose logs by mistake or some other reasons. This creates a common failure mode: when incidents occur, critical rebalance evidence is missing at the client side, making diagnosis slow and uncertain. 

What's more, today server-side observability is limited: coordinator logs can indicate rebalance activity, but logs alone are not a reliable integration surface for automated handling, and existing metrics do not carry the most actionable identifier (group.id). As a result, operators can observe aggregate rebalance trends, but cannot lightweightly attach custom processing to specific groups without log parsing or invasive client changes.

This KIP proposes a lightweight broker-side rebalance callback capability to expose key rebalance context (including group.id) from the coordinator. The goal is to provide a stable, centrally controlled extension point for observability and operational automation, without forcing client SDK upgrades and without introducing high-cardinality metric tags.This KIP proposes 
 

Public Interfaces

This KIP introduces:
 

...