Page History

...

This KIP proposes a lightweight broker-side rebalance callback capability to expose key rebalance context (including consume group id) on kafka Kafka broker. 

In short, the goal is to provide a stable, centrally controlled extension point for observability and operational automation, without forcing client SDK upgrades and without introducing high-cardinality metric tags. 

...

Alternative 1:  Use the client consume rebalance listener
Relying on client-side behavior is operationally fragile. In practice, callback availability and behavior depend on SDK adoption and upgrade cycles, which are difficult to control sometimes. Even when teams implemented the callback with log only. users may reduce or disable the logs by mistake or some other reasons. This creates a common failure mode: when incidents occur, critical rebalance evidence is missing at the client side, making diagnosis slow and uncertain

Alternative 2:  Enhance the existed server side metric

Enhancing existing coordinator metrics is not sufficient for this use case because the key troubleshooting dimension is group.id, while server metrics are intentionally aggregate. If we add group.id as a metric label/tag, it introduces high-cardinality time series (potentially thousands of groups), which increases memory and storage cost on both broker and monitoring systems and can degrade observability performance.