DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Status
Current state: Under Discussion
Discussion thread: N/A
JIRA:
KAFKA-19740
-
Getting issue details...
STATUS
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Following KIP-1022: Formatting and Updating Features, the group.coordinator.rebalance.protocols server configuration has become redundant. KIP-848 (New Group Coordinator), KIP-932 (Share Group), and KIP-1071 (Streams rebalance protocol) now implement group.version , streams.version , and share.version respectively. These feature versions enable cluster-wide upgrades and downgrades through kafka-features.sh, eliminating the need for server-level configuration to control feature versions.
Having a server-level configuration is problematic because individual brokers can be misconfigured, resulting in a cluster operating in a mixed state. Moreover, the current implementation shows transitional behavior across different coordinator types. While share groups have completely decoupled from group.coordinator.rebalance.protocols, the new group coordinator and streams still rely on this configuration. This means that even when the feature version is upgraded via kafka-features.sh, the new capabilities remain unavailable unless the server configuration is also updated. This dual requirement exists because the feature flag mechanism was introduced later in the development cycle, resulting in a transitional period where both systems coexist. This can lead to user confusion when feature upgrades don't immediately enable new capabilities.
Public Interfaces
- org.apache.kafka.coordinator.group.GroupCoordinatorConfig.GROUP_COORDINATOR_REBALANCE_PROTOCOLS_CONFIG
Proposed Changes
The group.coordinator.rebalance.protocols configuration will be deprecated in Kafka 4.x and removed in version 5.0, following the same approach as Share Groups. The configuration will be decoupled from feature functionality, meaning it will be accepted but ignored during the transition period. Both New Group Coordinator and Streams will follow the Share Groups model where functionality is controlled solely by group.version and streams.version feature via kafka-features.sh. This eliminates the dual requirement of setting both configuration and feature flags, providing a consistent experience across all coordinator types. Users should use kafka-features.sh for all feature version management, and can safely remove the configuration from their server.properties files as it no longer affects feature functionality.
Compatibility, Deprecation, and Migration Plan
For the group coordinator, after users upgrade, the system will maintain the same version as before the upgrade because feature information is persisted. Users can upgrade using kafka-features.sh without concern for the config value, as it has been decoupled from the feature version.
As for Share Groups and Streams rebalance protocol, the same applies. Additionally, since neither has reached general access yet, there are no migration concerns.
Test Plan
This change will be validated through unit and integration tests