DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Table of Contents |
|---|
Status
Current state: Under Discussion Accepted
Discussion thread: https://lists.apache.org/thread/m6m1qng5v9q8dd1mng0sygc9x83mx8kn
Voting thread: https://lists.apache.org/thread/w1myl1hsmd5bdko6y8y2fdtzvtcsfh13Discussion thread: N/A
JIRA:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
...
Following KIP-1022: Formatting and Updating Features, the group.coordinator.rebalance.protocols server configuration has become redundant. KIP-848 (New Group Coordinator), KIP-932 (Share groupsGroup), and KIP-1071 (Streams rebalance protocol) now implement group.version , streams.version , and share.version respectively. These feature versions enable cluster-wide upgrades and downgrades through kafka-features.sh, eliminating the need for server-level configuration to control feature versions.
Having a server-level configuration is problematic because individual brokers can be misconfigured, resulting in a cluster operating in a mixed state. Moreover, the current implementation shows transitional Additionally, there is inconsistent behavior across different coordinator types. While : while share groups have completely decoupled from group.coordinator.rebalance.protocols, the new group coordinator and streams still rely depend on this configuration. This means that even when the feature version is upgraded via via kafka-features.sh, the new capabilities remain unavailable unless the server configuration is also updated. This dual requirement exists because the feature flag mechanism was introduced later in the development cycle, resulting in a transitional period where both systems coexist. This can lead to user confusion when feature upgrades don't immediately enable new capabilities.
...
- org.apache.kafka.coordinator.group.GroupCoordinatorConfig.GROUP_COORDINATOR_REBALANCE_PROTOCOLS_CONFIG
Proposed Changes
The group.coordinator.rebalance.protocols configuration will be deprecated in Kafka 4.
...
3 and removed in version 5.0
...
.
...
Both New Group Coordinator and Streams will follow the Share Groups model, where functionality is controlled solely by group.version and streams.version
...
features via kafka-features.sh. During the deprecation period, if users are missing any of default values "classic, consumer, streams", a warning will inform them that this config will be deprecated and they cannot disable a protocol with it in version 5.0.
This eliminates the dual requirement of setting both configuration and feature flags, providing a consistent experience across all coordinator types. Users should use kafka-features.sh for all feature version management
...
and can safely remove
...
this configuration from their server.properties files, as it no longer affects feature functionality.
Compatibility, Deprecation, and Migration Plan
For the group coordinator, after users upgrade, the system will maintain the same version as before the upgrade because feature information is persisted. Users can upgrade using kafka-features.sh without concern for the config value, as it has been decoupled from the feature version.
As for Share Groups and Streams rebalance protocol, the same applies. Additionally, since neither has reached general access yet, there are no migration concerns.
Test Plan
Current Status (Kafka 4.2 - Upcoming Release)
Share Groups has already deprecated and decoupled from this configuration (though still listed in valid values in the config description).
Both Group Coordinator and Streams rebalance protocol are currently controlled by both the group.coordinator.rebalance.protocols configuration and their respective feature flags (`group.version` and `streams.version`).
Deprecation Timeline
Kafka 4.3 (Deprecation Period):
As described in the Proposed Changes, if users explicitly set group.coordinator.rebalance.protocols with values missing any of the defaults (classic, consumer, streams), users can still disable protocols, but will receive a deprecation warning advising them to either:
- Use the default values (classic, consumer, streams), or
- Remove the configuration
Kafka 5.0 (Removal):
This configuration will be completely removed. Any protocols previously disabled via this configuration will become available, as protocol control will be managed exclusively through feature flags.
Test Plan
This change will be validated through unit and integration tests