Status

Current state: Draft Accepted

Discussion thread: N/A: https://lists.apache.org/thread/m6m1qng5v9q8dd1mng0sygc9x83mx8kn

Voting thread: https://lists.apache.org/thread/w1myl1hsmd5bdko6y8y2fdtzvtcsfh13

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Following KIP-1022 (: Formatting and Updating Features), the group.coordinator.rebalance.protocols server configuration has become redundant. KIP-932 (share groups848 (New Group Coordinator), KIP-1071 (streams rebalance protocol932 (Share Group), and  and KIP-848 (group coordinator) now implement ShareVersion, StreamsVersion, and GroupVersion 1071 (Streams rebalance protocol) now implement group.version , streams.version , and share.version  respectively. These feature versions enable cluster-wide upgrades and downgrades through kafka-features.sh, eliminating the need for server-level configuration to control feature versions.

Having a server-level configuration is problematic because individual brokers can be misconfigured, resulting in a cluster operating in a mixed state. Feature flags provide a superior alternative by ensuring cluster-wide consistency and preventing configuration drift between brokersAdditionally, there is inconsistent behavior across coordinator types: while share groups have completely decoupled from group.coordinator.rebalance.protocols, the new group coordinator and streams still depend on this configuration. This means that even when the feature version is upgraded via kafka-features.sh, the new capabilities remain unavailable unless the server configuration is also updated. This dual requirement exists because the feature flag mechanism was introduced later in the development cycle, resulting in a transitional period where both systems coexist. This can lead to user confusion when feature upgrades don't immediately enable new capabilities.

Public Interfaces

• org.apache.kafka.coordinator.group.GroupCoordinatorConfig.GROUP_COORDINATOR_REBALANCE_PROTOCOLS_CONFIG

Proposed Changes

...

The group.coordinator.rebalance.protocols

...

configuration will be deprecated in Kafka 4.3

...

and removed in version 5.0. Both New Group Coordinator and Streams will follow the Share Groups model, where functionality is controlled solely by group.version  and streams.version features via kafka-features.sh. During the deprecation period, if users are missing any of default values "classic, consumer, streams", a warning will inform them that this config will be deprecated and they cannot disable a protocol with it in version 5.0.

This eliminates the dual requirement of setting both configuration and feature flags, providing a consistent experience across all coordinator types. Users should use kafka-features.sh for all feature version management and can safely remove this configuration from their server.properties files, as it no longer affects feature functionality.

...

Compatibility, Deprecation, and Migration Plan

...

Current Status (Kafka 4.2 - Upcoming Release)

Share Groups has already deprecated and decoupled from this configuration (though still listed in valid values in the config description). 
Both Group Coordinator and Streams rebalance protocol are currently controlled by both the group.coordinator.rebalance.protocols

...

  configuration and their respective feature flags (`group.version` and `streams.version`).

Deprecation Timeline

Kafka 4.3 (Deprecation Period):

As described in the Proposed Changes, if users explicitly set group.coordinator.rebalance.protocols

...

  with values missing any of the defaults (classic, consumer, streams), users can still disable protocols, but will receive a deprecation warning advising them to either:
- Use the default values (classic, consumer, streams), or
- Remove the configuration

Kafka 5.0 (Removal):
This configuration will be completely removed. Any protocols previously disabled via this configuration will become available, as protocol control will be managed exclusively through feature flags.

Test Plan

This change will be validated through unit and integration tests

Rejected Alternatives

References

• KIP-848: The Next Generation of the Consumer Rebalance Protocol

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

References

KIP-1022: Formatting and Updating Features
• KIP-932: Queues for Kafka
(Share Groups)
• KIP-1071: Rebalance Protocol for Kafka Streams1022: Formatting and Updating Features
• KIP-848: The Next Generation of the Consumer 1071: Streams Rebalance Protocol