...
Now that 3.0 is coming up, we can change the default ConsumerPartitionAssignor to something better than the RangeAssignor. The original plan was to switch over to the StickyAssignor, but now that we have incremental cooperative rebalancing, we should consider using the new CooperativeStickyAssignor instead: this will enable the consumer group to follow the COOPERATIVE protocol, improving the rebalancing experience OOTB.
...
- It won't affect to the current consumers if current consumers have explicitly set the
partition.assignment.strategy
value - If the consumers rely on the default value of New consumers with default
partition.assignment.strategy
will be affectedUpgrading the old consumer to the new consumer setting can , users can do 1 of the following options:Set the
partition.assignment.strategy
config to one of the non-cooperative assignors during the rolling upgrade if they wish to remain on EAGER and/or perform the upgrade in just a single rolling bounce.- Upgrading the consumer (refer to the following upgrade path section)
Proposed Changes
- Change the default setting of
partition.assignment.strategy
in implementation and tests - When changing the default assignor we need to make sure this is clearly documented in the upgrade guide for 3.0
...
this will require users to follow the upgrade path laid out in KIP-429 to safely perform a rolling upgrade. Please note: KAFKA-12477 is doing the upgrade experience improvement to reduce the safe upgrade path to just a single rolling bounce. It's still in progress so far. Before it completed, we still need to follow the following upgrade path:
<Copy from KIP-429>
From the user's perspective, the upgrade path of leveraging new protocols is similar to switching to a new assignor. For example, assuming the current version of Kafka consumer is 2.2 and "range" assignor is specified in the config (or no assignor is configured, which is identical as the RangeAssignor is the default below 3.0). The upgrade path would be:
...
The key point behind this two rolling bounce is that, we want to avoid the situation where leader is on old byte-code and only recognize "eager", but due to compatibility would still be able to deserialize the new protocol data from newer versioned members, and hence just go ahead and do the assignment while new versioned members did not revoke their partitions before joining the group. Note the difference with KIP-415 here: since on consumer we do not have the luxury to leverage on list of built-in assignors since it is user-customizable and hence would be black box to the consumer coordinator, we'd need two rolling bounces instead of one rolling bounce to complete the upgrade, whereas Connect only need one rolling bounce.
...