THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Code Block |
---|
public class StickyAssignorConsumerRebalanceListener implements ConsumerRebalanceListener { ... private Collection<TopicPartition> revokedPartitions; @Override public void onPartitionsRevoked(Collection<TopicPartition> partitions) { this.revokedPartitions = partitions; } @Override public void onPartitionsAssigned(Collection<TopicPartition> partitions) { this.revokedPartitions.removeAll(partitions); cleanup(this.revokedPartitions); } ... } |
Looking back at Example 2, when Consumer C0 is removed a rebalance occurs and with the round robin assignment the remaining consumers would perform partition clean up on all of their partitions before the rebalance (a total of 5). With the sticky assignor and the change proposed above, since both C1 and C2 preserve all their assigned partitions there would be no need to do any cleanup.
Rejected Alternatives
- Having the consumer group leader store the calculated topic partition assignment in an internal topic for other consumers to retrieve in case of a leadership change. It was decided that passing the calculated assignments as user data to all consumers after each rebalance is a more viable option.