Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
public class StickyAssignorConsumerRebalanceListener implements ConsumerRebalanceListener {

    ...
 
    private Collection<TopicPartition> revokedPartitions;
 
    @Override
    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
        this.revokedPartitions = partitions;
    }
 
    @Override
    public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
        this.revokedPartitions.removeAll(partitions);
        cleanup(this.revokedPartitions);
    }

    ...
}

Looking back at Example 2, when Consumer Cis removed a rebalance occurs and with the round robin assignment the remaining consumers would perform partition clean up on all of their partitions before the rebalance (a total of 5). With the sticky assignor and the change proposed above, since both C1 and C2 preserve all their assigned partitions there would be no need to do any cleanup.

 

Rejected Alternatives

  • Having the consumer group leader store the calculated topic partition assignment in an internal topic for other consumers to retrieve in case of a leadership change. It was decided that passing the calculated assignments as user data to all consumers after each rebalance is a more viable option.