...

In the AbstractPartitionAssignor, we would have a validateSubscription function which takes in the ownedPartitions across all members, and needs to be called by all assignors (it is the customized assignor's own responsibility to call it), to check that ownedPartitions do not have overlaps.

Note: This KIP doesn't have anything to do with the brokers. The "generation" field added, is in the subscription metadata, which will not be deserialized by brokers. The metadata is only deserialized by consumer lead. And for the consumer lead, the only thing the lead cared about, is the highest generation of the ownedPartitions among all the consumers. With the highest generation of the ownedPartitions, the consumer lead can distribute the partitions as sticky as possible, and most importantly, without errors.

Compatibility, Deprecation, and Migration Plan

...

This can still complete the assignment without any error, and reduce the network traffic between consumer/broker.

3. One scenario that we need to consider is what happens during a rolling upgrade. If the coordinator moves back and forth between brokers with different IBPs, it seems that the same epoch numbers could be reused for a group, if things are done in the obvious manner (old IBP = don't read or write epoch, new IBP = do).

--> This KIP doesn't care about the group epoch number at all. The subscription metadata is passed from each members to group coordinator, and then pass all of them back to consumer lead. So even if the epoch number is reused in a group, it should be fine.