...

In this KIP, we will combine the second and third strategies. In coordinator, there is a cache to store each topic hash. When a topic change, only single topic hash will be recalculated. Different groups can get the topic hash from the cache, so the coordinator doesn't need to calculate a same topic hash multiple times. In __consumer_offsets, the coordinator sums all subscribed topic hash to a single hash per group and store to the disk, so it doesn't uses too much disk usage to represent the change.

Public Interfaces

ConsumerGroupPartitionMetadataKey / ConsumerGroupPartitionMetadataValue / ShareGroupPartitionMetadataKey /

...

ShareGroupPartitionMetadataValue

These types Both ConsumerGroupPartitionMetdataValue and ShareGroupPartitionMetadataValue will be deleted, because the coordinator doesn't store topic metadata.

...

Compatibility, Deprecation, and Migration Plan

ConsumerGroupPartitionMetadataValue / ShareGroupPartitionMetadataValue

There two records will be removed. For compatibility, we should keep GroupMetadataManager#replay functions, so the coordinator can read old data.

...