Page History

...

If using the second strategy, it still uses lot of disk size to store redundant data. For example, a group subscribes a same topic listsome topics. Every time a subscribed topic change, the group needs to store a new subscription topic metadata map with a single entry changed. If using the third strategy, it wastes too much CPU resource to recalculate the hash when the metadata image is expired.

...

{
  "type": "data",
  "name": "ShareGroupMetadataValue",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Epoch", "type": "int32", "versions": "0+",
      "about": "The group epoch." },
    { "name": "MetadataHash", "versions": "0+", "type": "int64",
      "about": "The hash of all topics in the group." } <-- new field
  ]
}

StreamsGroupMetadataValue

Add a new field "MetadataHash".

...

When the coordinator initializes, the topic hash map cache is empty. After receiving the first consumer group heartbeat, the coordinator calculates subscribed topic hash, so it doesn't waste memory to store unsubscribed topic hash.

...

Example: the racks of a partition change

The rack of a partition is from There are two cases about rack change. One is users alter partition reassignment. Another is "broker.rack" configuration . This config can only change. The configuration is read only and needs to be changed after the broker reboots. When the broker is stopped, the broker id is removed from PartitionRegistration#isrMetadataHash. This is reflected For both cases, if the replica is ISR on a reassigned/rebooted broker, the change is reflected in PartitionRegistration#isr. This is also included in TopicDelta#partitionChanges. The renew process is similar to above. A new MetadataDelta contains the topic in TopicsDelta#changedTopics and the coordinator remove the topic hash after it receives the change. In the next group heartbeat, the coordinator re-compute the value.

The PartitionRegistration also contains other changes like elr. These change will make the topic hash recomputation. It's no harm, because the hash only reflects change about uuid / name / number of partition / racks of partitions. If non of these field change, the final hash will be the same, so it doesn't bump the group epoch.

All subscribed topic hash in Group

...