Page History

...

Different groups may subscribe to same topics. With topic hash map cache in the coordinator, it can avoid recalculation of same topic hash for different groups. The topic hash value should represent topic id, name, number of partitions, and partition racks.

...

Initial a topic hash

...

in the cache

...

The coordinator only cares topics which are subscribed by groups. When a coordinator starts, it replays all records in __consumer_offsets. When replaying ConsumerGroupMemberMetadtaValue / ShareGroupMemberMetadataValue, the group can get subscribed topic names from previous state. In the next group heartbeat, the group can rely on subscribed topic names to compute topic hash and keep it in the cache, so other groups don't need to recompute it.

Remove a topic hash in the cache

If a topic is not subscribed by any group, the coordinator doesn't need to maintain the hash value. When a coordinator starts, it replays all records in __consumer_offsetsmaintain the hash value. When replaying ConsumerGroupMemberMetadtaValue / ShareGroupMemberMetadataValue records, the coordinator adds a new topic to groupsByTopics if the topic is absent. The group coordinator can leverage it to add a new topic hash.

Remove topic hash from the cache in coordinator

A If a group unsubscribes a topic, the group is removed from groupsByTopics. If there is no group under a topic, the topic will be removed from groupsByTopics if there is no group subscribe to it. The group coordinator can remove the topic hash as well, so it can leverage same mechanism to remove a the topic hash.

...

Renew a topic hash in the cache

...

When there is a new metadata image, it contains changed topics and deleted topics in metadata delta. For both cases, the coordinator remove topic hash from the cache. The value will be recomputed when in the next consumer group heartbeat. The lazy evaluation is more efficient. If there is no further group heartbeat, the new topic hash value is useless, so we can just compute it in group heartbeat.

...