...
We basically want to re-joining leader to know that it is the current leader of the group while avoiding computing a new assignment. We propose to achieve this by adding a SkipAssignment
field in the JoinGroupResponse
. With this, When the group coordinator can correctly set the leader in the JoinGroupResponse
for the new leader while instructing it to skip the assignment by setting SkipAssignment
to true
detects that the static leader is rejoining the group, it will set SkipAssignment
to true, set the correct leader id and provide subscriptions of all members (which might be different from the leader's own subscriptions) so that the leader can monitor all topics subscribed in the group. Then the leader will send an empty SyncGroup
to collect its own (and existing) assignment. The group coordinator will do so only for consumers that support the latest version of the JoinGroup
API. Old Other consumers will continue to behave as they do today.
Note that the proposed solution is not perfect. The remaining issue is that the leader can not detect metadata changes (e.g. new partition added) while it is offline. Adding partitions to topics is a rather infrequent operations so the likelihood for it to happen exactly when the static leader is down is low. The proposed solution seems to be acceptable as a first step to fix the common issue.
Public Interfaces
The SkipAssignment
is added to the JoinGroupResponse
.
...