...

The idea of this KIP is to reduce number of rebalances by specifying the consumer member idintroducing a new concept: static membership. Core argument is: If the broker recognize this consumer as an existing member, it shouldn't trigger rebalance. The only exception is when leader rejoins the group, because there might be assignment protocol change that needs to be enforced, for example if the consumer group is using regex subscription and new matching topics show up.

...

In the current architecture, during each rebalance consumer group groups on broker side will assign new member id with a UUID randomly generated each time. This is to make sure we have unique identity for each group member. During client restart, consumer will send a JoinGroupRequest with a special UNKNOWN_MEMBER id, which has no intention to be treated as an existing member.  To make the KIP work, we need to change both client side and server side logic to make sure we persist member id identity throughout restarts, which means we could reduce number of rebalances since we are able to apply the same assignment based on member identities.

Proposed Changes

We will be introducing new terms:

• Static Membership: the membership protocol where the consumer group will not trigger rebalance unless 1. a new unknown member joins 2. a leader rejoins. 3. an existing member go offline over certain timeout.
• Member name: the unique identifier defined by user to distinguish each client instance.
• Member registration timeout: the max time we could tolerate a static member to go offline.
• Member expansion timeout: the max time we will wait since we receive a new member join request. (Details later)

On client side, we add a new config called MEMBER_ID NAME in ConsumerConfig. On consumer service init, if the MEMBER_ID NAME config is set, we will put it in the initial join group request to identify itself as a static member.; otherwise, we will still send UNKNOWN_MEMBER_ID to ask broker for allocating a new random ID. To distinguish between previous version of protocol, we will also increase the join group request version to v4 when MEMBER_ID is set. 

public static final STRING MEMBER_IDNAME = "member.id_A"; // default empty String

...