Page History

...

private val MEMBER_METADATA_V3 = new Schema(
  new Field(MEMBER_ID_KEY, STRING),
  new Field(MEMBER_NAME_KEY, STRING), // new
  new Field(CLIENT_ID_KEY, STRING),
  new Field(CLIENT_HOST_KEY, STRING),
  new Field(REBALANCE_TIMEOUT_KEY, INT32),
  new Field(SESSION_TIMEOUT_KEY, INT32),
  new Field(SUBSCRIPTION_KEY, BYTES),
  new Field(ASSIGNMENT_KEY, BYTES))

...

Upgrade from dynamic membership to static membership

The recommended upgrade process is as follow:

1. Upgrade your broker to include this KIP-change.
2. Rolling bounce your consumer group to set member name and session timeout to a reasonable number.
   

That's it! We believe that the static membership logic is compatible with the dynamic one, which means it is allowed to have static members and dynamic members co-exist within the same consumer group. This assumption could be further verified when we do some modeling of the protocol (through TLA maybe) or dev test. 

command line API for membership management

For the very first version, we hope to make membership transferring logic and human handling simple enough. We will define three only one command line APIs to help us better manage the groups:

public static MembershipChangeResult enableStaticMembership(String groupId, int registrationTimeout, int expansionTimeout)
public static MembershipChangeResult enableDynamicMembership(String groupId)
public static MembershipChangeResult forceStaticRebalance(String groupId)

enableStaticMembership will change the consumer group to static membership, along with changing registration timeout and expansion timeout. After that, all the joining members are required to set the member name field. Error will be returned if

1. the broker is on an old version. 
2. if the group is preparing rebalance/completing rebalance.
3. Any member within the current group has not set `member.name`
4. other potential failure cases.

Note that the client should already include member name field at this point. User could also use this API to change the timeout configs as they want, or leave it blank to use default value.

enableDynamicMembership will in the contrary just change the membership back to dynamic mode. Error will be returned if

1. the broker is on an old version. 
2. if the group is preparing rebalance/completing rebalance.
3. group is already on dynamic membership.
4. other potential failure cases.

forceStaticRebalance will trigger one rebalance immediately on static membership, which is mainly used for fast scale up/ down cases (we already detect consumer failure and wants to start rebalance immediately without waiting for session timeout). Error will be returned if 

1. the broker is on an old version.
2. if the group is preparing rebalance/completing rebalance.
3. group is on dynamic membershiphas dynamic members (without member name).
4. other potential failure cases.

...