Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
titleConsumerConfig.java
.defineInternal(LEAVE_GROUP_ON_CLOSE_CONFIG,
                Type.BOOLEAN,
                false,
                Importance.LOW)


When do we rebalance in static membership?

Rebalance happens rarely in static membership. There are two configs to control the behavior: registration timeout and expansion timeout.

Code Block
titleConsumerConfig.java
public static final STRING REGISTRATION_TIMEOUT_MS = "registration.timeout.ms";

Registration timeout is the timeout we will trigger rebalance when a member goes offline for too long. It should usually be set much larger than session timeout which is used to detect consumer health. By setting it to 15 ~ 30 minutes, we are loosening the track of static member progress, and transfer the member management to client application like K8. Of course, we should not wait forever for the member to back online simply for the purpose of reducing rebalances. Eventually the member will be kicked out of group and a final rebalance is triggered. Note that we are tracking the earliest offline member and compare with the registration timeout. Example below with registration timeout 15 min:

EventTimeearliest timeAction
Member A dropped 00:00 00:00 N/A

Member B dropped

 00:10 00:00N/A
Member A back online 00:14 00:10N/A
B gone for too long00:25 00:10Rebalance


  1.  00:00. Earliest offline member drop time: 00:00
  2. Member B dropped out of loop at time 00:10.
  3. Member A back online 




Switch between static and dynamic membership

For the very first version, we hope to make the logic simple without confusing people. As we have mentioned, the goal of static membership is to reduce multiple rebalances with intermittent failures. There should be an easy way to fallback to dynamic membership when user prefers to let broker handle task assignments, for example there is no handy tool like K8 or other scheduling framework. The fallback is simple: when the membership becomes stable, the first join group request will decide the membership protocol for next round. For example, when we are running stable with static membership, deploy a new change to the client app without member.name being set could invalidate the current hashmap on broker, and all the v4 request containing member.name will be treated as a normal dynamic member and perform the sticky assignment as usual. While we keep adding the static member into the group, the behavior will not change, when we shall wait until member.expansion.timeout to perform the rebalance. 


Image Added


Compatibility, Deprecation, and Migration Plan

...