...
PlantUML Render Macro | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
@startumlskinparam maxMessageSize 70 delegate assignment reconciliation (if needed) activate ARAR -> AR: determine partions to revoke create RPE AR -> RPE : new AR -> BEQ : Enqueue event for application thread AR -> MS : deactivate AR MS -> HRM : deactivate MS updateAssignment() |
In the above...
Step 2
PlantUML Render Macro | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
@startumlParticipant participant "Consumer\nRebalance\nListener" as CRL skinparam maxMessageSize 70
|
Rebalance State Machine
PlantUML Render Macro | ||
---|---|---|
| ||
@startuml [*] --> NOT_IN_GROUP UNJOINED ---> UNJOINED : Heartbeat on retries STABLE ---> RECONCILING : new assignment received\ntrigger callbacks RECONCILING ---> RECONCILING : Heartbeat RELEASING_ASSIGNMENT ---> RELEASING_ASSIGNMENT_COMPLETE : 'onLost' callback completed FATAL ---> [*] @enduml |
David's notes
Gliffy Diagram | ||||||||
---|---|---|---|---|---|---|---|---|
|
States:
- UNSUBSCRIBE: The consumer is not subscribed to any topics nor regex therefore it is not part of a consumer group.
- JOINING: The consumer has subscribed with topic names or a regex. Consumer send an HB request to join the group with epoch 0 and transitions to Stable.
- STABLE: While in this state, has nothing to do besides heartbeatting to remain in the group.
- RECONCILE_ASSIGNMENT: Whenever the consumer received a non-null assignment from the group coordinator, it transitions to this state and reconciles its assignment. It should revoke unnecessary partitions and assign the new ones. This also commits offsets and triggers the rebalance callbacks. When the reconciliation completes, it transitions to ACK_ASSIGNMENT.
- ACK_ASSIGNMENT: This signals to the HB manager than an HB request must be sent in the next run of the event loop event the HB internal has not expired. It transitions to STABLE when that signal is given.
- UNSUBSCRIBING: When the consumer calls unsubscribe or close (this can happen anytime), it transitions to this state, cancels any ongoing reconciliation (how to?), revoke partitions/commit offsets and send the last HB to leave the group. When done, it transitions to UNSUBSCRIBE.
- FENCED: When the group coordinator fences the member (this can happen anytime), it transition to this state, cancels any ongoing reconciliation (how to?), resets the member epoch and invokes onLost for all partitions. When done, it transitions to JOINING to rejoin the group.
- FATAL: The consumer enters this state whenever a fatal errors is encountered. This is not recoverable.
Notes
- When the subscriptions are changed, should we send the next HB immediately?
- Should we transition from FATAL to UNSUBSCRIBE when the subscriptions are changed? Let's imagine that the user subscribes with an invalid regex. In
...
- this case, the consumer transition to FATAL as this is not recoverable. However, the user may react to the exception and change the subscriptions. We may need to give it another try if we have new subscriptions.