Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

TBD

Partition Reconciliation

Partition reconciliation is the act of updating the consumer's internal state to reflect its assigned partitions. This reconciliation occurs in multiple steps, shown here:

Step 1

PlantUML Render Macro
border1
formatPNG
exportNamepartition-reconciliation-111.txt
titlePartition Reconciliation
hspace0
Step 1

@startuml
participant

@startuml
hide unlinked

participant  "ProtoAsyncConsumer"           as PAC
participant   "Heartbeat\nRequest\nManager"  as as HRM
participant   "Member\nState"                as as MS
participant   "Assignment\nReconciler"       as AR
participant   "Revoke\nPartitions\nEvent"    as as RPE
queue         "Backend\nEvent\nQueue"        as as BEQ

skinparam maxMessageSize 70

HRM -> MS: heartbeat received
activate MS

MS -> AR: delegate assignment reconciliation (if needed)    
activate AR

AR   -> AR: determine partions to revoke
AR  -> AR: get state
create RPE
AR   -> RPE : new
AR   -> BEQ : Enqueue event for application thread
AR -> MS : 
deactivate AR
MS -> HRM :
deactivate MS
@enduml

Step 2

PlantUML Render Macro
border1
formatPNG
exportNamepartition-reconciliation-step-2.png
titlePartition Reconciliation Step 2

@startuml
actor User as U
participant ProtoAsyncConsumer as PAC
participant "Revoke\nPartitions\nEvent" as RPE

participant "Partitions\nRevoked\nEvent" as PRE

Participant "Consumer\nRebalance\nListener" as CRL
queue "Application\nEvent\nQueue" as AEQ

queue "Backend\nEvent\nQueue" as BEQ

skinparam maxMessageSize 70

U -> PAC: poll()
activate PAC

PAC -> BEQ: poll()
activate BEQ

return

PAC -> PAC: process background events
...

PAC -> RPE: partitions()

activate RPE

return


PAC -> CRL: partitionsRevoked()
activate CRL
return

create PRE
PAC -> PRE : new
PAC -> AEQ : add()

activate AEQ

return

return
@enduml