...

A more detailed description of this process can be found in KIP-848 in Section Supporting Online Consumer Group Upgrade.  Classic Group

An example of a migration:

• classic group (generation=

...

• 23)
  
  • A
  • B

...

• assignment

• ...

  • • A - active=[0_0, 0_2, 0_

  ...

  • • 4], standby=[0_

  ...

  • • 1, 0_3, 0_5]
    • B - active=[0_1, 0_3, 0_

  ...

  • • 5], standby=[0_0, 0_2, 0_4]

  C joins using the Streams protocol. The classic group is transformed to a streams group.

  ...

  • streams group (group epoch=24)
    • A (classic)
    • B (classic)
    • C
  • target assignment (epoch=24)
    • A - active=[0_0, 0_2, 0_3], standby=[0_1, 0_5], warm-up=[]
  • • B - active=[0_1, 0_4, 0_5], standby=[0_

  ...

  • • 2, 0_3

  ...

  • • ], warm-up=[]

  ...

  • • C - active=[], standby=[0_

  ...

  • • 0

  ...

  • • , 0_

  ...

  • • 4],

  ...

  • • warm-up=[0_

  ...

  • • 2, 0_5]
  • member assignment
    • A
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_0, 0_2, 0_4], standby=[0_1, 0_3, 0_5]
      • JoinGroupResponse: generation ID=24
      • SyncGroupResponse: active=[0_0, 0_2, 0_3], standby=[0_1, 0_5]
    • B
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_1, 0_3, 0_5], standby=[0_0, 0_2, 0_4]
      • JoinGroupResponse: generation ID=24
      • SyncGroupResponse: active=[0_1, 0_4, 0_5], standby=[0_2, 0_3]
    • C
      • StreamsGroupHeartbeat: epoch=24, active=[], standby=[0_0, 0_4], warm-up=[0_2, 0_5]

      C's warm-up task 0_2 is caught up. A is requested to revoke active task 0_2, thus A does not increase its generation ID.

  • streams group (group epoch=25)
    • A (classic)
    • B (classic)
    • C
  • target assignment (epoch=25)
    • A - active=[0_0, 0_3], standby=[0_1, 0_5], warm-up=[]
    • B - active=[0_1, 0_4, 0_5], standby=[0_2, 0_3], warm-up=[]
    • C - active=[0_2], standby=[0_0, 0_4], warm-up=[0_5]
  • member assignment
    • A
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_0, 0_2, 0_3], standby=[0_1, 0_5], warm-up=[]
      • JoinGroupResponse: generation ID=24
      • SyncGroupResponse: active=[0_0, 0_3], standby=[0_1, 0_5]
    • B
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_1, 0_4, 0_5], standby=[0_2, 0_3]
      • JoinGroupResponse: generation ID=25
      • SyncGroupResponse: active=[0_1, 0_4, 0_5], standby=[0_2, 0_3]
    • C
      • StreamsGroupHeartbeat: epoch=25, active=[], standby=[0_0, 0_4], warm-up=[0_5]

  A follow-up rebalance is triggered so that A can report the revoked active task 0_2. Since A does not need to revoke tasks anymore the generation ID is increased.

  • streams group (group epoch=25)
    • A (classic)
    • B (classic)
    • C
  • target assignment (epoch=25)
    • A - active=[0_0, 0_3], standby=[0_1, 0_5], warm-up=[]
    • B - active=[0_1, 0_4, 0_5], standby=[0_2, 0_3], warm-up=[]
    • C - active=[0_2], standby=[0_0, 0_4], warm-up=[0_5]
  • member assignment
    • A
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_0, 0_3], standby=[0_1, 0_5], warm-up=[]
      • JoinGroupResponse: generation ID=25
      • SyncGroupResponse: active=[0_0, 0_3], standby=[0_1, 0_5]
    • B
      • Receives REBALANCE_IN_PROGRESS error in heartbeat response
      • JoinGroupRequest: active=[0_1, 0_4, 0_5], standby=[0_2, 0_3]
      • JoinGroupResponse: generation ID=25
      • SyncGroupResponse: active=[0_1, 0_4, 0_5], standby=[0_2, 0_3]
    • C
      • StreamsGroupHeartbeat: epoch=25, active=[0_2], standby=[0_0, 0_4], warm-up=[0_5]

  If C leaves the group now, the group coordinator transforms back the group to a classic group and only use JoinGroup, SyncGroup, and Heartbeat to communicate with the members. The records of the Streams protocol are deleted from the __consumer_offsets topic.4]

  Public Interfaces

  This section lists the changes impacting the public interfaces.

  ...