Status

Current state: Under Discussion Accepted

Discussion thread: here

JIRA:

...

In both cases, the zombie commit will therefore be rejected by the relaxed check.

Note that, as before, we will allow commits for partitions that are not owned by the member, if and only if the offset commit request contains the most recent broker-side member epoch. This is done to make the new rule strictly weaker than the previous check, which also allowed such commits.

Proposed Changes

Introducing an epoch per assigned partition

...

Where, for simplicity, we can assume the assignment epoch of a partition that is not assigned to that member to be the broker-side member epoch, to match the current behavior of accepting commits for unassigned partitions. Therefore, we allow commits for partitions that are not assigned to the member to be Integer.maxValue., as long as the committer uses an up-to-date member epoch, same as the current implementation.

Handling of commits without generation ID / member epoch

Requests that do not contain a valid member epoch, which can come from the admin client or a consumer with self-assigned partitions will be handled the same way as before. That is, for a non-existing groups, a simple group is created (which is not affected by this KIP). Offset commits with member epoch or generation ID -1 are unconditionally accepted of the group is empty, or the commit is transactional. 

Handling of legacy record members

...

When a static member leaves the group, the assignment epochs will be set to 0 in the current assignment record. When the static member rejoins, partitions will therefore be considered to be assigned "from the beginning" (i.e., from epoch 0) to the new member ID. All commits using the old memberId will be fenced, and all commits using the new memberId will be permitted (since all client-side member epochs are larger than 0).

Handling during protocol upgrade

If we upgrade from the classic protocol, we will set the assignment epoch of all partitions to the generation ID of the group. 

If we downgrade to the classic protocol, the assignment epochs will be abandoned, since they are not required in the classic protocol. 

Public Interfaces

ConsumerGroupCurrentMemberAssignmentValue

...