...

The core issue that this KIP addresses is the non-idempotency of the initial heartbeat in the new consumer group protocol.If the member ID is null or empty, the server will reject the request
with an InvalidRequestException. 
When an AsyncKafkaConsumer sends a join group heartbeat but closes before receiving the response, it may attempt to send a leave group heartbeat without a member ID, leading to a failure with UNKNOWN_MEMBER_ID. This scenario can result in the broker registering a new member for which it will never receive a proper leave request.
Moreover, if the response to the initial heartbeat is lost, the client will retry the request, potentially creating a new member each time. This can lead to the creation of several "ghost" members that will persist until their session timeout elapses. During this period, these ghost members continue to hold onto partitions, causing delays in partition reassignment and inefficiencies in group management.

...

The consumer instance must generate a member ID when it starts, and this ID should remain consistent for the entire lifetime of the process. The member ID acts as an incarnation ID of the process and should not be reset or changed, even if the consumer leaves and rejoins the group. It must remain the same until the process is completely stopped or terminated. 

Duplicate Member ID

Since we allow the client to generate the member ID themselves, duplicate member IDs might be a concern. However, as mentioned in the Motivation section, with improvements in third-party libraries like librdkafka , UUID generation has become easier today. We believe it is not difficult for the client to generate a unique ID, and moreover, the scope is limited to the consumer group. Therefore, we believe the risk of member ID collision within a group is negligible.

Compatibility, Deprecation, and Migration Plan

...