Page History

...

In the new version of the ConsumerGroupHeartbeat RPC, the client must generate a UUID as the member ID during the initial heartbeat. This member ID must be included in every subsequent request to ensure consistency. We highly recommend that users utilize a UUID as the member ID, but ultimately, the choice is up to the user. The server will validate that a valid UUID member ID is provided in the member ID field. If the member ID is missing null or invalidempty, the server will reject the request with an InvalidRequestException.

LastLastly, to accommodate these updates, we also propose bumping the ConsumerGroupHeartbeat RPC from version 0 to version 1. This version upgrade reflects the new requirement for client-generated member IDs. It's important to note that despite these behavioral changes, there are no modifications to the existing fields themselves.

...

To implement the changes proposed in the KIP, the client will might need to generate UUIDs. The simple specifications are as followsfollowing specifications serve as guidelines for users who choose to use UUIDs as the member ID:

UUID Generation Specifications

...

The consumer instance must generate a UUID to be used as the member ID, and this ID should remain consistent for the duration of the consumer's session. Here, a "session" is defined as the period from the consumer's first heartbeat until it leaves the group, either through a graceful shutdown, a heartbeat timeout, or the process stopping or dying. The consumer instance should reuse the same UUID as the member ID for all heartbeats and rejoin attempts to maintain continuity within the group.

If a conflict arises where the member ID (UUID) generated by the client is detected to be a duplicate within the same group (for example, the same UUID member ID is associated with another active member in the group), the server will handle this by comparing the memberEpoch values of the conflicting members. The member with the lower memberEpoch is considered outdated and will be fenced off by the server. When this occurs, the server responds with a FENCED_MEMBER_EPOCH error to the client, signaling it to rejoin the group with the same member ID while resetting the memberEpoch to zero. This ensures that the client properly resynchronizes and maintains the continuity and consistency of the group membership.

...

The objective of the test is to ensure that the client using the newer protocol correctly and securely generates a UUID and provides this UUID as the member ID.
At the same time, ensure backward compatibility with older versions of the  ConsumerGroupHeartbeat RPC.
According the objective, we should have the test scenarios as the followings:

...

• Retry Mechanism
  • Ensure the client retries with the same UUID member ID if a heartbeat request fails.
• Consistency
  • Test that the client maintains the same UUID member ID throughout the entire session and doesn't generate a new UUID member ID midway.
• Error Handling
  • No member ID is provided by the client: The server should reject the request with an InvalidRequestException and log an error indicating the missing member ID.An invalid UUID is provided: The server should validate the UUID format and reject the request with an InvalidRequestException if the UUID does not conform to the expected format. The client should handle this by generating a correct UUID and retrying the request.
  • A member ID mismatch occurs within a session: If the server detects a mismatch between the provided member ID and the expected member ID for an ongoing session, it should return a UNKNOWN_MEMBER_ID  error.
• Backward Compatibility
  
  • Ensure that clients using older versions of the RPC (where the server generates the member ID) still function correctly.
• Mixed-Version
  • Test scenarios where multiple clients, using both the new and old versions RPC, are communicating with the server.

...