Versions Compared

Old Version 5

changes.mady.by.user David Jacot

Saved on

compared with

New Version Current

changes.mady.by.user David Jacot

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Status

Current state: "Under Discussion" Accepted

Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

When a consumer leave the consumer group, it logs the reason. However, there is not information on the broker side which makes troubleshooting rebalances triggered by LeaveGroupRequest hard. Propagating the reason to the broker via the LeaveGroupRequest would be really helpful in such situations.

Public Interfaces

Protocol

A new field called Reason is added to each member in the JoinGroupRequest. 

...

Code Block
languagejs
{
  "apiKey": 13,
  "type": "request",
  "listeners": ["zkBroker", "broker"],
  "name": "LeaveGroupRequest",
  // Version 1 and 2 are the same as version 0.
  //
  // Version 3 defines batch processing scheme with group.instance.id + member.id for identity
  //
  // Version 4 is the first flexible version.
  //
  // Version 5 adds the Reason field (KIP-800).  
  "validVersions": "0-5",
  "flexibleVersions": "4+",
  "fields": [
    { "name": "GroupId", "type": "string", "versions": "0+", "entityType": "groupId",
      "about": "The ID of the group to leave." },
    { "name": "MemberId", "type": "string", "versions": "0-2",
      "about": "The member ID to remove from the group." },
    { "name": "Members", "type": "[]MemberIdentity", "versions": "3+",
      "about": "List of leaving member identities.", "fields": [
      { "name": "MemberId", "type": "string", "versions": "3+",
        "about": "The member ID to remove from the group." },
      { "name": "GroupInstanceId", "type": "string",
        "versions": "3+", "nullableVersions": "3+", "default": "null",
        "about": "The group instance ID to remove from the group." },
      // New Field //
      { "name": "Reason", "type": "string",
        "versions": "45+", "nullableVersions": "45+", "default": "null",
        "about": "The reason why the member left the group." }
    ]}
  ]
}

Admin API

The reason is added as an argument to removeMembersFromConsumerGroup optional attribute to RemoveMembersFromConsumerGroupOptions class which is used by the Admin#removeMembersFromConsumerGroup method.

Code Block
languagejava
/**
 * Remove members from the consumer group by given member identities.
 * <p>
 * For possible error codes, refer to {@link LeaveGroupResponse}.
 *
 * @param groupId The ID of the group to remove member from.
 * @param reason The reason why the member left the group.
 * @param options The options to carry removing members' information.
 * @return The MembershipChangeResult.
 */
RemoveMembersFromConsumerGroupResult removeMembersFromConsumerGroup(String groupId, String reason, RemoveMembersFromConsumerGroupOptions options);@InterfaceStability.Evolving
public class RemoveMembersFromConsumerGroupOptions extends AbstractOptions<RemoveMembersFromConsumerGroupOptions> {
  /**
   * Sets an optional reason.
   */
  void reason(final String reason);
  
  /**
   * Returns the optional reason or an empty String if not defined.
   */
  String reason();
}

Consumer API

The reason is added as an argument to enforceRebalance method.

Code Block
languagejava
/**
 * @see KafkaConsumer#enforceRebalance()
 */
void enforceRebalance(final String reason);

Proposed Changes

The KIP adds a per-member field called Reason to the LeaveGroupRequest. The field will be populated with the leaving reason by the Consumer when it wants to leave the group. The consumer already logs the reason so we will reuse the same message. The field will be populated by the AdminClient as well when removeMembersFromConsumerGroup is called. In this case, the reason will be "member was removed by an admin" by default or the provided reason prefixed by "member was removed by an admin".

The KIP also adds a field called Reason to the JoinGroupRequest. The field will be populated with the reason why the joining reason by the Consumer. This will be really helpful to understand why a rebalance is triggered. The field will be populated by the Consumer as well when enforceRebalance is called. In this case, the reason will be "rebalance enforced by user" by default or the provided reason prefixed by "rebalance enforced by user:".

Compatibility, Deprecation, and Migration Plan

The new field defaults to null and is optional so compatibility is not a concern.

Rejected Alternatives

The alternative would be to do nothing and to keep relying on client logs in this case.