Page History

...

These requests are used by clients to participate in a client group managed by Kafka. From a high level, each group in the cluster is assigned one the brokers (its group coordinator) to facilitate group management. Once the coordinator has been located, group members can join the group and synchronize state, and then use heartbeats to stay active in the group. When the client shuts down, it uses a leave group request to deregister from the group. More detail on protocol semantics is outlined in Kafka Client-side Assignment Proposal.

Group Coordinator Request

...

GroupCoordinatorResponse => ErrorCode Coordinator
  ErrorCode => int16
  Coordinator => BrokerId Host Port
    BrokerId => int32
    Host => string
    Port => int32

Join Group Request

The join group request is used by a client to become a member of a group. When a member first joins the group, the memberId will be empty (i.e. ""), but if a member is rejoining the group, then it should use the same memberId from the previous generation. The details of the protocol fields are described in more detail in Kafka Client-side Assignment Proposal.

JoinGroupRequest => GroupId SessionTimeout MemberId ProtocolType GroupProtocols
  GroupId => 

JoinGroupRequest => GroupId SessionTimeout MemberId ProtocolType GroupProtocols
  GroupId => string
  SessionTimeout => int32
  MemberId => 

...

 => string
  ProtocolType => string
  GroupProtocols => [ProtocolName ProtocolMetadata]
    ProtocolName => string
    ProtocolMetadata => bytes

Join Group Response

One of the group members will be elected the leader for each generation. The leader will receive the full list of members along with the associated metadata for the protocol chosen by the coordinator. Other members, followers, will receive an empty array of members. It is the responsibility of the leader to inspect the metadata of each member and assign state using SyncGroup request below.

JoinGroupResponse => ErrorCode GenerationId GroupProtocol LeaderId MemberId Members
  ErrorCode => int16
  GenerationId => int32
  GroupProtocol => string
  LeaderId => string
  MemberId => string
  Members => [MemberId MemberMetadata]
    MemberId => string
    MemberMetadata => bytes

SyncGroup Request

The sync group request is used by the leader to assign state (e.g. partition assignments) to all members of the current generation. All members send SyncGroup immediately after joining the group, but only the leader provides the group's assignment.

SyncGroupRequest => GroupId GenerationId MemberId GroupAssignment
  GroupId => string
  GenerationId => int32
  MemberId => string
  GroupAssignment => [MemberId MemberAssignment]
    MemberId => string
    MemberAssignment => bytes

Sync Group Response

Each member in the group will receive an assignment from the leader in the sync group response. 

SyncGroupResponse => ErrorCode MemberAssignment
  ErrorCode => int16
  MemberAssignment => bytes

Heartbeat Request

Once a member has joined and synced, it will begin sending periodic heartbeats to keep itself in the group. If not heartbeat has been received by the coordinator with the configured session timeout, the member will be kicked out of the group.

HeartbeatRequest => GroupId GenerationId MemberId
  GroupId => string
  GenerationId => int32
  MemberId => string  

Heartbeat Response

HeartbeatResponse => ErrorCode
  ErrorCode => int16

LeaveGroup Request

To explicitly leave a group, the client can send a leave group request. This is preferred over letting the session timeout expire since it allows the group to rebalance faster, which for the consumer means that less time will elapse before partitions can be reassigned to an active member.

LeaveGroupRequest => GroupId MemberId
  GroupId => string
  MemberId => string

LeaveGroup Response

LeaveGroupResponse => ErrorCode
  ErrorCode => int16

Administrative API

ListGroups Request

This API can be used to find the current groups managed by a broker. To get a list of all groups in the cluster, you must send ListGroup to all brokers.

ListGroupsRequest => 

ListGroups Response

ListGroupsResponse => ErrorCode Groups
  ErrorCode => int16
  Groups => [GroupId ProtocolType]
    GroupId => string
    ProtocolType => string

DescribeGroups Request

DescribeGroupsRequest => [GroupId]
  GroupId => string

DescribeGroups Response

DescribeGroupsResponse => [ErrorCode GroupId State ProtocolType Protocol Members]
  ErrorCode => int16
  GroupId => string
  State => string
  ProtocolType => string
  Protocol => string
  Members => [MemberId ClientId ClientHost MemberMetadata MemberAssignment]
    MemberId => string
    ClientId => string
    ClientHost => string
    MemberMetadata => bytes
    MemberAssignment => bytes  

Constants

Api Keys

The following are the numeric codes that the ApiKey in the request can take for each of the above request types.

...