...

Current state: Under discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here KAFKA-9130 [Change the link from KAFKA-1 to your own ticket]

...

At the moment, the ListGroups API always returns all groups. There are metrics that track the number of groups in several states but the only way to know the state of a group is to describe it using the DescribeGroups API. In large clusters with hundreds or thousands of groups, it can be hard for a cluster administrator to identify which groups are stable or which groups are dead and can be deleted for example. Basically this requires describing a large number of groups which is inefficiency in terms of time for the administrator and load for the cluster. For example listing stable groups allows to immediately tell if several applications are healthy if their groups are all returned without needing to describe each group. This would allow building tools and user interfaces that can easily and quickly display informations about live groups.

Public Interfaces

ListGroups API

Update ListGroupsRequest to include a new flexible field "states".

...

At the moment, consumer group state is already serialized as a STRING in DescribeGroupsResponse, so serializing it the same way here. ListGroupsResponse is left untouched.

AdminClient API

To enable expose this feature in the AdminClient API, ListConsumerGroupsOptions will be updated.

public class ListConsumerGroupsOptions extends AbstractOptions<ListConsumerGroupsOptions> {

    /**
     * Only groups in these states will be returned by listConsumerGroups()
     * By default, all groups are returned
     */
    public void inStates(List<ConsumerGroupState> states) {}

    public List<ConsumerGroupState> states() {}
}

Proposed Changes

• Update the ListGroups protocol definition
  
  • A new flexible field "states" is added to the request
  • The response stays the same
  • The broker will only return groups that are in the specified state. If no state is specified or version < 3, all groups are returned.

• Update the ListConsumerGroupsOptions object to expose this feature to the AdminClient
  • This allows to user to optionally specify a state groups should be in to be returned by the broker

...

Command line tools

To expose this feature in the command line tools, ConsumerGroupCommand tool will be updated to support filtering by group

...

states. A new

...

argument "--

...

states" will allow to specify a

...

list of group states. This new option will only be valid when used with "--list".

Compatibility, Deprecation, and Migration Plan

...