Status

Current state: Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KAFKA-3853 asks for an improvement to the describe option of the consumer group command for new (Java API based) consumers. This command, when passed a consumer group that has no consumer (i.e., when the group state is Empty), currently reports an error indicating that there is no active member:

...

The OffsetFetch protocol can be used to extract offsets associated with given topic partitions in a consumer group. The problem is, when consumer group is in Empty state or even when it is Stable but not all its topics are being consumed, currently there is no way to extract all its topic partitions that it has consumed from so far. We can modify the behavior of OffsetFetch protocol so it returns all topic partitions if a null value is passed instead of a list of topic partitions. 

Public Interfaces

This is the current schema for OffsetFetch (version 1, that applies when fetching from Kafka, and not ZooKeeper).

...

The other benefit here is with one API call all offsets within the group are returned. Whereas, in the existing describe group implementation, for each topic partition in the group one API call is made.

Proposed Changes

The proposal is to

1. Add to the implementation of the OffsetFetch API the scenario where null is passed as the list of topic partitions, and in response, offsets of all topic partitions associated with the group are returned.
2. Add a committedOffsets interface to Java Consumer that makes use of the updated OffsetFetch API above and returns offsets of all topic partitions associated with the consumer's group. 

...

With respect to the added interface to Java Consumer there would be no issue as that interface does not exist in current implementation.

Rejected Alternatives

1. Changing the DescribeGroups protocol so it also returns the offset information for all topic partitions from which the group has consumed from since its creation. More detailed can be found here.