You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

 

Status

Current state: "Under Discussion"

Discussion thread: here

JIRA: here  Unable to render Jira issues macro, execution error.


Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

This KIP is aim to add support for describing consumer groups and list consumer groups to `KafkaAdminClient` class. This functionality is required on the Streams Resetter Tool as describe here:  Unable to render Jira issues macro, execution error.

Public Interfaces

API


Add the following API in AdminClient:

public abstract class AdminClient implements AutoCloseable {
 	
	public abstract DescribeGroupsResult describeGroups(Collection<String> groupIds, DescribeGroupsOptions options);
 
	public DescribeGroupsResult describeGroups(Collection<String> groupIds) {
    	return describeGroups(groupIds, new DescribeGroupsOptions());
	}
 
    public abstract ListGroupsResult listGroups(ListGroupsOptions options);

    public ListGroupsResult listGroups() {
        return listGroups(new ListGroupsOptions());
    }

    public ListGroupsResult listGroups(ListGroupsOptions options) {
        //filtering groups by ConsumerProtocol.PROTOCOL_TYPE
    }

    public ListGroupsResult listGroups() {
        return listGroups(new ListGroupsOptions());
    }
}
 
public class DescribeGroupOptions extends AbstractOptions<DescribeGroupOptions> {
}
 
public class DescribeGroupResult {
    private final Map<String, KafkaFuture<GroupDescription>> futures;
    public DescribeGroupsResult(Map<String, KafkaFuture<GroupDescription>> futures) {
        this.futures = futures;
    }

    public Map<String, KafkaFuture<GroupDescription>> values() {
        return futures;
    }

    public KafkaFuture<Map<String, GroupDescription>> all() {
        return KafkaFuture.allOf(futures.values().toArray(new KafkaFuture[0])).
            thenApply(new KafkaFuture.Function<Void, Map<String, GroupDescription>>() {
                @Override
                public Map<String, GroupDescription> apply(Void v) {
                    Map<String, GroupDescription> descriptions = new HashMap<>(futures.size());
                    for (Map.Entry<String, KafkaFuture<GroupDescription>> entry : futures.entrySet()) {
                        try {
                            descriptions.put(entry.getKey(), entry.getValue().get());
                        } catch (InterruptedException | ExecutionException e) {
                            // This should be unreachable, because allOf ensured that all the futures
                            // completed successfully.
                            throw new RuntimeException(e);
                        }
                    }
                    return descriptions;
                }
            });
    }
}
 
public class GroupDescription {
    private final String groupId;
    private final String protocolType;
    private final List<ConsumerDescription> consumers;
}
 
public class MemberDescription {
    private final String consumerId;
    private final String clientId;
    private final String host;
    private final List<TopicPartition> assignment;
}

public class ListGroupsOptions extends AbstractOptions<ListGroupsOptions> {
}

public class ListGroupsResult {
    final KafkaFuture<Map<String, GroupListing>> future;

    ListGroupsResult(KafkaFuture<Map<String, GroupListing>> future) {
        this.future = future;
    }

    public KafkaFuture<Map<String, GroupListing>> namesToListings() {
        return future;
    }

    public KafkaFuture<Collection<GroupListing>> listings() {
        return future.thenApply(new KafkaFuture.Function<Map<String, GroupListing>, Collection<GroupListing>>() {
            @Override
            public Collection<GroupListing> apply(Map<String, GroupListing> namesToDescriptions) {
                return namesToDescriptions.values();
            }
        });
    }

    public KafkaFuture<Set<String>> names() {
        return future.thenApply(new KafkaFuture.Function<Map<String, GroupListing>, Set<String>>() {
            @Override
            public Set<String> apply(Map<String, GroupListing> namesToListings) {
                return namesToListings.keySet();
            }
        });
    }
}
 
public class GroupListing {
    private final String name;
    private final String protocolType;
}
 

This API returns a future object whose result will be available within RequestTimeoutMs, which is configured when user constructs the AdminClient.

Proposed Changes

  1. Add `#describeGroups(Collection<String> groupIds, DescribeGroupsOptions options)` and `#describeGroups(Collection<String> groupIds)` to `AdminClient` API 
  2. Add `#listGroups(ListGroupsOptions options)` and `#listGroups()` to `AdminClient` API and 
  3. Implement it on `KafkaAdminClient`.
  4. Solve  Unable to render Jira issues macro, execution error.

Compatibility, Deprecation, and Migration Plan

  • There won't be any impact on existing users.
  • There won't be any change of current behavior.
  • No migration tool required.

Rejected Alternatives

We are moving one more functionality from `core`'s `AdminClient` API to `clients`'s `AdminClient` API. This is aim to remove dependencies to `core` module.

Future Work

  • Streams Resetter Tool will be remove it's dependency to `core` module for `KafkaAdminClient`.

 

  • No labels