Page History

...

package org.apache.kafka.server.group.consumer;

public interface PartitionAssignor {

    class AssignmentSpec {
        /**
         * The members keyed by member id.
         */
        Map<String, AssignmentMemberSpec> members;

        /**
         * The topics' metadata keyed by topic id
         */
        Map<Uuid, AssignmentTopicMetadata> topics;
    }

    class AssignmentMemberSpec {
        /**
         * The instance ID if provided.
         */
        Optional<String> instanceId;

        /**
         * The rack ID if provided.
         */
        Optional<String> rackId;

        /**
         * The topics that the member is subscribed to.
         */
        Collection<String> subscribedTopics;

        /**
         * The current target partitions of the member.
         */
        Collection<TopicPartition> targetPartitions;
    }

    class AssignmentTopicMetadata {
        /**
         * The topic name.
         */
        String topicName;

        /**
		 * The number of partitions.
		 */
		int numPartitions; 
    }

    class GroupAssignment {
        /**
         * The member assignments keyed by member id.
         */
        Map<String, MemberAssignment> members;
    }

    class MemberAssignment {
        /**
         * The target partitions assigned to this member.
         */
        Collection<TopicPartition> targetPartitions;
    }

    /**
     * Unique name for this assignor.
     */
    String name();

    /**
     * Perform the group assignment given the current members and
     * topic metadata.
     *
     * @param assignmentSpec The assignment spec.
     * @return The new assignment for the group.
     */
    GroupAssignment assign(AssignmentSpec assignmentSpec) throws PartitionAssignorException;
}

Broker Metrics

Group Coordinator Runtime Metrics

kafka.server:type=group-coordinator-metrics,name=num-partitions,state=loading

• number of __consumer_offsets partitions in Loading state

kafka.server:type=group-coordinator-metrics,name=num-partitions,state=active

• number of __consumer_offsets partitions in Active state

kafka.server:type=group-coordinator-metrics,name=num-partitions,state=failed

• number of __consumer_offsets partitions in Failed state

kafka.server:type=group-coordinator-metrics,name=event-queue-size

• event accumulator queue size

partition load sensor: partition load time

• kafka.server:type=group-coordinator-metrics,name=partition-load-time-max
• kafka.server:type=group-coordinator-metrics,name=partition-load-time-avg

thread idle ratio sensor: thread busy - idle ratio

• kafka.server:type=group-coordinator-metrics,name=thread-idle-ratio-min
• kafka.server:type=group-coordinator-metrics,name=thread-idle-ratio-avg

Group Coordinator Metrics

Existing generic group metrics have been migrated, with the same metric names except for

NumGroups which reported the number of generic groups. This metric changed to

kafka.server:type=group-coordinator-metrics,name=consumergroup-groups-countcount,protocol={consumer|generic}

• number of consumer groupsgroups based on type where type is the rebalance protocol

kafka.server:type=group-coordinator-metrics,name=consumer-groupspartition-count,state=empty{loading|active|failed}

• number of consumer groups in Empty __consumer_offsets partitions based on state

kafka.server:type=group-coordinator-metrics,name=consumerevent-groups-count,state=assigningqueue-size

• event accumulator queue sizenumber of consumer groups in Assigning state

kafka.server:type=group-coordinator-metrics,name=consumer-groupsgroup-count,state={empty|assigning|reconciling|stable|dead}

• number of consumer groups in Reconciling based on state

consumer group rebalances sensor

• kafka.server:type=group-coordinator-metrics,name=consumer-group-

...

• number of consumer groups in Stable staterebalance-rate
• kafka.server:type=group-coordinator-metrics,name=consumer-

...

• group-rebalance-count

...

partition load sensor: __consumer_offsets partition load time

...

• number of consumer groups in Dead state

generic group preparing rebalance sensor

• kafka.server:type=group-coordinator-metrics,name=genericpartition-groupload-rebalancetime-ratemax
• kafka.server:type=group-coordinator-metrics,name=genericpartition-groupload-rebalancetime-countavg

thread idle ratio sensor: thread busy - idle ratioconsumer group rebalances sensor

• kafka.server:type=group-coordinator-metrics,name=consumerthread-groupidle-rebalanceratio-ratemin
• kafka.server:type=group-coordinator-metrics,name=consumerthread-groupidle-rebalanceratio-countavg

Broker Configurations

New properties in the broker configuration.

...