Page History

kafka.server:type=ShareGroupMetrics,name=TotalShareFetchRequestsPerSec

kafka.server:type=ShareGroupMetrics,name=TotalShareFetchRequestsPerSec,topic={topic}

kafka.server:type=ShareGroupMetrics,name=FailedShareFetchRequestsPerSec

kafka.server:type=ShareGroupMetrics,name=FailedShareFetchRequestsPerSec,topic={topic}

kafka.server:type=ShareGroupMetrics,name=FailedShareAcknowledgementRequestsPerSec

kafka.server:type=ShareGroupMetrics,name=FailedShareAcknowledgementRequestsPerSec,topic={topic}

In shared consumption, topic-partitions are locked before fetching data to ensure exclusive access for each share fetch request. Multiple consumers can request the same partitions, but only a subset can acquire locks.

This metric represents the ratio of successfully fetched topic-partitions to the total requested in a share fetch. A low ratio indicates that many requests are unable to fetch from a significant portion of the requested partitions.

Potential Causes for Low Ratios:

• High Share Consumer Count: A large number of share consumers can lead to increased competition for partitions.
• Strict Locks Limit: A low group.share.partition.max.record.locks value can exclude partition a share consumer can acquire, if reached.
• Fewer Topic Partitions: Fewer partitions can intensify competition, especially with a high share consumer count.
• Suboptimal Partition Assignments: Inefficient assignments can hinder efficient consumption.

This metric, in conjunction with other metrics and configuration parameters, can help optimize shared consumption by adjusting consumer counts, partition limits, and partition assignments.

Example:

If the metric reports low values and:

• InFlightMessageCount is high: Consider increasing group.share.partition.max.record.locks to allow more locks per share partition.
• InFlightMessageCount is not high: Consider pptimizing optimizing partition assignments for cooperative consumption. If assignments are optimal and lock contention persists, increasing topic partitions or reducing the number of share consumers might improve performance.
• If the aforementioned optimizations have been implemented, further investigation should focus on the time taken to complete log fetches and acquire records. This includes the duration for which the fetch lock is held. The FetchLockTimeMs metric provides the total time spent in this process. High values indicate potential optimization opportunities in the efficiency of fetching and acquiring messages.

The ratio of topic-partitions acquired to the total number of topic-partitions in share fetch request.

Tracks the number of acquisition locks for records which are not
acknowledged within the timeout. Timeout is set by group.share.record.lock.duration.ms.

The number of acquisition locks for records which are not
acknowledged within the timeout.

The number of in-flight messages for the share partition.

The number of in-flight batches for the share partition.

The number of messages in the in-flight batch.

The time elapsed (in milliseconds) while a share partition is held under lock for fetching messages.

The share session eviction rate per second.

The number of cached share partitions.

The number of cached share sessions.

The expired delayed share fetch operation rate per second.

MessageConversionsPerSec

(ShareFetch added to existing metric for Produce and Fetch)

The message format conversion rate, for Produce, Fetch or ShareFetch requests, per topic. Omitting ‘topic={…}’ will yield the all-topic rate.

The number of requests waiting in the share fetch purgatory. This is high if share consumers use a large value for fetch.wait.max.ms

The number of delayed operations for share fetch purgatory.