Versions Compared

Old Version 1

changes.mady.by.user Mahsa Seifikar

Saved on

compared with

New Version Current

changes.mady.by.user Mahsa Seifikar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Status

Current state: Under DiscussionAccepted

Vote thread: https://lists.apache.org/thread/ztfo36jc26mt73whmr36d6x3xrjmyjdf

Discussion thread: https://lists.apache.org/thread/ty80q7dll4dlx3vnso7rlnlrd7c092fp

JIRA:

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-19791
: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here [Change the link from KAFKA-1 to your own ticket]

Motivation

The MetadataLoader thread is responsible for consuming metadata from Raft and processing critical metadata updates, including metadata commits, snapshot loading, leader changes, and publisher management.  Currently, there is no metric to measure how much time the MetadataLoader thread spends idle waiting for new metadata events versus actively processing themis a critical component in KRaft clusters that processes metadata changes from the Raft layer and distributes them to various publishers. However, unlike similar event-driven components such as the QuorumController (which has ControllerEventManager::AvgIdleRatio ) and various coordinators, the MetadataLoader lacks the idle thread ratio metric for monitoring its event queue processing efficiency. This lack of visibility makes it challenging to assess the metadata processing pipeline's performance, detect potential bottlenecks in metadata propagation, or optimize resource allocation when metadata update frequency varies significantly.

Public Interfaces

Monitoring

NameTypeDescription
kafka.
metadata
server:type=MetadataLoader,name=AvgIdleRatioTimeRatio

The idle ratio measures the proportion of time the MetadataLoader event queue thread is not actively processing metadata events


The

. This metric uses the existing org.apache.kafka.

raft

server.

internals

metrics.TimeRatio implementation, which tracks idleness over the actual measurement interval.

The metric

is calculated using the following formula and

's value ranges from 0 to 1, where 0 indicates the

MetadataLoader

thread is constantly processing

metadata

events without breaks, and 1 signifies the thread

spends most of its time

is almost always waiting for work.


Compatibility, Deprecation, and Migration Plan

This KIP introduces a single new metric , with no changes to without modifying existing behavior, APIs, or protocols. This new metric will be available immediately after upgrader. There will be no impact on existing Kafka versions

Test Plan

Unit tests will be added to verify the metrics.

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.NA