Versions Compared

Old Version 1

changes.mady.by.user Mahsa Seifikar

Saved on

compared with

New Version 2

changes.mady.by.user Mahsa Seifikar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Current state: Under Discussion

Discussion thread: here

JIRA:

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-19791
[Change the link from the KIP proposal email archive to your own email thread]
JIRA: here [Change the link from KAFKA-1 to your own ticket]

Motivation

The MetadataLoader thread is responsible for consuming metadata from Raft and processing critical metadata updates, including metadata commits, snapshot loading, leader changes, and publisher management.  Currently, there is no metric to measure how much time the MetadataLoader thread spends idle waiting for new metadata events versus actively processing them. This lack of visibility makes it challenging to assess the metadata processing pipeline's performance, detect potential bottlenecks in metadata propagation, or optimize resource allocation when metadata update frequency varies significantly.

Public Interfaces

Monitoring

NameTypeDescription
kafka.metadata:type=MetadataLoader,name=AvgIdleRatioTimeRatioThe idle ratio measures the proportion of time the MetadataLoader thread is not actively processing metadata events

The metric uses the existing org.apache.kafka.raft.internals.TimeRatio implementation which tracks idleness over the actual measurement interval.
The metric is calculated using the following formula and value ranges from 0 to 1, where 0 indicates the MetadataLoader thread is constantly processing metadata events without breaks, 1 signifies the thread spends most of its time waiting for work


Compatibility, Deprecation, and Migration Plan

...