Versions Compared

Old Version 7

changes.mady.by.user Mahsa Seifikar

Saved on

compared with

New Version 8

changes.mady.by.user Mahsa Seifikar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The controller thread is responsible for critical cluster coordination tasks, including leadership elections, topic metadata updates, partition assignments, and Raft-based state changes. However, there is currently While we have metrics for event queue and processing durations — kafka.controller:type=ControllerEventManager,name=EventQueueTimeMs tracks the time an event waits in the queue before processing, and kafka.controller:type=ControllerEventManager,name=EventQueueProcessingTimeMs measures the time an event takes to be processed— there is no metric to measure how much time the controller spends actively processing work versus is idle either waiting for new events or waiting Raft callbacks. This lack of visibility makes it challenging to assess the controller’s performance, detect potential bottlenecks, or optimize resource allocation in high-load scenarios.

This KIP proposes introducing a new metric to monitor the controller thread’s idleness ratio.

Public Interfaces

Monitoring

NameTypeDescription
kafka.controller:type=ControllerEventManager,name=ControllerThreadIdleRatioGaugeThe idle ratio measures the proportion of time the controller thread is
idle relative to the total time (idle+active). Values range
not actively processing an event.

The metric is calculated using the following formula and value ranges from 0 to 1, where 0 indicates the controller thread is

fully busy and 1 indicates it is completely idle.

constantly processing without breaks, 1 signifies the controller spends most of its time waiting for work:


                      controller idle ratio = idle_time/(idle_time+active_time)


The components are defined as follows:

  • active_time: This is the total time the controller spends actively processing events. It is equivalent to EventQueueProcessingTimeMs metric, measuring the duration from when an event's processing begins until it completes.
  • idle_time: This is the time the controller thread spends waiting for either new events to appear in the queue or Raft callback.

The idle_time can be part of active_time in specific scenarios - particularly when the controller has sent records to the Raft layer and is awaiting commitment confirmation, but not when simply waiting for new events to enter the queue. 

Compatibility, Deprecation, and Migration Plan

...