Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

NameContextTypeDescription
kafka.controller:type=KafkaController,name=TimedOutBrokerHeartbeatCountControllerLong

The number of broker heartbeats that timed out on this controller since the process was started. Note that only active controllers handle heartbeats, so only they will see increases in this metric.

kafka.controller:type=KafkaController,name=EventQueueOperationsPerformedCountControllersLongThe total number of event queue operations that were performed. This includes deferred operations.
kafka.controller:type=KafkaController,name=EventQueueOperationsTimedOutCountControllersLongThe total number of event queue operations that timed out before they could be performed.
kafka.controller:type=KafkaController,name=NewActiveControllersCountControllerLongCounts the number of times this node has seen a new controller elected. A transition to the "no leader" state is not counted here. If the same controller as before becomes active, that still counts.
kafka.server:type=MetadataLoader,name=CurrentMetadataVersionBroker and ControllerIntegerOutputs the feature level of the current effective metadata version.
kafka.server:type=MetadataLoader,name=HandleLoadSnapshotCountBroker and ControllerLongThe total number of times we have loaded a KRaft snapshot since the process was started.
kafka.server:type=MetadataLoader,name=LatestSnapshotGeneratedBytesBroker and ControllerLongThe total size in bytes of the latest snapshot that the node has generated. If none have been generated yet, this is the size of the latest snapshot that was loaded. If no snapshots have been generated or loaded, this is 0.
kafka.server:type=MetadataLoader,name=LatestSnapshotGeneratedAgeMsBroker and ControllerLongThe interval in miliseconds since the latest snapshot that the node has generated. If none have been generated yet, this is approximately the time delta since the process was started.
kafka.server:type=ForwardingManager,name=QueueTimeMsBrokerHistogramA histogram describing the amount of time in milliseconds each admin request spends in the broker's forwarding manager queue, waiting to be sent to the controller. This does not include the time that the request spends waiting for a response from the controller.
kafka.server:type=ForwardingManager,name=QueueLengthBrokerIntegerThe current number of RPCs that are waiting in the broker's forwarding manager queue, waiting to be sent to the controller.
kafka.server:type=ForwardingManager,name=RemoteTimeMsBrokerHistogramA histogram describing the amount of time in milliseconds each request sent by the ForwardingManager spends waiting for a response. This does not include the time spent in the queue.

Implementation Notes

Lockless

In order to avoid excessive performance impacts from these new metrics, none of them will require locks to read. (Except for any locks inside the Yammer library, JMX implementation, and so on.)

Histogram details

Histograms have the standard Yammer MBean fields of 50thPercentile, 75thPercentile, 95thPercentile, 98thPercentile, 999thPercentile, 99thPercentile, Count, Max, Mean, Min, StdDev.

Rationale

TimedOutBrokerHeartbeats

...

This metric is useful to monitor how long it has been since the node last generated a snapshot. If this time grows too large, it may indicate a potential problem, since loading times might also become very large.

ForwardingManager QueueTimeMs, QueueLength, RemoteTimeMs

These metrics are useful to understand how much of the latency of admin commands is due to waiting for the forwarding manager, versus other causes.

Compatibility, Deprecation, and Migration Plan

...