This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Accepted"

Discussion thread: here

Voting thread: here

JIRA: KAFKA-7992

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

As with all software systems, having the proper observability into their health and well-being is critical for enabling their proper operation. Such information allows administrators to more appropriately take care of the system - be it reduce load, increase capacity or simply know to stay alert for problems.

In many deployment platforms (be them custom-built or open-source), tasks like restarting a misbehaving application in a cluster are completely automated.
With Kafka, monitoring systems have no definitive source of truth to gauge when a server/client has been started. They are left to either use arbitrary Kafka-specific metrics as a heuristic or the JVM's RuntimeMXBean's StartTime, which is not exactly indicative of when the application itself started.

It would be useful for Kafka to have a metric that exposes when it actually started. Similar systems like NATS, MongoDB and PostgreSQL all provide such a metric.

Public Interfaces

Add a new `start-time-ms` attribute to the component-specific app-info JMX metric.


MBean names
kafka.producer:client-id=<client-id>,type=app-info

kafka.consumer:type=app-info,client-id=<client-id>


kafka.server:type=app-info

kafka.admin.client:type=app-info,client-id=<client-id>


kafka.connect:type=app-info,client-id=<client-id>
Metric/Attribute NameDescription
start-time-msThe UNIX time at which this application started



Proposed Changes

We propose extending the already exposed `app-info` JMX metric with a new `start-time-ms` attribute. Said attribute would expose the unix time in milliseconds (e.g 1550990991562, corresponding to "Sun Feb 24 2019 06:49:51").

This has the added benefit of being exposed in the Kafka clients (producer, consumer, admin client) and Kafka Connect in addition to the Kafka broker.

Compatibility, Deprecation, and Migration Plan

This KIP simply adds a new metric attribute.

Rejected Alternatives

  • Add a single `startTimeMs` under the `kafka.server` domain
    • Does not standardize the metric across the other components comprising the Kafka ecosystem
  • No labels