Status

Current state: Accepted

Under Discussion thread: here
Discussion

Vote thread: here

JIRA:

Motivation

All Kafka component components register AppInfo   metrics to track the application details such as start time or and commit-id etc. These metrics are useful valuable for monitoring and debugging. However, the AppInfo doesn't provide AppInfo does not include the client-id for Worker and MM2 clients, which is an important piece of information for custom metrics reporter. reporters. While most Kafka clients (consumer, producer, admin) already register their client-id in the metrics config, there are cases in MM2 and Worker where AppInfo is registered without the client-id.

The AppInfoParser As the AppInfoParser  class registers a JMX MBean with the provided client-id, but when it adds adding metrics to the Metrics registry, the client-id is not included. This KIP proposes adding the client-id as a metric tag.

Public Interfaces

Currently MetricName for Worker and MM2 clients:

• [name=start-time-ms, group=app-info, description=Metric indicating start-time-ms, tags={}]
• [name=commit-id, group=app-info, description=Metric indicating commit-id, tags={}]
• [name=version, group=app-info, description=Metric indicating version, tags={}]

New MetricName for Worker and MM2 clients:

• [name=start-time-ms, group=app-info, description=Metric indicating start-time-ms, tags={client-id=<component-id>}]
• [name=commit-id, group=app-info, description=Metric indicating commit-id, tags={client-id=<component-id>}]
• [name=version, group=app-info, description=Metric indicating version, tags={client-id=<component-id>}]org.apache.kafka.common.utils.AppInfoParser.AppInfoMBean

Proposed Changes

1) Update the AppInfoMBean interface to The new MBean will include a new method getClientId().tag, client-id

publicprivate interfacestatic AppInfoMBean {
   void registerMetrics(Metrics metrics, AppInfo appInfo, String getVersion(clientId); {
    Stringif getCommitId(metrics == null) return;
    // Most Kafka Longclients getStartTimeMs(producer/consumer/admin);
 set the  String getClientId();
}

2) Deprecated the old AppInfoMBean implementation and marked it as @Deprecated.

@Deprecated(since = "4.2")
public static class DeprecatedAppInfo implements AppInfoMBean {
} 

3) Introduced a new implementation of AppInfoMBean.

public static class AppInfo implements AppInfoMBean {

    private final Long startTimeMs;

    public AppInfo(long startTimeMs) {
        this.startTimeMs = startTimeMs;
        log.info("Kafka version: {}", AppInfoParser.getVersionclient-id tag in the metrics config.
    // Although we don’t explicitly parse client-id here, these metrics are automatically tagged with client-id.
    metrics.addMetric(metricName(metrics, "version", Map.of()), (Gauge<String>) (config, now) -> appInfo.getVersion());
    metrics.addMetric(metricName(metrics, "commit-id", Map.of()), (Gauge<String>) (config, now) -> appInfo.getCommitId());
    metrics.addMetric(metricName(metrics, "start-time-ms", Map.of()), (Gauge<Long>) (config, now) -> appInfo.getStartTimeMs());
    // MirrorMaker/Worker doesn't  log.info("Kafka commitId: {}", AppInfoParser.getCommitId());
   set client-id tag into the metrics config, so we need to set it here.
     log.info("Kafka startTimeMs: {}", startTimeMs);if (!metrics.config().tags().containsKey("client-id") && clientId != null) {
        logmetrics.info("Kafka client id: {}", AppInfoParser.getClientIdaddMetric(metricName(metrics, "version", Map.of("client-id", clientId)), (Gauge<String>) (config, now) -> appInfo.getVersion());
    }

	// skip...

    @Override    metrics.addMetric(metricName(metrics, "commit-id", Map.of("client-id", clientId)), (Gauge<String>) (config, now) -> appInfo.getCommitId());
    public String getClientId() {
        return AppInfoParser.getClientId() metrics.addMetric(metricName(metrics, "start-time-ms", Map.of("client-id", clientId)), (Gauge<Long>) (config, now) -> appInfo.getStartTimeMs());
    }
}

42) When AppInfoParser registers unregisters an MBean, it will register both also remove the new and the deprecated implementationsMBean.

publicprivate static synchronized void registerAppInfo(String prefix, String id, unregisterMetrics(Metrics metrics, longString nowMsclientId) {
    tryif {
(metrics == null) return;

     // skip...metrics.removeMetric(metricName(metrics, "version", Map.of()));
    metrics.removeMetric(metricName(metrics, "commit-id", Map.of()));
  DeprecatedAppInfo deprecatedMBean = new DeprecatedAppInfo(nowMs metrics.removeMetric(metricName(metrics, "start-time-ms", Map.of()));

    if (!metrics.config().tags().containsKey("client-id") && clientId AppInfo mBean != new AppInfo(nowMs);null) {
        servermetrics.registerMBeanremoveMetric(metricName(deprecatedMBeanmetrics, name);
        server.registerMBean(mBean, name);
"version", Map.of("client-id", clientId)));
        registerMetricsmetrics.removeMetric(metricName(metrics, mBean); // prefix will be added later by JmxReporter
    } catch (JMException e) {"commit-id", Map.of("client-id", clientId)));
        logmetrics.warn("Error registering AppInfo mbean", eremoveMetric(metricName(metrics, "start-time-ms", Map.of("client-id", clientId)));
    }
}

Compatibility, Deprecation, and Migration Plan

For users currently relying on the AppInfoMBean metric name without client-id tag, it will remain available until Kafka 5.0. New users are encouraged to use the new AppInfoMBeanmetric name, which includes the client-id tag. This deprecation will be documented in upgrade.html.

In Kafka 5.0, the deprecated AppInfoMBean following metric name will be removed. 

• [name=start-time-ms, group=app-info, description=Metric indicating start-time-ms, tags={}]
• [name=commit-id, group=app-info, description=Metric indicating commit-id, tags={}]
• [name=version, group=app-info, description=Metric indicating version, tags={}]

Test Plan

change unit test or Integration test to verify the new method.

...