...
In addition to the above, clients and streams libraries will read in additional metrics.context.<field>=<value>
entries passed in via configurationand add them to the metric context before calling contextChange(MetricsContext metricsContext)
.
Kafka Brokers
Kafka brokers will pass in fully resolved fields kafka.broker.id
and kafka.cluster.id
to the metric context.
Connect Workers
Connect workers will pass in connect.kafka.cluster.id
to indicate the Kafka cluster used in distributed mode, as well as connect.group.id
to indicate the group id used for worker coordination, and also propagate this information to the its clients.
Compatibility, Deprecation, and Migration Plan
contextChange(MetricsContext metricsContext)
will be added as a default method to MetricsReporter
to support backward compatibility with existing implementations. No changes are required for existing metric reporters since we are not changing any other behavior.
...
Add additional metadata to the tags defined in
MetricsConfig
MetricsConfig
tags are tightly coupled toMetricNameTemplate
objects, and require metric names to provide placeholders for all the tags, which make it difficult to add new tags without changing the metric names.Add additional metadata to the ClusterResource class
Metrics reporters could already get some Kafka cluster metadata information (e.g, Kafka cluster id) by implementing the ClusterResourceListener interface introduced in KIP-78. We considered adding additional context metadata to this class, but ultimately rejected that idea. ClusterResource is tightly coupled to a Kafka cluster. Adding metadata of other components to ClusterResource would be ill-suited and cause for confusion. TheClusterResourceListener.onUpdate(ClusterResource)
method method also has well-defined semantics for broker and client metadata updates, which would make implementing MetricsReporters more difficult.Pass in additional context via the Configurable/Reconfigurable interface
As noted, some context metadata such asbroker.id
is currently available via configuration, but relies on workarounds to manipulate the config passed to metrics reporters when they are defined dynamically. While it might be possible to usereconfigure(...)
instead to pass context values when they are known, this would require metric reporters to know all possible context keys in advance to return inreconfigurableConfigs
. As a result, one would have to make code changes to reporter plugins with every release just to support new fields added to the context. That would be akin to asking metrics reporter to know what metric tags are exposed in advance, which seemed convoluted.