Status

Current state: Under DiscussionAccepted (vote thread)

Discussion thread: here

JIRA: https://issues.apache.org/jira/browse/KAFKA-9106

...

Two new configurations will be introduce to allow filtering the set of metrics exposed via JMX on brokers and clients.

The configurations would apply to brokers, as well as clients, streams, and connect.On the broker, the configurations will also be marked as dynamic to allow updates at runtime.On the broker, the configs will be defined as follows:

kafka.metrics.jmx.whitelist=<regular expression of mbeans to expose> (defaults to ".*")

kafka.metrics.jmx.blacklist=<regular expression of mbeans to hide> (defaults to "", takes precedence over the whitelist)

Similarly, for clients, streams, and connect, configuration will follow existing client metric config prefixes defined in CommonClientConfigs

metrics.jmx.whitelist=<regular expression of mbeans to expose> (defaults to ".*")

metrics.jmx.blacklist=<regular expression of mbeans to hide> (defaults to "", takes precedence over the whitelist)

These configurations will apply to both JMX metrics exposed via Yammer metrics (only applies to brokers) as well as JMX metrics exposed via Kafka's own metrics library.

By default, this would not change the set of metrics exposed via JMX: all metrics would be exposed.On the broker, updates to the configurations at runtime would both add and remove JMX metrics as needed to reflect the new filters.

All metrics continue to be collected in the background regardless of those settings, so this only affects the visibility in JMX and not the actual metric values.

Brokers specifics:

• Configurations will apply to JMX metrics exposed via Yammer metrics as well Kafka's internal metrics library.

• Configurations will be marked as dynamic to allow updates at runtime: adding and removing JMX metrics as needed to reflect the new filters.

Clients, streams, and connect:

• Configuration will apply to all the metrics exposed in JMX metrics via Kafka's internal metrics library.

Example

To hide log-level metrics LogEndOffset, LogStartOffset, and NumLogSegments, as well as any partition level metrics, except  UnderMinIsr, one could exclude those by specifying the following property in their server.properties file:

kafka.metrics.jmx.blacklist=(kafka.log:type=Log,name=(LogEndOffset|LogStartOffset|NumLogSegments),.*\

...