Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleMonitorable.java
package org.apache.kafka.common.metrics;

public interface Monitorable {

    /**
     * Set the PluginMetrics instance from the clientcomponent that instantiates the plugin.
     */
    void withPluginMetrics(PluginMetrics metrics);

}

...

The PluginMetrics interface implements Closeable. Calling the close() method removed all metrics and sensors created by this plugin. It will be the responsibility of the plugin that create creates metrics to call close() of their the PluginMetrics instance they were given to remove their metrics.

New methods will be added to AbstractConfig so new plugin instances implementing Monitorable can get a PluginMetrics instance.

...

For Connectors and Converters, the name of the connector (connector) will be added as a tag (connector). Tasks will add the name of the connector name (connector) and the task id (task) added as tags. Transformations and Predicates will have the connector name, the task id and their alias (aliastransformation/predicate) added as tags. For example:

  • for a task: kafka.connect:type=plugins

...

  • ,connector=my-sink,task=0
  • for a predicate: kafka.connect:type=plugins,connector=my-sink,task=0,predicate=my-predicate

For configurations that accept a list of classes, for example interceptor.classes, if the same class is provided multiple times, their metrics may collide. This is deemed highly unlikely to occur as there are no use cases where providing multiple times the same class is useful.

This proposal supersedes KIP-608 which only aimed at providing this functionality to Authorizer implementationsplugins. KIP-608 was adopted in 2020 but never implemented. This KIP proposes to alter the names of the metrics from KIP-608 to the following so all plugins use the same naming format:

Full NameTypeDescription
kafka.server:type=plugins,config=authorizer.class.name,class=MyAuthorizer,name=acls-total-countGaugeTotal ACLs created in the broker
kafka.server:type=plugins,config=authorizer.class.name,class=MyAuthorizer,name=authorization-request-rate-per-minuteRateTotal number of authorization requests per minute
kafka.server:type=plugins,config=authorizer.class.name,class=MyAuthorizer,name=authorization-allowed-rate-per-minuteRateTotal number of authorization allowed per minute
kafka.server:type=plugins,config=authorizer.class.name,class=MyAuthorizer,name=authorization-denied-rate-per-minuteRateTotal number of authorization denied per minute 

Supported plugins

The goal is allow this feature to be used by all plugins that are Closeable, AutoCloseable or have a close() methods apart from MetricsReporter since instances are created before the Metrics instance. Also all Connect connector plugins will be supported. The javadoc of all supported plugins will be updated to mention they are able to implement the Monitorable interface to define their own metrics.

...