Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Connect currently does not offer a way to dynamically change log levels. This means debugging an issue with a connector or the framework can involve multiple cycles of "change log4j.properties file and restart connect". This proposal presents a feature to adjust log levels in Connect.

JMX based adjustments

...

in Kafka Brokers

Kafka core contains brokers contain a Log4jController utility that allows log level adjustment from a JMX tool (jconsole or a rest interface using Jolokia, for example). The screenshot below shows how JConsole can be used to navigate to the Kafka mbean, from where log levels can be found out using the getLogLevel operation, or set through setLogLevel

Proposed Approach

Even though Connect has a REST interface and it is probably easier to implement this through a simple Connect Rest Extension (requires no new additional configs or any setup steps), there is a lot of value in a consistent user experience. Hence, we We propose the following approach:

  1. Rewrite the JMX logging utilities (kafka.utils.Log4jController) in Java, and move them from kafka-core to clients -common (the org.apache.kafka.common package in clients module), which holds other common utilities including but not limited to ConfigDef, Exceptions and Metrics. 
  2. Kafka and Connect depend on will initialize this module on startup, and invoke helper functions that to add "List Log Levels", "Set Log Level" and "Get Log Level" functionality into their JMX interface. 

...