You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Status

Current state: Draft (WIP).

Discussion thread: here

Vote thread: here

JIRA: KAFKA-7772

Released: AK x.y.z

Pull request: https://github.com/apache/kafka/pull/6069

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Connect currently does not offer a way to dynamically change log levels. This means debugging an issue with a connector or the framework can involve multiple cycles of "change log4j.properties file and restart connect". This proposal presents a feature to adjust log levels in Connect.

JMX based adjustments in Kafka Brokers

Kafka brokers contain a Log4jController utility that allows log level adjustment from a JMX tool (jconsole, for example). The screenshot below shows how JConsole can be used to navigate to the Kafka mbean, from where log levels can be found out using the getLogLevel operation, or set through setLogLevel. 

Proposed Approach

We propose the following approach:

  1. Rewrite the JMX logging utilities (kafka.utils.Log4jController) in Java, and move them from kafka-core to clients (the org.apache.kafka.common package in clients module), which holds other common utilities including but not limited to ConfigDef, Exceptions and Metrics. 
  2. Kafka and Connect will initialize this module on startup, and invoke helper functions to add "List Log Levels", "Set Log Level" and "Get Log Level" functionality into their JMX interface. 

Public Interfaces

Expose the following new operations via the kafka.connect.Log4jController mbean:

  • getLogLevel: that accepts a classname, and returns the effective log4j log level.
  • setLogLevel: that sets the given log level for a class.

Since there are standard tools to interface with JMX, we will not be adding any new tools or scripts to access the proposed functionality. The existing kafka.Log4jController mbean will not be modified.

Rejected Alternatives

  • Changing log levels in a single node of a component will affect other nodes in the cluster (for example, changing log level of a class in one Connect worker will update levels in all workers in a Connect cluster) and this new level will be persisted across node restarts. This is beyond the scope of this proposal.
  • Rest extensions were rejected, as Kafka doesn't offer a REST server and we are aiming for a consistent experience across Kafka and Connect in this proposal.
  • No labels