Status

Current state: Under Discussion

Discussion thread: here

JIRA: KAFKA-13625

Motivation

This proposal is a following work of KIP-412, KIP-653, and KIP-719.

Adjusting log levels dynamically is so desired feature in maintenance, since it enables controlling logging messages without restart. As of present, Kafka supports two ways of this functionality:

A. using JMX: a traditional way.
B. using Admin: introduced in KIP-412. This approach does not require JMX authentication/authorization.

However, there are some inconsistencies between A and B:

1. A supports all log4j logging levels; but B does not support OFF and ALL.
2. If the user specifies an unsupported log level, A regards it as DEBUG; in contrast, B rejects the request with InvalidConfigurationException. Their semantics are slightly different from each other, the documentation does not mention it.

For 1, The OFF level was excluded in KIP-412 since it seemed there was no a compelling reason to turn off application logs. However, there are some cases when turning on/off the running logger. One good example is kafka.request.logger, which helps diagnose undergoing problems:

There is also some logging that may be useful to turn on when debugging issues with Kafka. One such logger is `kafka.request.logger`, turned on at either the DEBUG or TRACE levels. This logs information about every request sent to the broker. ... At either level, this logger generates a significant amount of data, and it is not recommended to enable it unless necessary for debugging. (Source: Gwen Shapira et al, Kafka: The Definitive Guide (2nd Edition), 2021)

Public Interfaces

A new log level definition, OFF, will be added to LogLevelConfig:

package org.apache.kafka.common.config;
 
public class LogLevelConfig {
 
    /**
     * The <code>TRACE</code> Level designates finer-grained
     * informational events than the <code>DEBUG</code level.
     */
    public static final String TRACE_LOG_LEVEL = "TRACE";
 
    /**
     * The <code>OFF</code> Level turns off the logger entirely.
     */
    public static final String OFF_LOG_LEVEL = "OFF"; // NEW

    public static final Set<String> VALID_LOG_LEVELS = new HashSet<>(Arrays.asList(
            FATAL_LOG_LEVEL, ERROR_LOG_LEVEL, WARN_LOG_LEVEL,
            INFO_LOG_LEVEL, DEBUG_LOG_LEVEL, TRACE_LOG_LEVEL,
            OFF_LOG_LEVEL  // NEW
    ));
}

Proposed Changes

  1. Calling Admin.IncrementalAlterConfigs with OFF log level will not raise INVALID_CONFIG (40) error (InvalidConfigurationException) anymore.
  2. LogLevelConfig's documentation will state the differences between JMX API and Admin API on handling unsupported log levels.

Compatibility, Deprecation, and Migration Plan

None.

Rejected Alternatives

Remove OFF level support from JMX API

This approach is not backward-compatible, also removing the desired feature in debugging.

Support ALL level also

We can use TRACE instead.

  • No labels