Current state: Under Discussion
Discussion thread: here
JIRA: KAFKA-13625
This proposal is a following work of KIP-412, KIP-653, and KIP-719.
Adjusting log levels dynamically is so desired feature in maintenance, since it enables controlling logging messages without restart. As of present, Kafka supports two ways of this functionality:
A. using JMX: a traditional way.
B. using Admin
: introduced in KIP-412. This approach does not require JMX authentication/authorization.
However, there are some inconsistencies between A and B:
1. A supports all log4j logging levels; but B does not support OFF
and ALL
.
2. If the user specifies an unsupported log level, A regards it as DEBUG; in contrast, B rejects the request with InvalidConfigurationException. Their semantics are slightly different from each other, the documentation does not mention it.
For 1, The OFF
level was excluded in KIP-412 since it seemed there was no a compelling reason to turn off application logs. However, there are some cases when turning on/off the running logger. One good example is kafka.request.logger
, which helps diagnose undergoing problems:
There is also some logging that may be useful to turn on when debugging issues with Kafka. One such logger is `kafka.request.logger`, turned on at either the DEBUG or TRACE levels. This logs information about every request sent to the broker. ... At either level, this logger generates a significant amount of data, and it is not recommended to enable it unless necessary for debugging. (Source: Gwen Shapira et al, Kafka: The Definitive Guide (2nd Edition), 2021)
A new log level definition, OFF
, will be added to LogLevelConfig
:
package org.apache.kafka.common.config; public class LogLevelConfig { /** * The <code>TRACE</code> Level designates finer-grained * informational events than the <code>DEBUG</code level. */ public static final String TRACE_LOG_LEVEL = "TRACE"; /** * The <code>OFF</code> Level turns off the logger entirely. */ public static final String OFF_LOG_LEVEL = "OFF"; // NEW public static final Set<String> VALID_LOG_LEVELS = new HashSet<>(Arrays.asList( FATAL_LOG_LEVEL, ERROR_LOG_LEVEL, WARN_LOG_LEVEL, INFO_LOG_LEVEL, DEBUG_LOG_LEVEL, TRACE_LOG_LEVEL, OFF_LOG_LEVEL // NEW )); } |
Admin.IncrementalAlterConfigs
with OFF
log level will not raise INVALID_CONFIG (40)
error (InvalidConfigurationException
) anymore.LogLevelConfig
's documentation will state the differences between JMX API and Admin
API on handling unsupported log levels.None.
This approach is not backward-compatible, also removing the desired feature in debugging.
We can use TRACE
instead.