...

In May 2012, the log4j dev team released log4j 1.2.17 and stopped their support to 1.x releases. And from then on, Apache Kafka is still using it in its core and the other submodules. Since this release has not only several critical subprojects.

The problem caused by the obsolete log4j version is not limited to security problems like CVE-2019-17571 but also an obsolete interface, it should be replaced into log4j2. Most users are now familiar with log4j2 configurations, not log4j. But when they want to customize the logging configuration of Apache Kafka, Kafka Connect, and Kafka Streams application, they have to work with outdated, dismissed old configuration format.

Public Interfaces

This KIP proposes the following:

...

For backward compatibility, Kafka broker will use the log4j configuration file by default. But for informational purpose, the following message will be shown in default mode:

DEPRECATED: using log4j 1.x configuration. To use log4j 2.x configuration, run with: 'export KAFKA_LOG4J_OPTS="-Dlog4j.configurationFile=file:$base_dir/../config/log4j2.properties"'

As the message above states, the user can
If the user wants to run Kafka broker with lgo4j2 log4j2 config file , they can do it by setting `export KAFKA_LOG4J_OPTS="-Dlog4j.configurationFile={log4j2-config-file-path}"`. A . Thanks to log4j12-api, a compatibility bridge between log4j and log4j2, Kafka broker can be run without any changes. Since a log4j2 equivalent for traditional built-in log4j config (log4j2.properties) will be provided, the user can make use of it if they want.

The test logging configuration (src/test/resources/log4j.properties) will be updated into log4j2. In this case, we don't care about the backward-compatibility.

2. Client

Use log4j2 configurationThe test logging configuration (src/test/resources/log4j.properties) will be updated into log4j2.

3. Connect

Like core, connect will use the log4j configuration file by default. A log4j2 equivalent for traditional log4j config (connect-log4j2.properties) will be also provided. The users can use it by setting `export it will show the following message also if it is in default mode:

DEPRECATED: using log4j 1.x configuration. To use log4j 2.x configuration, run with: 'export KAFKA_LOG4J_OPTS=\"-Dlog4j.configurationFile=file:$base_dir/../config/connect-log4j2.properties\"'

A log4j2 equivalent for traditional log4j config (connect-log4j2.properties) will be also provided, like other subprojects{log4j2-config-file-path}"`.

Add to this, the changes between log4j and log4j2 introduces the following changes:

• From log4j2, the name of the root logger becomes empty string from 'root'. It impacts Kafka connect's dynamic logging control feature. (And should be documented.)
• From log4j2, the logging level can't be null; GET `/admin/loggers` has been returning loggers with a non-null level (i.e., includes loggers with OFF level) but from log4j2, it returns the loggers with non-OFF levels only.
  
• If the user sets the root logger with PUT `/admin/loggers/{logger}`, all descendant loggers without the null level were affected. From log4j2, they will be affected since null level is regared as OFF level.

Like core, the test logging configuration (src/test/resources/log4j.properties) will be updated into log4j2.

4. Streams

Archetype log4j configuration will be updated into log4j2 equivalent (log4j2.properties).

...

Its log4j2 counterpart should be provided in near future. However, it is above the scope of this KIP.

6. Tools

Like core and connect, it will run with log4j configuration by default, but the users can use it by setting `export KAFKA_LOG4J_OPTS="-Dlog4j.configurationFile={log4j2-config-file-path}"`. A log4j2 equivalent for traditional log4j config (tools-log4j2.properties) will also be provided.

...

Compatibility, Deprecation, and Migration Plan

1. Core

Thanks to At some time or other,the default logging configuration format will be switched into log4j2. However, without removing log4j12-api , a compatibility bridge between log4j and log4j2, Kafka broker can be run without any changes; However, if the user runs `bin/kafka-server-start.sh ` or `bin/zookeeper-server-start.sh ` with log4j configuration it will show a deprecation message.it can also support log4j configuration format. In that point, The informational message will be also changed into:

Using log4j 2.x configuration. To use log4j 1.x configuration, run with: 'export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"'

2. Client

None.

3. Connect

Like core, if the user runs 'connect-standalone.sh' or 'bin/connect-distributed.sh' with log4j configuration it will show a deprecation message.Same to core.

4. Strams

None.

5. Log4j-appender

NoneThis subproject will be remained for backward-compatibility, when the default logging format is changed into log4j2. Instead, its log4j2 counterpart, log4j2-appender, will be provided at that moment.

6. Tools

Same as core and connect; Running 'bin/connect-mirror-maker.sh' with log4j configuration will show a deprecation message.

7. ETC

None.

Rejected Alternatives

...