Status

Current state: Adopted

Discussion thread: here

JIRA: Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-750, which specifies that Apache Kafka 4.0 will require Java 11, was approved in June 2021. This was a few months before Java 17 was released and well before there was clarity regarding the timeline of Apache Kafka 4.0.

A few things have changed since:

  1. Java 17 was released in September 2021 including Records (JEP 395), Sealed Classes (JEP 409), Switch Expressions (JEP 361), Text Blocks (JEP 378), Pattern Matching for instanceof (JEP 394), Stream.toList and more.
  2. Java 21 was released in September 2023. 
  3. Apache Kafka 4.0 is likely to be released in Q3 2024.
  4. Several popular projects have set Java 17 (or newer) as the minimum supported version or have announced plans to do so in the near future: Jetty 12, Apache Spark 4.0, Spring 6, Spring Boot 3, Quarkus 3.7, Apache Tomcat 11. This is also being considered for Apache Flink 2.0, but hasn't been voted on yet.
  5. Apache Kafka 3.7, due in January 2024, will include official docker images for the first time. This deployment model includes the Java runtime (Java 21 in the initial version) and it aims to abstract the Java version from users who do not run custom code within the kafka broker (the majority of users).

There are a few additional considerations:

  1. Even though Java 11 was released in September 2018 (over 5 years ago) and usage is on a downward trend, it is still commonly used.
  2. A deprecation period is required before we remove support for a given Java version and the removal should happen in a major Kafka release.
  3. Important projects we depend on may remove support for Java 11 before we do, which may result in challenges when it comes to updates required due to CVEs. One example is Jetty 12, which requires Java 17.
  4. It is often harder to upgrade the Java version in applications that embed libraries (like kafka clients or kafka streams) than it is to do so for services (like kafka brokers). Connect is somewhere in between as some connectors may have library dependencies that do not work with newer Java versions.

Balancing the various aspects, we propose only deprecating Java 11 support for the kafka broker, its dependent modules and the tools module in Apache Kafka 3.7 with support being dropped in Apache Kafka 4.0. The remaining modules (clients, streams, connect, etc.) will continue to support Java 11. This approach is similar to the one introduced by Elasticsearch 8.0 where the server requires Java 17, but the client requires an older Java version.

Public Interfaces

None.

Proposed Changes

  • Deprecate Java 11 build for the kafka broker and its dependent modules in Kafka 3.7: update documentation including the downloads page.
  • In Kafka 4.0:

    • Set sourceCompatibility/targetCompatibility/release to 17 for the relevant modules in build.gradle.

    • Remove all Java 11 specific code from the relevant modules.
    • Update Readme and documentation (including the downloads page) to specify Java 17 as a requirement for the kafka broker and relevant modules. All other modules (including clients, streams and connect) would continue to require Java 11 (as specified by KIP-750).
  • The affected modules are: kafka-server-common, kafka-server, kafka_2.13, kafka-storage, kafka-metadata, kafka-group-coordinator, kafka-raft, kafka-shell, kafka-tools.

Compatibility, Deprecation, and Migration Plan

Users who have not and cannot upgrade the Java version used by the kafka brokers to Java 17 can continue to use Kafka 3.x until they can upgrade the given cluster. This can be done incrementally since the protocol is compatible independently of the Java version.

Rejected Alternatives

  1. Continue supporting Java 11 for the server modules.


  • No labels