Status

Current state: Under Discussion Accepted

Discussion thread:   https://lists.apache.org/thread/90zkqvmmw3y8j6tkgbg3md78m7hs4yn6

JIRA:  n/a

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

1. Mark KRaft as production-ready for new clusters in the upcoming Kafka 3.3 release. This is also known as "GA" or "general access."
2. Deprecate ZooKeeper mode in the upcoming Kafka 3.4 5 release
3. Plan to remove ZooKeeper mode entirely in Kafka 4.0.

...

KIP-746: Revise KRaft Metadata Records fixed several issues with the original record definitions established in Kafka 2.8. KIP-778: KRaft upgrades spelled out a clear path for handling changes to KRaft metadata in a backwards-compatible way. Overall, we feel that the next release of Kafka, the 3.3 release, will be a good time to  remove the "only for testing" warning from KRaft and mark it as ready for production.

...

Once ZK mode is removed in Kafka 4.0, we will be able to avoid maintaining parallel codebases for important infrastructure such as the Kafka controller.

Release Timeline

Note: this timeline is very rough and subject to change. However, it's intended to help people associated dates with release numbers. We are assuming that Kafka will continue to use time-based releases that happen every 4 months.

Kafka 3.3 (released)

• August 2022
• /08: KRaft mode declared production-ready in

Kafka 3.4 (released)

• January 2023
• Migration 32022/12: Upgrade from ZK mode supported in as Early Access (EA)

Kafka 3.5

• May 2023
• SCRAM support added for KRaft
• ZK mode deprecated
• Migration from ZK mode supported as Preview.

Kafka 3.6

• September 2023
• Migration from ZK mode supported as GA.
• Delegation token support in KRaft (planned)
• JBOD support in KRaft (planned)

Kafka 3.7

• January 2024
• Final release with ZK mode

Kafka 4.0

• April 2024
• Only
4. ZooKeeper mode deprecated.
• 2023/04: Kafka 3.5 released with both KRaft and ZK support
• 2023/08: Kafka 4.0 released with only KRaft mode supported.

Missing Features

As of the date of writing, several the following features have not yet been implemented in KRaft mode.

...

• Supporting JBOD configurations with multiple storage directories

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key KAFKA-14127

• Modifying  Modifying certain dynamic configurations on the standalone KRaft controller

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key KAFKA-14349

• Delegation tokens
  Upgrade from ZooKeeper mode

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key KAFKA-15219

As outlined aboveWhile the lack of these features in the current KRaft codebase is frustrating, we expect to close these gaps over the course of the next few months.The last omission (upgrade from ZK) is the most significant. We expect to release a KIP soon describing in detail how the upgrade from ZK mode will work. This work is currently targetted at the next release (3.3).

Compatibility, Deprecation, and Migration Plan

As discussed in KIP-500 and the follow-on KIPs, KRaft clusters expose the same public APIs as ZK-based Kafka clusters. Since the modern Kafka producer and consumer do not directly access ZooKeeper, no changes are needed to existing Kafka clients. While some administrative tools still have options which enable them to perform access to ZooKeeper, this mode of access has been deprecated since mid-2020, as described in KIP-555. Therefore, we think the transition for clients should be smooth.

Bridge Releases

Starting with 3.5, all the 3.x releases will be "bridge releases" as described in KIP-500. What this means is that they can act as a bridge from the ZK world to the KRaft world.

Users who have a ZooKeeper-based Kafka and want to upgrade to a post-ZK version (4.0 and beyond) will have to first upgrade to one of these bridge releases. Then, once they are in KRaft mode, they will be able to upgrade to any 4.x release.

Potential Objections and Responses

One possible objection is that we should not declare KRaft to be production ready until it has reached full feature parity with ZooKeeper mode. My response to this is that we expect to close the feature gaps relatively quickly – especially upgrade, which should be fully taken care of in 3.4. There is still a lot of value in marking KRaft as production ready now. In addition to enabling more usage, it will ensure that we do not end up with more feature gaps to fill over the next few months.

...