Status

Current state: "Under Discussion"

Discussion thread:

JIRA: None (don't think this needs a Jira ticket)

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Kafka 3.7.x was meant to be the last minor release within the 3.x series followed then by a 4.0 release where Apache Zookeeper will be removed from the code base.

During the "Road to Kafka 4.0" discussion in the mailing list (https://lists.apache.org/thread/mxltb2k5y17q89vg0k9492dfc8z66jw1), it became apparent that there are some features that are not yet present in KRaft that the community is relying on to successfully operate and run Kafka. Thus, making it either hard or not feasible for a subset of the community to move to KRaft (or at least fully test KRaft) before Zookeeper's complete removal in 4.0.0. Some community members might only be able to try KRaft when Zookeeper is gone, leaving only a downgrade as a way out path if something doesn't work as expected (instead of config switching)

This KIP proposes the creation of a 3.8.0 release and the scope of this KIP is to define which are the respective KRaft KIPs that should be part of it so the community can safely embrace KRaft. This way community members can safely test KRaft without the need to perform downgrades or manual workarounds.

Public Interfaces

No direct interfaces will be changed by this KIP

Proposed Changes

We propose the following:

• Have a release 3.8.x after 3.7.0
  • The aim of this release would be KRaft strategic feature parity with Zookeeper
• Once 3.8.0 is released, immediately start with the 4.0.x release
• 3.8.x would be the last minor release within the 3.x series.

Scope

The following KIPs were identified by the community as blockers items that prevent them from migrating Kafka to KRaft.

List of KIPs that are deemed "must have"'s in the scope of KRaft for 3.8.0:

Timeline

Release 3.7.0: TBD, probably early January 2024

Release 3.8.0: 3 to 4 months after 3.7 (or earlier if all required KIPs are already merged)

Release 4.0.0: 3 to 4 months after 3.8

Compatibility, Deprecation, and Migration Plan

Not relevant for this KIP

Test Plan

This release will follow the same testing plan as usual, but we encourage community members who haven't migrated yet to KRaft, to test the Release Candidates (or even specific trunk builds) in order to detect as many potential problems with KRaft when used under these, potentially, new scenarios.

Rejected Alternatives

Some rejected alternatives are:

Parallel Development

• Have a multiple release heads: keep working on 3.x branches while also working on 4.x branches.
  • This will create a huge overhead in terms of management and conflict resolution

Mark 4.0 as Experimental Release

• Mark release 4.0 as experimental and work towards a 4.x which would be stable
  • We never did anything like this
  • This would create confusion
  • Risk of losing credibility among the community