Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Kafka has a dependency on Zookeeper 3.6.3, which reached its end of life in December 2022. We would like to upgrade Zookeeper to version 3.8.1 2 which is the latest release of the 3.8.x versions.

Zookeeper 3.8.1 2 server supports clients no older than 3.5.x (i.e. Kafka version 2.4.0) and Zookeeper 3.8.12. clients support server versions no older than 3.5.x (i.e. Kafka version 2.4.0).

...

Similarly to https://github.com/apache/kafka/pull/12620/files we would like to upgrade to 3.8.12.

Compatibility, Deprecation, and Migration Plan

...

Users of Kafka clusters with Zookeeper clients older than 3.5.x won't be able to communicate with a 3.8.1 2 Zookeeper cluster. As mentioned in the accompanying JIRA ticket Kafka has been using Zookeeper clients 3.5.x since version 2.4 so versions above and including it should be safe for this upgrade. It is acceptable to break compatibility with Kafka versions prior to 2.4 as they are considered beyond their end of life and are not maintained. (source: Time Based Release Plan#WhatIsOurEOLPolicy).

...


Zookeeper cluster (< 3.5)Zookeeper cluster (>= 3.5)
Kafka cluster (< 2.4)
  1. Upgrade the Kafka cluster to version >= 2.4
  2. According to Upgrading to 3.5.0 you will need to carry out a rolling upgrade to Zookeeper 3.4.6 before upgrading the Zookeeper cluster to version 3.8.12
  1. Upgrade the Kafka cluster to version >= 2.4
  2. Upgrade the Zookeeper cluster to version 3.8.12
Kafka client (< 2.4)
  1. Upgrade the Kafka clients to >= 2.4.
  2. According to Upgrading to 3.5.0 you will need to carry out a rolling upgrade to Zookeeper 3.4.6 before upgrading the Zookeeper cluster to version 3.8.12
  1. Upgrade the Kafka clients to >= 2.4.
  2. Upgrade the Zookeeper cluster to version 3.8.12

Kafka cluster (>= 2.4)

  1. According to Upgrading to 3.5.0 you will need to carry out a rolling upgrade to Zookeeper 3.4.6 before upgrading the Zookeeper cluster to version 3.8.12
  1. Upgrade the Zookeeper cluster to version 3.8.12

Kafka client (>= 2.4)

  1. According to Upgrading to 3.5.0 you will need to carry out a rolling upgrade to Zookeeper 3.4.6 before upgrading the Zookeeper cluster to version 3.8.12
  1. Upgrade the Zookeeper cluster to version 3.8.12

This is a compatibility matrix between Kafka and Zookeeper versions.

The two coloured rectangles indicate versions of Kafka and Zookeeper which are compatible with one another. When a new version of Kafka having a dependency on Zookeeper 3.8.1 2 is released the bridge release will be any of the version where the two squares intersect.

...

We ran the following test on the latest trunk of Kafka with Zookeeper 3.6.3 and Zookeeper 3.8.12:

1) Start 1 Zookeeper node on an m5.4xlarge machine

...

5) Using 4 admin clients sequentially delete all topics


Zookeeper 3.8.1 2 request latency (PROPOSED)

...