THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Status
Current state: Under Discussion
Discussion thread:
JIRA: none yet
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The KIP-500 effort to remove Kafka's dependency on Apache ZooKeeper requires some changes to the way Kafka system administration tools work. All of the functionality that was previously available through direct ZooKeeper access must now be accessible through other means.
Several previous KIPs have prepared the way for this change. KIP-455, KIP-497, KIP-554, and KIP-543 added broker-side APIs for operations that had previously only been supported via direct Zookeeper access. KIP-555 deprecated the --zookeeper flag in administrative tools where it was still supported.
Now, we must take the final steps towards removing the ZooKeeper flags from the administrative tools. This step will pave the way for the bridge release, a release which can be upgraded to a ZooKeeper-free state.
Public Interfaces
The --zookeeper flag will be removed from all the places where it was deprecated in KIP-555, with only one exception which will be described in a little bit.
Here is a table of all the changes:
| Command Name | Changes Needed |
|---|---|
| kafka-configs.sh | Support --zookeeper only for pre-startup connection initialization |
| kafka-leader-election.sh | Remove --zookeeper flag |
| kafka-preferred-leader-election.sh | Remove --zookeeper flag |
| kafka-reassign-partition.sh | Remove --zookeeper flag |
| kafka-topics.sh | Remove --zookeeper flag |
kafka-configs.sh Changes
Unlike all the other tools, kafka-configs.sh will keep the --zookeeper flag. It will be allowed only in two scenarios:
- In order to make dynamic configuration updates before and brokers are registered in ZooKeeper.
- In order to make SCRAM configuration updates before any brokers are registered in ZooKeeper.
The command will not succeed if any brokers are registered. It is intended to be used before any brokers are started, to set up SSL and SCRAM settings in ZooKeeper.
The rationale behind keeping the --zookeeper flag for this use-case is that we don't yet have an equivalent replacement. Although we are developing a new broker-side SCRAM API, this will only be usable once the brokers have started.
Compatibility, Deprecation, and Migration Plan
This is an incompatible change. Therefore, it must be done in the next major release of Kafka, the 3.0 release.
The flags that are being removed have already been deprecated for at least one release.
The migration plan is to use --bootstrap-server for these commands instead.
Rejected Alternatives
Do not allow bootstrapping network connection settings
We could have disabled the --zookeeper flag for all uses, including bootstrapping network connection settings. However, this would have caused problems for people who are currently using this functionality.