Table of Contents |
---|
Status
Current state: Under DiscussionAccepted
Discussion thread: here
JIRA: KAFKA-4060
Released: <Kafka Version>0.10.2.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
This KIP removes the Zookeeper dependency from Kafka Streams. Currently, for Kafka Streams API accesses Zookeeper directly to create internal topics. We will use the client admin API introduced in KIP-4 to manage the internal topics via Kafka broker instead of directly accessing Zookeeper.
Public Interfaces
This KIP does not change any public interfacesZookeeper related config value, ZOOKEEPER_CONNECT_CONFIG in StreamsConfig, is deprecated.
Proposed Changes
We add a new KafkaStreams client which is used for internal topic management in Kafka Streams API. We will remove the dependency to Zookeeper from Kafka Streams API..
Compatibility, Deprecation, and Migration Plan
This feature is not backward compatible from API perspective and no changes need to be done to streaming apps. However, if kafka cluster is secured the cluster admin should grant required privileges to the streaming app as described below.
Zookeeper related config value, ZOOKEEPER_CONNECT_CONFIG in StreamsConfig, is deprecated.
...
- Create topics
- Delete Topics Fetch topic metadata(Without delete topic privilege streaming apps would still work if they don't have to delete topics.)
User information need to be set the same way it is set in the streaming app if the cluster is secured.
Test Plan
The same tests that are currently used will be used for testing the new internal topic creation.
...