...

Additionally, if we supported multiple metadata storage options, we would have to use "least common denominator" APIs.  In other words, we could not use any API unless all possible metadata storage options supported it.  In practice, this would make it difficult to optimize the system.

Follow-on Work

This KIP expresses a vision of how we would like to evolve Kafka in the future.  We will create follow-on KIPs to hash out the concrete details of each change.

These KIPs will include:

• A KIP to implement Raft replication in Kafka.  This will specify the new replication protocol and the details of each new RPC.
• A KIP for allowing kafka-configs.sh to change topic configurations without using ZooKeeper.  It can already change broker configurations without Zookeeper, but it needs to be able to change all configurations without ZooKeeper.
• A KIP for adding APIs to replace direct ZK access by the brokers.
• A KIP describing the controller changes.  This will also specify how metadata is stored, and so on.

There may be other KIPs in addition to these.  However, hopefully this gives some context about the follow-on work.

References

The Raft consensus algorithm

...