Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

After considering a few approaches, listed in Rejected Alternatives section, below is what we think is the least obtrusive approach to support namespaces in Kafka. We suggest to represent namespaces at storage layer, i.e., storage layout of Zookeeper entities and logs on disk. Internal and public APIs can pass around namespaces, as part of, prepended to, topic names.

Changes in storage layouts

Following are the changes in layout structures on ZK and disks. 

TypeExisting layoutProposed layout
ZK /brokers/topics/<topic>/brokers/topics/<namespace>/<topic>
ZK/configs/<entity_name>/<entity>/configs/<namespace>/<entity_name>/<entity>
ZK/admin/delete_topics/<topic>/admin/delete_topics/<namespace>/<topic>
ZK/kafka_acls/<entity_name>/<entity> /kafka_acls/<namespace>/<entity_name>/<entity>
Disk/<log_dir>/<topic>_<partition>/<log_dir>/namespace/<topic>_<partition>

...