Page History

...

After considering a few approaches, listed in Rejected Alternatives section, below is what we think is the least obtrusive approach to support namespaces in Kafka. We suggest to represent namespaces at storage layer, i.e., storage layout of Zookeeper entities and logs on disk. Internal and public APIs can pass around namespaces, as part of, prepended to, topic names. However, we need someway to separate namespace and topic while interacting with storage layers. This can be done by using a delimiter character that is not allowed in Kafka topics. Currently, Kafka allows a topic name to contain characters only from in [a-zA-Z0-9\\._\\-]. That gives us a few options to decide on the delimiting char. We suggest to have ":" as the delimiting char, but it can be any of the following.

Possible Delimiters

1. <namespace>:<topic>
2. <namespace>%<topic>
3. <namespace>@<topic>
4. <namespace>|<topic>
5. <namespace>#<topic>
6. <namespace>%<topic>
7. <namespace>@<topic>
8. <namespace>*<topic>
9. <namespace>~<topic>
10. <namespace>$<topic>
11. <namespace>^<topic>
12. <namespace>&<topic>
13. <namespace>><topic>

...

1. Just prepend namespace to topic names, inheritance will be tricky and not so intuitive. Will not enable encrypting namespaces with different keys or namespace level management.
2. Manage namespaces separately, this will still have the issue of topic name collisions even if they belong to separate namespaces.
3. Modify request/ response formats to take namespace specifically. Solves the issue of demarcation string delimiting string required in proposed approach. However, will be backwards incompatible changerequire protocol and lots of API changes.
4. Add namespace to session object. Will avoid each request and response to have namespace with topic name, however this probably is violating separation of concerns.
5. To have delimiter char configurable. Will add yet another config without a clear gain.