...

As the controller must wait for all replicas to delete their local replicas, deletes can also become blocked, preventing topics from being created creation of a topic with the same name is blocked until the deletion is complete on all replicas . have successfully deleted a topic's data. This can mean that downtime for a single broker can effectively cause a complete outage for everyone producing/consuming to that topic name, as the topic cannot be recreated without manual intervention.

Topic IDs aim to address this issue by associating a truly unique ID with each topic, ensuring a newly created topic with a previously used name cannot be confused with a previous topic with that name.

Topic IDs solve a few other several additional problems:

1. Renaming topics becomes feasible (although there may still be some complexity with the need to support the old name for a while as part of migration, etc.)  Renaming topics may seem minor, but it will be difficult to have hierarchical topics without having some kind of renaming support.
2. We can eventually get rid of the "deleting" state for topics. If a broker is down but there is some topic data there that is no longer relevant, it won't cause problems later on. It can be deleted when the broker rejoins the cluster and realizes that the relevant topic ID is not present any more. We gain some additional safety where stale/deleted replicas may currently interact with live ones.
3. Sending 16 byte UUIDs instead of Strings over Kafka RPCs can be smaller. A string is 2 bytes plus the data, whereas the UUID is fixed 16 bytes.  For any topic name with more than than 14 single byte characters (16 bytes serialized), UUIDs will be smaller. They will also be faster to compare and more friendly to the garbage collector.
4. They will provide a true measure of topic uniqueness across clusters. This may be important in multi-cluster Kafka deployments where additional safety and debuggability is desired.

Overall, topic IDs provide a safer way for brokers to interact with respect to a topic replicate topics without any chance of incorrectly interacting with a stale topic topics with the same name. By preventing these situations, we can simplify a number of other interactions , e.g. such as topic deletes, that which are currently more complicated and problematic than necessary due to the need to avoid these issues.

Public Interfaces

No changes to public interfaces will be made in this release. However, it may be dangerous to use older versions of Kafka tools with new broker versions when using their --zookeeper flags.

...