Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Some of the implemented request types are also relevant to clients. Adding full support for topic IDs in the clients would add an additional measure of safety when producing and consuming data. Fully supporting Topic IDs in clients is out of scope for this KIP due to the numerous public APIs that will need adjustments.

__consumer_offsets topic

Ideally, consumer offsets stored in the __consumer_offsets topic would be associated with the topic ID for which they were read. However, given the way the __consumer_offsets is compacted, this may be difficult to achieve in a forwards compatible way. This change will be left until topic IDs are implemented in the clients. Another future improvement opportunity is to use topicId in GroupMetadataManager.offsetCommitKey in the offset_commit topic. This may save some space.

...

One idea was to support authorizing a principal for a topic ID rather than a topic name. For now, this would be a breaking change, and it would be hard to support prefixed ACLs with topic IDs.

TopicIdPartition

Replacing a topic name with a topic ID introduced an issue with how the servers and clients should handle creating TopicPartition objects used when handling the requests. One id was to create a public TopicIdPartition object that could contain either a topic ID or a topic name along with the partition depending on the request version. However, TopicPartitions are used in numerous public APIs and all of those would have to be adapted to handle this new object. Due to the scope of these changes, the TopicIdPartition is out of scope and placed in future work. 

Another alternative is adding an id field to the already existing TopicPartition object. However, even in this case, it is not yet clear all the implications, as some uses of TopicPartition require the topic name to be present. 

Since this question requires a bit more thought and result in widespread changes, it is suggested that this change should be future work, perhaps its own KIP.