Versions Compared

Old Version 104

changes.mady.by.user Justine Olshan

Saved on

compared with

New Version 105

changes.mady.by.user Justine Olshan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Overall, topic IDs provide a safer way for brokers to replicate topics without any chance of incorrectly interacting with stale topics with the same name. By preventing such scenarios, we can simplify a number of other interactions such as topic deletes which are currently more complicated and problematic than necessary.

Public Interfaces

...


Uuid

A new UUID Uuid class will be exposed under /org/apache/kafka/common

/*
 * This class defines an immutable universally unique identifier (UUIDUuid). It represents a 128-bit value.
 * More specifically, the random UUIDsUuids in this class are variant 2 (Leach-Salz) version 4 UUIDsUuids.
 * This definition is very similar to java.util.UUID. One notable difference is that the The toString() method prints

 * using the base64 string encoding. Likewise, the fromString method expects a base64 string encoding.
 */
public class UUIDUuid {

  /**
   * A UUIDUuid where all bits are zero. It represents a null or empty UUIDUuid.
   */
  public static final UUIDUuid ZERO_UUID
  
  /**
   * Constructs a 128-bit type 4 UUIDUuid where the first long represents the the most significant 64 bits
   * and the second long represents the least significant 64 bits.
   */
  public UUIDUuid(long mostSigBits, long leastSigBits)
  
  /**
   * Static factory to retrieve a type 4 (pseudo randomly generated) UUIDUuid.
   */
  public static UUIDUuid randomUUIDrandomUuid()


  /**
   * Returns the most significant bits of the UUIDUuid's 128 bit value.
   */
  public long getMostSignificantBits()

  /**
   * Returns the least significant bits of the UUIDUuid's 128 bit value.
   */
  public long getLeastSignificantBits()

  /**
   * Returns true iff the obj is another UUIDUuid with the same value.
   */
  public boolean equals(Object obj)
  
  /**
   * Returns a hash code for this UUIDUuid
   */
  public int hashCode()

  /**
   * Returns a base64 string encoding of the UUIDUuid.
   */
  public String toString()

  /**
   * Creates a UUIDUuid based on a base64 string encoding used in the toString() method.
   */
  public static UUIDUuid fromString(String str) 
}

Additionally, it may be dangerous to use older versions of Kafka tools with new broker versions when using their --zookeeper flags. Use of older tools in this way is not supported today.

...

The protocol for CreateTopicsResponse will also need a slight modification.

CreateTopicsResponse v7

CreateTopics Response (Version: 7) => throttle_time_ms [topics]
  throttle_time_ms => INT32
  topics => name topic_id* error_code error_message topic_config_error_code num_partitions replication_factor [configs]
         name => STRING
    topic_id* => UUID
    error_code => INT16
    error_message => STRING
    topic_config_error_code => INT16
    num_partitions => INT32
    replication_factor => INT16 
    configs => name value read_only config_source is_sensitive
      name => STRING
      value => STRING
      read_only => BOOL
      config_source => INT8
      is_sensitive => BOOL

Describe Topics

There are two use cases we want to support. 1) Obtaining topic IDs when asking to describe topics and 2) supplying topic IDs to get a description of the topics

...

/**
* A unique identifier for the topic.
*/
public UUID topicId()

MetadataResponse v10

Metadata Response (Version: 10) => throttle_time_ms [brokers] cluster_id controller_id [topics] cluster_authorized_operations
    throttle_time_ms => INT32
    brokers => node_id host port rack
        node_id => INT32
        host => STRING
        port => INT32
        rack => STRING
    cluster_id => STRING
    controller_id => INT32
    topics => error_code name topic_id* is_internal [partitions] topic_authorized_operations
        error_code => INT16
        name => STRING 
        topic_id* => UUID
        is_internal => BOOL
        partitions => error_code partition_index leader_id leader_epoch [replica_nodes] [isr_nodes] [offline_replicas]
            error_code => INT16
            partition_index => INT32
            leader_id => INT32
            leader_epoch => INT32
            replica_nodes => INT32
            isr_nodes => INT32
            offline_replicas => INT32
        topic_authorized_operations => INT32
    cluster_authorized_operations => INT32

When topic IDs are supported, the response will contain both the topic name and the topic ID.

...

ID will be checked first, but if the value is the default zero UUID, topic name will be used instead.  If an ID is specified and the ID does not exist, the request will fail regardless of allow_auto_topic_creation. 
If the topic ID is not found, the request will return an UNKNOWN_TOPIC_ID error for the topic indicating the topic ID did not exist. The check for the topic ID will occur before checking authorization on the topic. Thus, topic IDs are not considered sensitive information.

MetadataRequest v10

Metadata Request (Version: 10) => [topics] allow_auto_topic_creation include_cluster_authorized_operations include_topic_authorized_operations
    topics => name topic_id*
      name => STRING (nullable)*
      topic_id* => UUID
    allow_auto_topic_creation => BOOL
    include_cluster_authorized_operations => BOOL
    include_topic_authorized_operations => BOOL

DeleteTopics

It will be useful for the AdminClient to be able to specify a list of topic Ids to delete to ensure the correct topics are being deleted. New methods will need to be added to the Admin interface and KafkaAdminClient

...

DeleteTopics Request and Response should be modified.

DeleteTopicsRequest v6

DeleteTopics Request (Version: 6) => [topics] timeout_ms
    topics => name topic_id*
      name => STRING (nullable)*
      topic_id* => UUID
    timeout_ms => INT32

Like the MetadataRequst, ID will be checked first, but if the value is the default zero UUID, topic name will be used instead. If an ID is specified and the ID does not exist, the request will return UNKNOWN_TOPIC_ID error for the topic indicating the topic ID did not exist. The check for the topic ID will occur before checking authorization on the topic. Thus, topic IDs are not considered sensitive information.

DeleteTopicsResponse v6

DeleteTopics Response (Version: 6) => throttle_time_ms [responses]
    throttle_time_ms => INT32
    responses => name topic_id* error_code error_message
      name => STRING
      topic_id* => UUID
      error_code => INT16
      error_message => STRING

Although only topic ID or only topic name are included in the request, if topic Ids are supported, the response will contain both the name and the ID.

...