Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

 

Status

Current state:  ["Under Discussion"] Accepted

Discussion thread: hereTBD

JIRA:

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQuerykey = KAFKA-6170 OR key = KAFKA-6150 OR key = KAFKA-6126
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

...

  • Purge repartition data on commit: this is summarized in 
    Jira
    serverASF JIRA
    serverId5aa69414-a9e9-3523-82ec-879b028fb15b
    keyKAFKA-6150
    . The AdminClient's deleteRecords API (adding in KIP-204) will be used upon committing intervals.
  • Create internal topic within InternalTopicManager: we will use the create topic API of AdminClient.
  • We'll also remove the endless-loop checking after the creation within StreamPartitionAssignor (
    Jira
    serverASF JIRA
    serverId5aa69414-a9e9-3523-82ec-879b028fb15b
    keyKAFKA-6126
    ); instead after the rebalance we will let the restoration retry within the main loop if the metadata is not known yet.
  • We'll add an ADMIN_PREFIX (String contant valued "admin.") in the StreamsConfig for admin client usage which will allow certain config parameters for the admin client only, also add the adminClientPrefix function; this is similar to PRODUCER_PREFIX and CONSUMER_PREFIX.
  • Compatibility check: we will remove this functionality and defer to the runtime to detect the returned error code and throw exception.

 

In addition, the following new public APIs will be added as part of this KIP (details of these changes can be found in https://github.com/apache/kafka/pull/4315):

  • Overloaded KafkaStreams constructor to allow overriding the `Time` object.
  • Overloaded AbstractConfig#originalsWithPrefix() to allow specifying whether or not to strip the prefix in the key of the returned map.

 

 

Regarding the changes to replace StreamsKafkaClient with the KafkaAdminClient, the only differences are the configs. Here is a list of configs defined in StreamsConfig that will be used in StreamsKafkaClient:

...