...

Discussion thread: here

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

This (NewTopic) is what will be passed in AdminClient#createTopics. The default values will remain the same (sentinel value of -1).

The `CreateTopicsRequest` will bump its protocol version to 4 so that any requests sent to old brokers with the default values and no replica assignments will fail with a UnsupportedVersionException.

To exploit this new feature in KafkaStreams, we update the default value of Streams configuration parameter `replication.factor` from `1` to `-1`. Cf KIP-733: change Kafka Streams default replication factor config

Proposed Changes

The change is straightforward, AdminManager on the broker will modify its verification logic to the following:

...

• No request that was previously valid will become invalid, and no request that was previously valid will change behavior.
• Some previously invalid requests (e.g. valid partitions and no replication factor or manual assignments) will now succeed.
• Sending a request with default partitions/replicas to an old broker will throw `UnsupportedVersionException`

Rejected Alternatives

• Add a new configuration to differentiate default.replication.factor for auto-topic-create and for normal behavior (i.e. feature flagging this change). I believe this will just confuse users, and the current configuration name is descriptive enough.
• Change the broker protocol to accept a new type of request that does not have any value (instead of a sentinel value) for the replication factor/partitions.
  
  • The original proposal in this KIP will be easier to perform upgrades - since all Kafka brokers will be able to deserialize the incoming request, the only difference is whether or not the request succeeds (i.e. creates the topic) or fails (i.e. sends an error message). 

...