...

This KIP proposes to migrate the IncrementalAlterConfigs API for syncing topic configurations in MirrorMaker. The IncrementalAlterConfigs API has been around for several years since it was introduced in Kafka 2.3.0 and addresses the shortcoming of the AlterConfigs API. In order not to break the compatibility, we will introduce a new setting to MirrorMaker to give the option to enable and disable IncrementalAlterConfigs API API for syncing topic configurations. This new setting is expected to serve as a temporary measure until the next major release when the API is always used. 

It also proposes adding the ability to use the target cluster's default for target topic configurations, rather than using the source cluster's default. For instance, users would be able to configure ConfigPropertyFilter to check if the configuration on the source topic is the default and decide whether or not to replicate it to the target topic. 

Proposed Changes

1) Add a new method updateTopicConfigsUsingIncrementalAlterConfigs() to MirrorSourceConnector.java which takes a list topic configurations as a parameter and calls incrementalAlterConfigs() to the target cluster. 2) Extend ConfigPropertyFilter  class to have a new method called shouldReplicateSourceDefault which would always return true by default. User would be able to configure this method to return false for a configuration that is DEFAULT_CONFIG.

3) Update MirrorSourceConnector.java class to sync a configuration only if shouldReplicateSourceDefault returns true. 

2) Add a new configuration setting to MirrorMaker:

...

• When explicitly set to never, MirrorMaker will use the deprecated AlterConfigs API for syncing topic configurations.
• When explicitly set to required, MirrorMaker will use the IncrementalAlterConfigs API for syncing topic configurations. If it receives an error from an incompatible broker, the MirrorMaker will report this to the user and fail the connector.

3) Update syncTopicConfigs()  in MirrorSourceConnector.java to call the updateTopicConfigsUsingIncrementalAlterConfigs() instead of updateTopicConfigs() if enable.incremental.alter.configs is set to true.4) From Kafka 4.0, syncTopicConfigs() will only call  updateTopicConfigsUsingIncrementalAlterConfigs() by default. The existing updateTopicConfigs() method will be removed.

...

By default, the new setting will be set to false so it does not change the current behaviour"required"  which means we would have the fallback logic if the broker is incompatible with IncrementalAlterConfig API. This setting will be marked as deprecated immediately. 

...

• use.incremental.alter.configs will be removed removed.
• MirrorMaker will use the IncrementalAlterConfigs API to sync topic configurations by default. 

Users running the latest MirrorMaker connector against a target cluster with brokers older than Kafka 2.3.0, will have to upgrade.

...