Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

MirrorMaker should not clear topic configurations that are excluded from replication.

In addition, source and target topic configurations could have different default values. Currently, Mirrormaker overwrites configurations on target topic even if the configurations on the source topic have not been changed but happened to have different default values. However, user may only want to sync configurations that they have changed but leave the ones with default values. 

Public Interfaces

This KIP proposes to migrate the IncrementalAlterConfigs API for syncing topic configurations in MirrorMaker. The IncrementalAlterConfigs API has been around for several years since it was introduced in Kafka 2.3.0 and addresses the shortcoming of the AlterConfigs API. In order not to break the compatibility, we will introduce a new setting to MirrorMaker to give the option to enable and disable IncrementalAlterConfigs API for syncing topic configurations. This new setting is expected to serve as a temporary measure until the next major release when the API is always used.

- If we're using the incremental API, for any default topic config
properties on the source cluster, manually wipe them from the target
cluster, which also matches existing behavior

If users delete a topic configuration in the source cluster, currently it would also be deleted from the target cluster due to the replace all behaviour of AlterConfigs API. However with IncrementalAlterConfigs, the topic configuration in the target cluster would potentially be left unchanged. Therefore any default topic configuration on the source cluster would need to be deleted from the target cluster. The KIP It also proposes adding the ability for users to choose using between target cluster's default or source cluster's default when using IncrementalAlterConfigs for target syncing topic configurations. For instance, users would be able to configure ConfigPropertyFilter to check if the configuration on the source topic is the default and decide choose whether or not to replicate it to the target topic.

...