Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Currently Mirrormaker uses deprecated alterConfigs  API when syncing topic configurations for broker compatibility to 0.11.0. However, the deprecated alterConfigs() API   API replaces any existing configuration with the new configuration. Due to this, additional configurations set on a remote topic get cleared up when MirrorMaker syncs topic configurations even if the configurations are filtered.  For example this prevents running Cruise Control on the target cluster as it may set follower.replication.throttled.replicas and leader.replication.throttled.replicas.

...

This KIP proposes to migrate to incrementalAlterConfigs  API for syncing topic configurations in MirrorMaker. The incrementalAlterConfigs  API has been around for several years since it was introduced in Kafka 2.3.0 and addresses the shortcoming of the deprecated alterConfigs  API. In order to not break the compatibility, we will introduce a new configuration to MirrorMaker connector to give the option to enable incrementalAlterConfigs API for syncing topic configurations. This new configuration is expected to serve as a temporary measure until the next major release when incrementalAlterConfigs the API is always used. 

Proposed Changes

1) Add a new method updateTopicConfigsUsingIncrementalAlterConfigs() to MirrorSourceConnector.java which takes a list topic configurations as a parameter and calls incrementalAlterConfigs() to the target cluster. 

2) Add a ) A new configuration setting to MirrorMaker:

Name: enable.incremental.alter.configs

Description: Deprecated. Whether to automatically use incrementalAlterConfigs API for syncing topic configurations. This configuration will be removed in Kafka 4.0 and the incrementalAlterConfigs API will be used by default. Users should make sure that the target cluster is running at least Kafka 2.

...

3.0 or later. 

Type: Boolean

Default: false

Update Mode: 

When set to false, MirrorMaker will keep using deprecated alterConfigs API for syncing topic configurations.

When explicitly set to true, MirrorMaker will use incrementalAlterConfigs API for syncing topic configurations. If it receives an error from an incompatible broker, the MirrorMaker will report this to the user and fail the connector. 

...

3)

...

Update `syncTopicConfigs`

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?
  • If we are changing behavior how will we phase out the older behavior?
  • If we need special migration tools, describe them here.
  • When will we remove the existing behavior?

...