You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Status

Current state: Under Discussion

Discussion thread: here

JIRA: here

Motivation

Old users of Kafka that still use the original MirrorMaker need a path to migrate to MirrorMaker2. This involves implementing a new ReplicationPolicy and due to the differences between MirrorMaker1 and MirrorMaker2 its proposed to add a new method called canTrackSource to the public interface of ReplicationPolicy so that its possible to abstract over replication policies that can either can or cannot track back to the original source.

Note that there is already a PR that implements both the original motivation (IdentityReplicationPolicy) along with the adjustments to the interface at https://github.com/apache/kafka/pull/10648. The PR is currently passing all tests, only this KIP is required to proceed.

Public Interfaces

  • ReplicationPolicy

Proposed Changes

The proposal is to add a boolean canTrackSource method to the ReplicationPolicy interface

Compatibility, Deprecation, and Migration Plan

  • There is currently an implementation of both the original motivation (implemention IdentityReplicationPolicy aka LegacyReplicationPolicy) at https://github.com/apache/kafka/pull/10648. There are not any backwards breaking behavior changes however custom ReplicationPolicies that implement the ReplicationPolicy interface would need to implement the canTrackSource method. For this reason its recommended to do this change in a new release of Kafka (ideally 3.0.0)

Rejected Alternatives

Although technically not a rejected alternative there is an alternative proposal for a Legacy/Identity ReplicationPolicy at https://github.com/apache/kafka/pull/10652 which attempts to solve the same problem without needing to make adjustments to the ReplicationPolicy. Do note however that this implementation is currently not working (tests are failing) and that arguably the implementation at https://github.com/apache/kafka/pull/10648 is a cleaner solution since doesn't need to back calculate the source from topic names. Hence there is an argument that irrespective of the IdentityReplicationPolicy, adding such a method can make custom implementations of a ReplicationPolicy cleaner

  • No labels