Page History

...

Discussion thread: here

JIRA: 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

For these reasons, MirrorMaker is insufficient for many use cases, including backup, disaster recovery, and failover fail-over scenarios. Several other Kafka replication tools have been created to address some of these limitations, but Apache Kafka has no adequate replication strategy to date. Moreover, the lack of a native solution makes it difficult to build generic tooling for multi-cluster environments.

I propose to replace MirrorMaker with a new multi-cluster, cross-datacenter data-center replication engine based on the Connect framework, MirrorMaker 2.0 (MM2). The new engine will be fundamentally different from the legacy MirrorMaker in many ways, but will provide a drop-in replacement for existing deployments.

...

• KafkaSourceConnector, KafkaSinkConnector, KafkaSourceTask, KafkaSinkTask classes.

• New configuration properties in addition to those supported by legacy MirrorMakerbylegacyMirrorMaker.

• Heartbeat and checkpoint topics and associated schemas; however, these will be considered internal.

• RemoteClusterUtils class for querying remote cluster reachability and lag, and for translating consumer offsets between clusters.

• Scripts for launching MM2 workers, including support for a legacy mode compatible with MirrorMaker.

...