...

• GET /clusters - list of cluster aliases
• GET /status - MirrorMaker status information
• GET /metrics - MirrorMaker metrics snapshot
• GET /cluster/cluster/status - status information for specific cluster
• GET /cluster/cluster/metrics - metrics snapshot for specific cluster
• GET /from:source/to:target/status - status information for specific replication flow
• GET /from:source/to:target/metrics - metrics snapshot for specific replication flow
• GET /from:source/to:target/connectors - list of connectors
• GET /from:source/to:target/connectors/connector/config - connector configuration
• PUT /from:source/to:target/connectors/connector/config - update connector configuration
• GET /from:source/to:target/connectors/connector/status - status information for a specific connector
• PUT /from:source/to:target/connectors/connector/restart - restart a specific connector
• PUT /from:source/to:target/connectors/connector/pause- pause a specific connector
• PUT /from:source/to:target/connectors/connector/resume - resume a specific connector
• DELETE /from:source/to:target/connectors/connector- delete a specific connector

Walkthrough: Running MirrorMaker 2.0

There are four ways to run MM2:

• As a dedicated MirrorMaker cluster.
• As a Connector in a distributed Connect cluster.
• As a standalone Connect worker.
• In legacy mode using existing MirrorMaker scripts.

Running a dedicated MirrorMaker cluster

In this mode, MirrorMaker does not require an existing Connect cluster. Instead, a high-level driver and API abstract over a collection of Connect workers.

First, specify Kafka cluster information in a configuration file:

# mm2.properties
clusters = us-west, us-east
cluster.us-west.brokers.list = host1:9092
cluster.us-east.brokers.list = host2:9092

Optionally, you can override default MirrorMaker properties:

topics = .*
groups = .*
emit.checkpoints.interval.seconds = 10

You can also override default properties for specific clusters or connectors:

cluster.us-west.offset.storage.topic = mm2-offsets
connector.us-west->us-east.emit.heartbeats = false

Second, launch one or more MirrorMaker cluster nodes:

$ ./bin/connect-mirror-maker.sh mm2.properties

Finally, remote-control the MirrorMaker cluster using the REST API or CLI. The REST API provides a sub-set of the Connect REST API, including the ability to start, stop, and reconfigure connectors. For example:

PUT /from:us-west/to:us-east/connectors/MirrorSourceConnector/config

{
    "topics": ".*"
}

Running a standalone MirrorMaker connector

In this mode, a single Connect worker runs MirrorSourceConnector. This does not support multinode clusters, but is useful for small workloads or for testing.

First, create a "worker" configuration file:

# worker.properties
bootstrap.servers = host2:9092

An example is provided in ./config/connect-standalone.properties.

Second, create a "connector" configuration file:

# connector.properties
name = local-mirror-source
connector.class = org.apache.kafka.connect.mirror.MirrorSourceConnector
tasks.max = 1
topics=.*

source.cluster.alias = upstream
source.cluster.bootstrap.servers = host1:9092
target.cluster.bootstrap.servers = host2:9092

# use ByteArrayConverter to ensure that in and out records are exactly the same
key.converter = org.apache.kafka.connect.converters.ByteArrayConverter
value.converter = org.apache.kafka.connect.converters.ByteArrayConverter

Finally, launch a single Connect worker:

$ ./bin/connecti-standalone.sh worker.properties connector.properties

Running MirrorMaker in a Connect cluster

If you already have a Connect cluster, you can configure it to run MirrorMaker connectors.

There are four such connectors:

• MirrorSourceConnector
• MirrorSinkConnector
• MirrorCheckpointConnector
• MirrorHeartbeatConnector

Configure these using the Connect REST API:

PUT /connectors/us-west-source/config HTTP/1.1

{
    "name": "us-west-source",
    "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
    "source.cluster.alias": "us-west",
    "target.cluster.alias": "us-east",
    "source.cluster.brokers.list": "us-west-host1:9091",
    "topics": ".*"
}

Running MirrorMaker in legacy mode

After legacy MirrorMaker is deprecated, the existing ./bin/kafka-mirror-maker.sh scripts will be updated to run MM2 in legacy mode:

$ ./bin/kafka-mirror-maker.sh --consumer consumer.properties --producer producer.properties

Compatibility, Deprecation, and Migration Plan

A new version of ./bin/kafka-mirror-maker.sh will be implemented to run MM2 in "legacy mode", i.e. with new features disabled and supporting existing options and configuration properties. Existing deployments will be able to upgrade to MM2 seamlessly.

...