...

Cluster mirrors can be configured using the following properties:

Topic Configuration

Broker Configuration

Mirror Configuration

Metrics

A core set of metrics will be provided with the initial implementation.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

...

Cluster Mirroring will be introduced through a phased rollout across multiple Kafka releases to ensure stability and gather community feedback. 

Phases

Early access

Cluster Mirroring is introduced as an early access feature, disabled by default to prevent accidental production usage. To enable it, all cluster nodes (controllers and brokers) must explicitly enable unstable API versions and unstable feature versions in all configuration files. After starting the cluster with a minimum metadata version, administrators can dynamically enable the mirror version feature to activate Cluster Mirroring. This stage is intended for testing and evaluation in non-production environments only, as the new APIs and metadata record formats may change in subsequent releases without backward compatibility guarantees.

Preview

In a future release, Cluster Mirroring will transition to preview status with frozen protocol and metadata schemas. The feature will still require explicit enablement via dynamic feature upgrades but will no longer require the unstable API and feature configuration. The feature remains disabled by default to ensure administrators consciously opt-in, but the upgrade path from early access clusters will be officially supported with compatibility guarantees. This stage is suitable for pre-production testing and pilot deployments where API stability is required but production-grade maturity is not yet needed.

General availability

When Cluster Mirroring reaches general availability, the feature will be enabled by default when clusters reach the corresponding production metadata version. All new APIs will become stable production APIs with all unstable markers removed from their definition. No special configuration flags or explicit feature enablement will be required beyond setting an appropriate metadata version, and the feature will be fully supported for mission-critical production workloads under Kafka's standard compatibility guarantees. Clusters using Cluster Mirroring in preview can upgrade seamlessly to GA releases without migration steps. Downgrade is also supported, but it would require manual cleanup of the internal topic.

Migration from MirrorMaker 2

Cluster Mirror is not compatible with MirrorMaker 2 (MM2). This is a critical consideration for users planning to migrate from MirrorMaker 2 to Cluster Mirroring.

MM2 and Cluster Mirroring use different internal topic structures and naming conventions for storing metadata and offsets. The two systems track and store consumer offsets differently, making it impossible to seamlessly transition between them.

Follow this process to switch from MirrorMaker 2 to Cluster Mirroring:

1. Stop MM2 replication
2. Delete mirrored topics on destination cluster, including MM2 internal topics
3. Start fresh with Cluster Mirroring

Compatibility Matrix

Note that some features require support from the source cluster.

Performance

MirrorFetcherThread uses the same fetch protocol optimizations as ReplicaFetcherThread:

• Fetch Sessions: Incremental fetch sessions (KIP-227) reduce fetch request size by sending only changed partition metadata. This optimization is critical for cross-cluster replication where WAN latency is higher than LAN latency.
• Pipelining: Multiple fetch requests can be in-flight simultaneously, improving throughput over high-latency connections. The number of in-flight requests is controlled by standard replica fetcher settings.
• Compression: Record batches are transferred in their original compressed format, minimizing network bandwidth. The destination cluster decompresses and recompresses based on its own compression settings only if the compression codec differs.
• Zero-Copy Transfer: Within the destination cluster, replication from read-only leaders to followers uses zero-copy transfers where supported by the operating system.

Cluster Mirroring introduces additional replication threads and network I/O on brokers configured as read-only leaders for mirror partitions. The performance impact on existing intra-cluster replication is minimized through resource isolation:

• Separate Thread Pools: Cross-cluster fetcher threads run in a dedicated thread pool, which is independent from the intra-cluster fetcher thread pool. This separation ensures that cross-cluster replication latency does not impact local replica synchronization.
• Network I/O Overhead: Read-only leaders perform additional network I/O to fetch from source clusters. This overhead is proportional to the number of mirror partitions and the replication throughput. Brokers with many mirror partitions may experience increased CPU usage for network processing and data serialization.
• Memory Footprint: Each mirror fetcher thread maintains its own fetch session state, partition state map, and response buffers. With default configuration, memory overhead is comparable to standard replica fetchers. The metadata manager maintains connection pools and metadata caches, adding minimal memory overhead.
• Bandwidth Consumption: Cross-cluster traffic between source and destination clusters consumes WAN bandwidth. For large-scale deployments, administrators should provision adequate inter-datacenter connectivity or configure throttling.
• State Management: Mirror partition state management is evenly distributed to available brokers to avoid any hot spot, especially during rolling update or restart events.

Test Plan

Unit Tests

Unit tests will cover individual component behavior:

• MirrorCoordinator: State loading, partition assignment, metadata persistence.
• MirrorMetadataManager: Topic creation, config sync, offset commit, ACL sync.
• MirrorFetcherThread: Epoch tracking, fetch processing, leader change handling.
• MirrorCommand: Command-line parsing, Admin API invocation, error handling.
• Protocol Serialization: Extended and new APIs serialization and deserialization.

Integration Tests

Integration tests will validate end-to-end functionality across multiple brokers:

• CLI Workflow: Create mirror with kafka-mirrors.sh, add topics, verify replication
• Basic Replication: Create mirror via API, replicate topic, verify data consistency
• Metadata Sync: Modify topic config in source, verify automatic sync to destination
• Partition Expansion: Add partitions to source topic, verify destination expands
• Consumer Groups: Commit offsets in source, verify replication to destination
• ACL Replication: Create ACL in source, verify creation in destination
• Leader Changes: Trigger leader election in source, verify fetcher reconnects
• Broker Failures: Stop destination broker, verify replication continues after recovery

System Tests

System tests will validate behavior under realistic production conditions:

• Performance Benchmark: Measure replication throughput and latency across WAN.
• Scalability Test: Replicate 1000 topics with 100,000 partitions across clusters.
• Failover Test: Simulate source cluster failure, measure consumer recovery time.
• Long-Running Stability: Run continuous replication for 7 days, verify no memory leaks or performance degradation.
• Security Validation: Test all authentication mechanisms (SASL PLAIN, SCRAM, Kerberos, mTLS) via kafka-mirrors.sh config files.

Rejected Alternatives

1. Keep using MirrorMaker 2:
   This KIP is to address the drawbacks existing MirrorMaker 2 as described in the motivation section.