DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
This KIP introduces native cluster mirroring to address the limitations of MirrorMaker 2 described in the motivation section. The following tables provide a detailed comparison across deployment, features, and performance characteristics.
Deployment Comparison
| MirrorMaker2 | KIP-1279 Cluster Mirroring | |
|---|---|---|
| Architecture | External Connect workers (separate JVM) | Integrated into Kafka brokers using native replication protocol |
| Operational Complexity | Manage Connect cluster lifecycle independently | Unified with broker operations |
| Monitoring | Separate Connect metrics and dashboards | Standard Kafka JMX metrics |
Feature Support Comparison
| MirrorMaker2 | KIP-1279 Cluster Mirroring | |
|---|---|---|
Offset Translation | Lossy, requires remapping, causes reprocessing overhead | None needed - offsets preserved exactly |
Metadata Sync | Requires separate connector configuration (MirrorSourceConnector, MirrorCheckpointConnector) | Automatic (topics, configs, consumer groups, ACLs) |
Transactional Topics | Markers copied as regular records, incomplete transactions possible during replication | Markers mirrored, LSO truncation ensures consistency before failover |
Topic Write Protection | Not supported - mirrored topics always writable | Read-only enforcement during mirroring, writable only after explicit failover |
Tiered Storage | Fetches from broker (which reads from remote storage) | Not initially supported (future work) |
Active-Active | Supported via topic prefixing and cycle detection | Not supported (read-only enforcement prevents cycles) |
Share Groups | Not supported | Supported |
Failback | Full re-mirror from offset 0 | Delta sync using LastMirroredOffset |
Topic Name Preservation | No - destination topics prefixed with source cluster alias (e.g., source.topic-name) | Yes - same topic name as source |
Topic ID Preservation | No - destination gets new topic ID | Yes - same topic ID as source |
Performance & Isolation Comparison
| MirrorMaker2 | KIP-1279 Cluster Mirroring | |
|---|---|---|
Compression Overhead | Decompress + recompress records | Preserve source compression (zero overhead) |
Failover Time | Offset translation + consumer group sync + bootstrap reconfiguration | Consumer group sync + bootstrap reconfiguration only |
Bandwidth Control (Source) | None (unless client quotas manually configured) | Client quotas (current), dedicated mirror throttling (future) |
Bandwidth Control (Destination) | None (unless client quotas manually) | Replica-level throttling (mirror.replication.throttled.rate) |
Resource Isolation (Source) | Shares network bandwidth and disk I/O with all consumers | Shares network bandwidth and disk I/O with all consumers (same - current), Isolated on replica-level (future) |
Resource Isolation (Destination | Separate JVM heap (memory isolated from brokers) but still consume resources from the broker as any client | Shares network, disk I/O; dedicated thread pool within broker JVM |
Malformed Batch Handling | Crashes Connect task, all partitions in task restart | Fails individual partition (FAILED state), others continue |
Blast Radius(Failure) | All partitions in task affected | Single partition affected |
Catch-Up Surge Protection | Connect worker heap may OOM, affects all tasks
| Throttling on destination replica + source quota prevent memory spikes
|
Unclean Leader Election | No protection against log divergence | Explicitly unsupported with warning logs |
Use Case Guidance:
- MirrorMaker 2: Best for active-active topologies, multi-region writes, complex routing scenarios
- Cluster Mirroring: Best for disaster recovery, failover, migration, simplified operations with exact offset preservation
Support Unclean Leader Election
...