Page History

...

Cluster Mirroring fully supports log compacted topics, preserving both compacted records and offset gaps from the source cluster. When a topic uses cleanup.policy=compact, Kafka removes obsolete records with duplicate keys, creating gaps in the offset sequence. For example, if a source partition contains offsets 0-100 0–100 and compaction removes records at offsets 30-40 and 60-7030–40 and 60–70, the remaining records will have gaps: offsets 0-29, 41-59, and 71-100 0–29, 41–59, and 71–100 are present, while offsets 30–40 and 60–70 are missing. The mirror leader replicates these compacted log segments exactly as they exist in the source cluster, maintaining the same offset assignments and gaps. After failover, when the mirror topic becomes writable, log compaction continues normally in the destination cluster according to the topic's compaction policy, and any new records produced locally will fill in after the highest mirrored offset.

...

Cluster Mirroring handles topic retention policies by periodically synchronizing the topic configurations from the source cluster, ensuring that the topic retention policies are consistent. When the source cluster applies retention policies, older log segments are deleted and the log start offset (LSO) advances. For example, if a topic originally contained offsets 0-100 and retention deletes offsets 0-99, the source cluster's log start offset becomes 100. When the mirror leader fetches from the source, it discovers the new log start offset LSO and updates its local log start offset LSO to match, creating the same offset gap. If a mirror follower attempts to fetch from an offset below the source cluster's log start offset LSO (e.g. fetching offset 50 when log start offset is 100), the source broker returns an OffsetOutOfRangeException. The mirror leader handles this by truncating its local log to the source's current log start offset LSO and resuming fetching from that point. This ensures the destination cluster mirrors the current retention state of the source cluster without attempting to replicate already-deleted data.

...

Each phase lists only groups of its own type on both source and destination, preventing a consumer group on the source from overwriting a share group with the same name on the destination (or vice versa). Kafka enforces that consumer group and share group names must be unique within a single cluster. When a name conflict does occur across types, the offset commit operation will fail for the affected group, which is logged and skipped without affecting other groups. Users can resolve these conflicts by deleting the conflicting group in the destination cluster or excluding it from offset synchronization. These conflicts affect only offset synchronization and do not impact data mirroring itself. Fetched offsets are filtered to only include partitions belonging to actively mirrored topics, preventing offset ping-pong in bidirectional setups and excluding internal topics. Groups that are active on the destination (non-EMPTY, non-DEAD) are skipped. Offset sync prepares the destination for failover; once consumers are active there, they own their offsets. When they stop and the group becomes EMPTY, sync resumes automatically.

During offset synchronization, the committed offset in the destination cluster may temporarily exceed the current log end offset (LEO) of the mirror topic. For example, if a consumer commits offset 100 in the source cluster but the destination cluster has only mirrored up to offset 80, the offset 100 is still committed to the destination. This is acceptable because the mirror leader continues fetching data and the LEO will eventually catch up. However, if a failover occurs before the mirrored data catches up, consumers attempting to resume from offset 100 will receive an OffsetOutOfRangeException. To handle this gracefully, consumers should configure auto.offset.reset=latest (default) when consuming from mirror topics, ensuring that if a committed offset is beyond the current LEO after failover, the consumer automatically resets to the latest available offset.

...

Cluster Mirroring supports comprehensive security controls through both authorization and authentication mechanisms. This ensures that only authorized principals can establish and manage cluster mirrors. When configuring a mirror, operators specify the access control lists (ACLs) that should be synchronized from the source cluster, and these ACLs are periodically replicated to the destination cluster in order to maintain consistent access control policies across both environments. An operator can grant ClusterMirror:*:CREATE,ALTER,DESCRIBE for full mirror management, or scope it to specific mirrors like ClusterMirror:my-mirror:DESCRIBE for read-only monitoring of a single mirror, without granting any broker-level privileges.

When connecting to the source cluster, Cluster Mirroring requires only the bootstrap server address and appropriate credentials, no other sensitive cluster information is exposed or required. The destination cluster's mirror configuration supports all standard Kafka authentication mechanisms including TLS/SSL for encrypted transport and SASL for client authentication. Each mirror can be configured with its own security settings, allowing different mirrors to connect to source clusters with varying security requirements. This enables secure cross-cluster replication even when source and destination clusters use different authentication protocols or when connecting across security boundaries such as on-premises to cloud environments. All credentials are stored as mirror configuration records in the destination cluster metadata log, and used exclusively for establishing authenticated connections to the source cluster.

...

An operator can grant ClusterMirror:*:CREATE,ALTER,DESCRIBE for full mirror management, or scope it to specific mirrors like ClusterMirror:prod-dr:DESCRIBE for read-only monitoring of a single mirror, without granting any broker-level privileges.

Idempotent Producer

Idempotent Producer

During The approach is to proactively expire the stale producer state on failover. The key insight is that, during mirroring, the destination partition is read-only: no local producers exist, so all ProducerStateManager (PSM) entries originate from mirrored data. When mirroring stops (failover), all PSM entries are stale and can be safely expired. Records from the source are stored as-is on the destination, with no PID modification, which otherwise would require a checksum recalculation. A MIRROR_PID_RESET control record is written to each destination partition's log during the STOPPING state transition, just before the partition becomes writable.

...

• Active-passive (A to B): B mirrors from A, stores records as-is. On failover, the MIRROR_PID_RESET record expires all PSM entries. Local producers get fresh PIDs from the coordinator with no collision risk.
• Failback (A to B, then B to A): After failover, B becomes writable. Later, A starts mirroring from B and truncates its log to the LSOLME. A then stores B's records as-is. B's MIRROR_PID_RESET record is included in the fetched data and appended to A's log, triggering PSM expiration on A. This is consistent with the general rule: when the MIRROR_PID_RESET record is encountered during append or during log recovery, all producer entries are removed from the PSM. A will write its own MIRROR_PID_RESET record when it eventually stops mirroring from B, producing a clean slate before A becomes writable again.
• Fan-out (A to B, A to C): B and C mirror independently from A, each with its own PSM per partition. On failover, each writes its own MIRROR_PID_RESET record independently.
• Fan-in (A to C, B to C, different topics): Each topic's partitions have independent PSMs. The MIRROR_PID_RESET record is written per partition during the STOPPING transition of each mirror.
• Chain (A to B to C): B mirrors from A, stores records as-is. C mirrors from B, stores records as-is. On failover at any point in the chain, the MIRROR_PID_RESET record expires all PSM entries on the stopping node. Longer chains work inductively by the same principle.

Exactly-Once Semantics

Cluster mirroring Mirroring does not support exactly-once semantics (EOS) across clusters. Transactional support means that after The only guarantee is that, after a failover, the destination cluster will not have hanging transactions that block READ_COMMITTED consumers, but it does not guarantee that committed records from the source are atomically synced to the destination. The mirror fetcher thread uses READ_UNCOMMITTED isolation, so records from uncommitted transactions are replicated to the destination before the source decides them. This reduces replication lag compared to READ_COMMITTED, but means uncommitted data is visible to READ_UNCOMMITTED consumers on the destination before failover. When stop mirroring is triggered, ongoing transactions are decided by appending explicit ABORT markers, preserving all previously committed data.

...

Transaction 4001 was committed at the source but aborted at the destination because the COMMIT marker (offset 3) had not yet been replicated. Transaction 4002 was correctly aborted at both clusters. Applications that require strict transactional guarantees across clusters should implement deduplication or reconciliation logic after failover. Additionally, the kafka-transactions tool can only abort transactions originated from the local cluster. It cannot abort transactions replicated via mirroring because the __transaction_state topic is not mirrored. Hanging Ongoing transactions from mirrored data are resolved exclusively by the by the STOPPING transition flow described above.

...

Currently, mirroring is asynchronous. The source cluster acknowledges the producer without waiting for the destination to replicate the data. Sync Synchronous mirroring would guarantee that records are replicated to the destination cluster before the source acknowledges the produce request, providing stronger durability guarantees at the cost of higher latency. This would be useful for workloads where zero data loss across clusters is a strict requirement.

...