...

New Approach: Barrier Control Batch

First of all the PID mapping logic is updated to skip it for positive PIDs, making it idempotent. Rather than addressing collisions at runtimeRather than transforming PIDs at write time, this approach proactively eliminates expires stale producer state on failover. When mirroring stops, no mirrored producer remains active: the fetcher has been removed and the partition is about to become writable. All negative PIDs in the ProducerStateManager (PSM) The key insight is that during mirroring, the destination partition is read-only: no local producers exist, so all PSM entries originate from mirrored data. When mirroring stops, all PSM entries are stale and can be safely expired. Records from the source are stored as-is on the destination, with no PID modification, which otherwise would require a checksum recalculation. A MIRROR_PID_RESET control record  control batch (type 7) is written to each destination partition's log during the STOPPING state transition, after the fetcher has been removed and truncation to last stable offset is complete, but before the partition becomes writable. The dump tool is enhanced to deserialize and display

The mirror partition state transitions are:

• STOPPING: remove fetchers, truncate to LSO, persist last mirrored offsets, write MIRROR_PID_RESET barrier
• STOPPED:  partition is writable (terminal state, no actions)

The key follows the standard control record format records. Control batches are filtered out by the consumer fetcher, so the barrier is invisible to application consumers, just like transaction markers.The key is a standard control record key (version=0, type=7), while the value has the following schema. The value uses the MirrorPidResetRecord schema:

{
  "type": "data",
  "name": "MirrorPidResetRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Version", "type": "int16", "versions": "0",
      "about": "The version of the mirror PID reset record."},
    { "name": "SourceClusterId", "type": "string", "versions": "0",
      "about": "The source cluster UUID for verification."}
  ]
}

The SourceClusterId field serves two purposes:

...

 field records which source cluster the mirrored data

...

came from, enabling future validation (e.g. detecting unexpected source cluster changes) and data provenance tracing from the log itself

...

.

When the leader writes this barrier :

...

batch is

...

On replica recovery or log loading, the barrier is replayed from the log, triggering the same PSM expiration on followers. This ensures all replicas converge to the same clean producer state.

...

encountered during append or during log recovery, all producer entries are removed from the PSM. This ensures leaders, followers, and recovery all handle the barrier consistently. Because the partition is read-only during mirroring, all PSM entries originate from mirrored data. Expiring all entries is safe: no local producer state exists to preserve. Control batches are filtered out by the consumer fetcher via isControlBatch checks. The barrier is invisible to application consumers, just like transaction markers (COMMIT/ABORT). The log dump tool is enhanced to deserialize and display MIRROR_PID_RESET records.

Supported Topologies

The barrier approach works correctly with all practical mirroring topologies:                                                                                                                                                                                                            

• Active-passive (A to B): B mirrors from A, stores records as-is. On failover, barrier expires all PSM entries. Local producers get fresh PIDs from the coordinator with no collision risk.                                                  

...

•                                          
• Failback (A to B, then B to A): After failover, B becomes writable. Later, A starts mirroring from B, stores B's records as-is. B's barrier record is included in the fetched data and appended to A's log, but it does not trigger PSM expiration on A. The propagated barrier is inert during active mirroring: it sits in the log as a passive control batch. When A later stops mirroring from B, A's own barrier expires all PSM entries, producing a clean slate. The

With this setup, all negative PIDs are always coming from a single source cluster where no collision is possible.

When we have a mirroring chain from A to B to C :

• While B mirrors from A, B is read-only. No local producers exist on B, so no positive PIDs are created. All PIDs in B's log are negative (mapped from A's source PIDs).
• When B stops mirroring (failover), the barrier expires all negative PIDs in B's PSM.
• If C was mirroring from B, C received batches with negative PIDs from B. The pid >= 0 guard prevented C from double-mapping these: they were stored as-is on C.
• When C stops mirroring, C's barrier expires all negative PIDs, producing a clean slate.
• After failover, fresh local producers on B or C get positive PIDs from the coordinator, which never collide with the now-expired negative space.

Reverse Mirroring and Truncation

After a failover from A to B, the operator may later reverse the direction and mirror B back to A (failback).

When A begins mirroring from B:

• A transitions through PREPARING, which truncates its log to the last mirrored offset. This removes any local data A may have accumulated after B originally stopped mirroring from it, realigning A's log with B's offsets.
• B's local producers (created after failover) have positive PIDs. A maps them to negative space using the mapping rule.
• When the barrier record is replayed on A, it does not trigger PSM expiration of any negative PIDs, because this only happens when transitioning to STOPPED state.
• The stop cycle can repeat safely in either direction because each failover produces a clean PSM state via the barrier, and each new mirroring session starts from a truncated, offset-aligned log.
• Fan-out (A to B, A to C): B and C mirror independently from A, each with its own PSM per partition. On failover, each writes its own barrier independently.
• Fan-in (A to C, B to C, different topics): Each topic's partitions have independent PSMs. The barrier is written per partition during the STOPPING transition of each mirror.
• Chain (A to B to C): B mirrors from A, stores records as-is. C mirrors from B, stores records as-is. On failover at any point in the chain, the barrier expires all PSM entries on the stopping node. Longer chains work inductively by the same principle.