Page History

A producer ID (PID) is a 64-bit identifier assigned by the broker to each idempotent or transactional producer. It has three key properties:

1. Unique: uniquely identifies a producer for idempotent deduplication and transaction tracking.
2. Stable: once assigned, a PID persists across producer sessions (for transactional producers) or until expiration.
3. Non-negative: valid PIDs are >= 0. The value -1 (NO_PRODUCER_ID) marks non-idempotent batches.

...

We identified the following scenarios caused by interleaving records from a PID collision:

1. Same epoch, wrong sequence: No OutOfOrderSequenceException. Batches are silently accepted under the same PID as they are coming from the leader (append origin == REPLICATION). The PSM cache entry is updated with whatever arrives last. Silent data corruption with zero signals, not even a warning.
2. Different epochs: No fencing exception. Lower epoch batch is accepted with a warning log. Both producers coexist under the same PID. Silent corruption, only a WARN log line as a hint.
3. Transactional interleaving: Commit/abort markers from one producer close the other's transaction. No exception. Silent transaction corruption.

...

{
  "type": "data",
  "name": "MirrorPidResetRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Version", "type": "int16", "versions": "0",
      "about": "The version of the mirror PID reset record."},
    { "name": "SourceClusterId", "type": "string", "versions": "0",
      "about": "The source cluster UUID for verification."}
  ]
}

The SourceClusterId field serves two purposes:

1. Validation: It enables future consistency checks to detect and reject unexpected source cluster changes during a mirror's lifetime, which would indicate a misconfiguration.
2. Diagnostics: It provides an immutable, durable record of which source cluster the mirrored data originated from, making it possible to trace data provenance from the log itself without relying on external state.

When the leader writes this barrier:

...