Page History

...

On collision, the cluster hash is perturbed with a random salt until a free slot is found among free regions. Collision resolutions must survive leader elections. Without persistence, a new leader would recompute the mapped PID because the ephemeral origins map is lost. The computed salt is therefore stored as a record in the __mirror_state internal topic, which is replicated across the cluster.

Key schema:

# key
{
  "apiKey": 3,
  "type": "coordinator-key",
  "name": "MirrorPidCollisionKey",
  "validVersions": "0",
  "flexibleVersions": "none",
  "fields": [
    { "name": "TopicId", "type": "uuid", "versions": "0",
      "about": "The destination topic ID where the collision occurred."},
    { "name": "Partition", "type": "int32", "versions": "0",
      "about": "The partition index."},
    { "name": "SourceClusterId", "type": "string", "versions": "0",
      "about": "The source cluster UUID whose producer ID was remapped."},
    { "name": "OriginalPid", "type": "int64", "versions": "0",
      "about": "The original producer ID on the source cluster before mapping."}
  ]
}

Value schema:

# value
{
  "apiKey": 3,
  "type": "coordinator-value",
  "name": "MirrorPidCollisionValue",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Salt", "type": "int64", "versions": "0",
      "about": "The salt added to the cluster hash to resolve the collision."},
    { "name": "CollidingClusterId", "type": "string", "versions": "0",
      "about": "The cluster UUID that already held the mapped PID slot."},
    { "name": "TimestampMs", "type": "int64", "versions": "0",
      "about": "Wall clock time when the collision was resolved."}
  ]
}

On leader election, the coordinator loads collision records from from __mirror_state  and populates the collision cache. When mapping a PID, checks this cache first. If a known salt exists, it is applied directly without re-probing the origins map. A null value (tombstone) deletes the record when the corresponding producer state expires.

...