Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

RPC

Component

ACL Operation

ACL Resource

Purpose

FetchMFTReadTopicData replication
MetadataMMMDescribeTopicTopic discovery and leader tracking
DescribeConfigsMMMDescribeConfigsTopicTopic configuration sync
ListGroupsMMMDescribeGroupConsumer group offset sync
OffsetFetchMMMDescribeGroupConsumer group offset sync
DescribeAclsMMMDescribeClusterACL synchronization
LastMirroredEpochsDescribeMirrorsMCReadClusterLog truncation when preparing
ApiVersionsMMM

Feature negotiation
ListOffsetsMFTDescribeTopicOffset bounds discovery
OffsetsForLeaderEpochMFTDescribeTopicLeader epoch validation for truncation

...

  1. User sends RemoveTopicsFromMirror request with topics and mirror name.
  2. The controller validates each topic belongs to the specified mirror and is in MIRRORING state. It then updates the topic config by appending the .removed suffix, e.g. mirror.name=my-mirror.removed, generating a ConfigRecord.
  3. When the MirrorMetadataManager gets notified, it detects the .removed suffix on mirror.name. It queries the current mirror partition state from the coordinator, and transitions the mirror partition to STOPPING.
  4. During STOPPING:
    1. The MirrorFetcherManager removes all fetcher threads for the affected partitions, stopping replication.
    2. Bump the leader epoch for the partitions to ensure monotonically increasing epochs for new records.
    3. The log is truncated to LSO for transactional consistency.
    4. The LME is recorded as LastMirroredEpochsKeysKey/LastMirroredEpochsValue LastMirrorEpochsValue records into the __mirror_state topic for potential future failback.
    5. A MIRROR_PID_RESET control record is written to the partition log, which expires all ProducerStateManager entries so that new producers get fresh PIDs with no collision risk.
  5. The state transitions from STOPPING to STOPPED. The read only flag is cleared and the topic becomes writable. New producers can start producing with fresh PIDs starting at sequence 0 and a higher leader epoch.

...

  1. The user sends DescribeMirrorsRequest with optional mirror names (empty means all mirrors).
  2. The broker handler queries two sources:

    1. The ReplicaManager which provides source offset, destination offset, and lag for each partition.

    2. The MirrorCoordinator which provides the current partition state from the metadata manager cache.
  3. The request is forwarded to each broker that only reports partitions for which it has lag information or is the partition leader. This avoids duplicate reporting across brokers.

  4. For each partition, the response includes: mirror name, topic name, partition ID, source offset, destination offset, lag, and current state, and LME.
  5. No metadata records are written. This is a read only operation.

...

Returns the current mirroring status, state, and configuration for the specified mirror topics on the destination cluster. Allows destination cluster partition leaders to query the LME from the source cluster.

DescribeMirrorsRequest

Code Block
{
  "apiKey": TBD,
  "type": "request",
  "listeners": ["broker"],
  "name": "DescribeMirrorsRequest",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "MirrorNames", "type": "[]string", "versions": "0+", "entityType": "mirrorName",
      "about": "The names of the mirrors to describe. Null or empty array means all mirrors." },
    { "name": "IncludeAuthorizedOperations", "type": "bool", "versions": "0+", "default": "false",
      "about": "Whether to include authorized operations." }
  ]
}

...

Code Block
{
  "apiKey": TBD,
  "type": "response",
  "name": "DescribeMirrorsResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
      "about": "The top-level error message, or null if there was no error." },
    { "name": "Mirrors", "type": "[]DescribedMirror", "versions": "0+",
      "about": "Each described mirror.", "fields": [
      { "name": "ErrorCode", "type": "int16", "versions": "0+",
        "about": "The error code, or 0 if there was no error." },
      { "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
        "about": "The cluster mirror name." },
      { "name": "Topics", "type": "[]TopicPartitions", "versions": "0+",
        "about": "Each topic in the mirror.", "fields": [
        { "name": "TopicName", "type": "string", "versions": "0+",
          "about": "The topic name." },
        { "name": "Partitions", "type": "[]PartitionDetail", "versions": "0+",
          "about": "Each partition detail.", "fields": [
          { "name": "PartitionIndex", "type": "int32", "versions": "0+",
            "about": "The partition index." },
          { "name": "SourceOffset", "type": "int64", "versions": "0+", "default": "-1",
            "about": "The high watermark offset from the source cluster leader, or -1 if not yet available." },
          { "name": "DestinationOffset", "type": "int64", "versions": "0+", "default": "-1",
            "about": "The log end offset on the destination cluster, or -1 if not yet available." },
          { "name": "Lag", "type": "int64", "versions": "0+", "default": "-1",
            "about": "The lag (source offset - destination offset), or -1 if not yet available." },
          { "name": "State", "type": "string", "versions": "0+",
            "about": "The partition state." },
          { "name": "", "type": "int32", "versions": "0+", "default": "-1",
            "about": "The last mirror leader epoch, or -1 if not available." } 
        ]}
      ]}
    ]}
  ]
}

ReadMirrorStates

...

Code Block
{
  "apiKey": TBD,
  "type": "response",
  "name": "ReadMirrorStatesResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
      "about": "The top-level error message, or null if there was no error." },
    { "name": "Topics", "type": "[]TopicResult", "versions": "0",
      "about": "The read results for the topics.", "fields": [
      { "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionResult", "versions": "0",
        "about": "The results for the partitions.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0",
          "about": "The partition index." },
        { "name": "LastMirroredOffsetLastMirrorEpoch", "type": "int64int32", "versions": "0", default": "-1",
          "about": "The last mirrored offset mirror leader epoch, or -1 if not available." },
        { "name": "State", "type": "int8", "versions": "0+",
          "about": "The mirror partition state." },
        { "name": "ErrorCode", "type": "int16", "versions": "0",
          "about": "The error code, or 0 if there was no error." }
      ]}
    ]}
  ]
}

...

Code Block
{
  "apiKey": TBD,
  "type": "request",
  "listeners": ["broker", "controller"],
  "name": "WriteMirrorStatesRequest",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
      "about": "The mirror name." },
    { "name": "Topics", "type": "[]TopicData", "versions": "0",
      "about": "The data for the topics.", "fields": [
      { "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionData", "versions": "0",
        "about": "The data for the partitions.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0",
          "about": "The partition index." },
        { "name": "LastMirroredOffset", "type": "int64int32", "versions": "0", "default": "-1",
          "about": "The last mirrored offset mirror leader epoch, or -1 if not available." },
        { "name": "State", "type": "int8", "versions": "0+",
          "about": "The mirror partition state." }
      ]}
    ]},
    { "name": "RemovedTopics", "type": "[]string", "versions": "0+", "about": "The topic names to be removed." }
  ]
}

...

Code Block
{
  "apiKey": TBD,
  "type": "response",
  "name": "WriteMirrorStatesResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
      "about": "The top-level error message, or null if there was no error." },
    { "name": "Topics", "type": "[]TopicResult", "versions": "0",
      "about": "The write results for the topics.", "fields": [
      { "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionResult", "versions": "0",
        "about": "The results for the partitions.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0",
          "about": "The partition index." },
        { "name": "ErrorCode", "type": "int16", "versions": "0",
          "about": "The error code, or 0 if there was no error." }
      ]}
    ]}
  ]
}

...

BumpLeaderEpochs

Allows destination cluster partition leaders to query the LME from the source cluster.

LastMirroredEpochsRequest

Code Block
{
  "apiKey": TBD,
  "type": "request",
  "listeners": ["broker", "controller"],
  "name": "LastMirroredEpochsRequest",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
      "about": "The cluster mirror name." },
    { "name": "Topics", "type": "[]TopicData", "versions": "0",
      "about": "The data for the topics.", "fields": [
      { "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionData", "versions": "0",
        "about": "The data for the partitions.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0",
          "about": "The partition index." }
      ]}
    ]}
  ]
}

LastMirroredEpochsResponse

Code Block
{
  "apiKey": 97,
  "type": "response",
  "name": "LastMirroredEpochsResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
      "about": "The top-level error message, or null if there was no error." },
    { "name": "Topics", "type": "[]TopicResult", "versions": "0",
      "about": "The results for the topics.", "fields": [
      { "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionResult", "versions": "0",
        "about": "The results for the partitions.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0",
          "about": "The partition index." },
        { "name": "LastMirroredEpoch", "type": "int32", "versions": "0", "default": "-1",
          "about": "The last mirrored epoch, or -1 if not available." },
        { "name": "ErrorCode", "type": "int16", "versions": "0",
          "about": "The error code, or 0 if there was no error." }
      ]}
    ]}
  ]
}

BumpLeaderEpochs

Allows destination cluster partition leaders to bump the leader epoch on the destination bump the leader epoch on the destination controller so that it is at least as high as the source cluster's leader epoch.

...

Code Block
{
  "apiKey": TBD,
  "type": "response",
  "name": "BumpLeaderEpochsResponse",
  // Version 0 is the initial version.
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "Topics", "type": "[]TopicPartitions", "versions": "0+",
      "about": "Each topic in the mirror.", "fields": [
      { "name": "TopicName", "type": "string", "versions": "0+",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]PartitionDetail", "versions": "0+",
        "about": "Each partition state.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "ErrorCode", "type": "int16", "versions": "0",
          "about": "The error code, or 0 if there was no error." }
      ]}
    ]}
  ]
}

Metadata Records

...

LastMirrorEpochs

The greatest leader epoch of a given partition that a destination cluster recognizes from the source cluster.

Code Block
{
  "apiKey": 1,
  "type": "coordinator-key",
  "name": "LastMirroredEpochsKeysKey",
  "validVersions": "0",
  "flexibleVersions": "none",
  "fields": [
    { "name": "MirrorName", "type": "string", "versions": "0",
      "about": "The cluster mirror name."}
  ]
}

{
  "apiKey": 1,
  "type": "coordinator-value",
  "name": "LastMirroredEpochsValueLastMirrorEpochsValue",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "Topics", "type": "[]Topic", "versions": "0+",
      "about": "The mirror topics for which we want to store the last mirrored epochs.",  "fields": [
      { "name": "Name", "type": "string", "versions": "0",
        "about": "The topic name." },
      { "name": "Partitions", "type": "[]Partition", "versions": "0+",
        "about": "Each partition to record the last mirrored epochs.", "fields": [
        { "name": "PartitionIndex", "type": "int32", "versions": "0+",
          "about": "The partition index." },
        { "name": "LastMirroredEpoch", "type": "int32", "versions": "0+",
          "about": "The last mirror mirroredleader epoch for this partition." }
      ]}
    ]}
  ]
}

...

Key

Description

Default

mirror.name

Identifies the mirror that manages this topic. Topics with this configuration set are read-only and can only be modified through mirror management APIs. This property is filtered out from DescribeConfigs responses to avoid exposing internal state to users.


mirror.replication.throttled.replicas

A list of replicas for which log replication should be throttled on the mirror follower node. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic."


mirror.support.unclean.leader.election

When enabled, LastMirroredEpoch log LME log truncation waits for all replicas (not just ISR members) to join the ISR and complete the truncation.

false

...

CodeNameMessageUsed By
3UNKNOWN_TOPIC_OR_PARTITIONThe topic does not exist on the target clusterRemoveTopicsFromMirror, PauseMirrorTopics, ResumeMirrorTopics
15COORDINATOR_NOT_AVAILABLEThe mirror coordinator is not activeWriteMirrorStates, ReadMirrorStates, LastMirroredOffsets
31CLUSTER_AUTHORIZATION_FAILEDThe client is not authorized to perform the mirror operation

WriteMirrorStates, ReadMirrorStates, LastMirroredOffsets

35UNSUPPORTED_VERSIONCluster mirroring is disabled (mirror.version=0)

CreateMirror, AddTopicsToMirror, RemoveTopicsFromMirror, PauseMirrorTopics, ResumeMirrorTopics, ListMirrors, DescribeMirrors, DeleteMirror

TBDMIRROR_AUTHORIZATION_FAILEDMirror authorization failed

CreateMirror, AddTopicsToMirror, RemoveTopicsFromMirror, PauseMirrorTopics, ResumeMirrorTopics, DeleteMirror

TBDREAD_ONLY_TOPICThe topic is read-only because it is a mirror topic on the target clusterProduce
TBDINVALID_MIRROR_NAMEThe mirror name does not meet the naming rulesCreateMirror
TBDUNKNOWN_MIRRORThe topic is not assigned to any mirrorRemoveTopicsFromMirror, PauseMirrorTopics, ResumeMirrorTopics
TBDTOPIC_ALREADY_IN_MIRRORThe topic is already assigned to a mirrorAddTopicsToMirror
TBDTOPIC_NOT_IN_MIRRORThe topic does not belong to the specified mirrorRemoveTopicsFromMirror, PauseMirrorTopics, ResumeMirrorTopics
TBDMIRROR_TOPIC_ALREADY_PAUSEDThe mirror topic is already pausedPauseMirrorTopics
TBDMIRROR_TOPIC_NOT_PAUSEDThe mirror topic is not pausedResumeMirrorTopics
TBDMIRROR_TOPIC_BEING_REMOVEDThe mirror topic is being removedResumeMirrorTopics
TBDMIRROR_NOT_EMPTYThe mirror still has active or non-removed topicsDeleteMirror

...


MirrorMaker 2Cluster Mirroring

Offset Translation

Lossy, requires remapping, causes reprocessing overhead

None needed, offsets preserved exactly

Metadata Sync

Requires separate connector configuration (MirrorSourceConnector, MirrorCheckpointConnector)

Automatic (topics, configs, consumer groups, ACLs)

Transactional Topics

Markers copied as regular records, incomplete transactions possible during replication

Markers mirrored, LSO truncation ensures consistency before failover

Topic Write Protection

Not supported (mirror topics always writable)

Read-only enforcement during mirroring, writable only after explicit failover

Tiered Storage

Fetches from broker (which reads from remote storage)

Not initially supported (future work)

Active-Active

Supported via topic prefixing and cycle detection

Not supported (read-only enforcement prevents cycles)

Share Groups

Not supported

Supported

Failback

Full re-mirror from offset 0

Delta sync using LastMirroredOffsetDescribeMirror

Topic Name Preservation

No, destination topics prefixed with source cluster alias (e.g., source.topic-name)

Yes, same topic name as source

Topic ID Preservation

No, destination gets new topic ID

Yes: same topic ID as source

...