DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
A cluster mirror is a named, unidirectional replication channel from a remote source cluster to the local destination cluster. It is created by specifying a unique mirror name along with the source cluster's bootstrap servers and security credentials. Once a mirror is created, individual topics on the source cluster can be started, stopped, or paused for replication within it. Each mirror is a first-class entity managed through the Admin API and the kafka-cluster-mirrors.sh CLI tool, with its state persisted in a coordinator that manages cross-cluster replication.
...
The mirror name is stored as a topic-level internal configuration called mirror.name that has the same validation rules of topic names, and propagates through Kafka's metadata log as configuration change records. When topics are added to a mirror, the quorum controller generates configuration metadata records that are replicated to all brokers through the standard metadata update mechanism. Brokers monitor these configuration changes to detect when partitions they lead belong to a mirror, triggering the creation of mirror fetchers and enforcement of read-only semantics. This design ensures that mirror associations are visible, auditable, and manageable through standard Kafka tools while maintaining strict control over how mirroring relationships are established and modified.
Main Components
...
ClusterMirrorCoordinator
The ClusterMirrorCoordinator (MCCMC) manages Cluster Mirroring state using a partitioned coordinator partitioned coordinator pattern similar to the group and transaction coordinators.
...
RPC | Component | ACL Operation | ACL Resource | Purpose |
| Fetch | MFT | Read | Topic | Data replication |
| Metadata | MMM | Describe | Topic | Topic discovery and leader tracking |
| DescribeConfigs | MMM | DescribeConfigs | Topic | Topic configuration sync |
| ListGroups | MMM | Describe | Group | Consumer group offset sync |
| OffsetFetch | MMM | Describe | Group | Consumer group offset sync |
| DescribeAcls | MMM | Describe | Cluster | ACL synchronization |
| DescribeMirrorsDescribeClusterMirrors | MC | Read | Cluster | Log truncation |
| ApiVersions | MMM | Feature negotiation | ||
| ListOffsets | MFT | Describe | Topic | Offset bounds discovery |
| OffsetsForLeaderEpoch | MFT | Describe | Topic | Leader epoch validation for truncation |
...
RPC | Component | ACL Operation | ACL Resource | Purpose |
| CreateMirrorCreateClusterMirror | Controller | Create | ClusterMirror | New cluster mirror creation |
| StartMirrorTopics | Controller | Alter | ClusterMirror | Mirror topics creation |
| StartMirrorTopics | Controller | AlterConfigs | Topic | Mirror topics creation |
| StopMirrorTopics | Controller | Alter | ClusterMirror | Mirror topics removal (failover) |
| StopMirrorTopics | Controller | AlterConfigs | Topic | Mirror topics removal (failover) |
| PauseMirrorTopics | Controller | Alter | ClusterMirror | Mirror topics pause |
| PauseMirrorTopics | Controller | AlterConfigs | Topic | Mirror topics pause |
| ResumeMirrorTopics | Controller | Alter | ClusterMirror | Mirror topics resume |
| ResumeMirrorTopics | Controller | AlterConfigs | Topic | Mirror topics resume |
| DeleteMirrorDeleteClusterMirror | Controller | Alter | ClusterMirror | Delete a cluster mirror |
| ListMirrorsListClusterMirrors | Broker | Describe | ClusterMirror | Mirror topic listing |
| DescribeMirrorsDescribeClusterMirrors | Broker | Describe | ClusterMirror | Mirror topic describe (state, lag) |
| DescribeConfigs | Broker | DescribeConfigs | ClusterMirror | Mirror configuration describe |
| WriteMirrorStates | MC | ClusterAction | Cluster | Mirror partition state write |
| ReadMirrorStates | MC | ClusterAction | Cluster | Mirror partition state read |
| BumpLeaderEpochs | MC | ClusterAction | Cluster | Leader epoch bump when stopping |
| FindCoordinator | Broker | ClusterAction | Cluster | Mirror coordinator location |
| CreateTopics | MMM | Create | Topic | Topic creation |
| CreatePartitions | MMM | Alter | Topic | Partitions scaling |
| IncrementalAlterConfigs | MMM | AlterConfigs | ClusterMirror | Mirror configuration update |
| OffsetCommit | MMM | Read | Topic | Source CG offsets commit |
| OffsetCommit | MMM | Read | Group | Source CG offsets commit |
| CreateAcls | MMM | Alter | Cluster | Source ACLs creation |
| DeleteAcls | MMM | Alter | Cluster | Source ACLs removal |
...
| Code Block | ||
|---|---|---|
| ||
# when the source cluster is back, the operator can failback by creating a mirror with the same name echo "bootstrap.servers=localhost:9094" > /tmp/my-mirror.properties bin/kafka-cluster-mirrors.sh --bootstrap-server :9091 --create --mirror my-mirror --mirror-config /tmp/my-mirror.properties bin/kafka-cluster-mirrors.sh --bootstrap-server :"9091 --start --topic .* --mirror my-mirror # 9091 (destination) <----- 9094 (source) |
Create Mirror
- The user sends CreateMirror CreateClusterMirror requests to any broker with the mirror name and mirror related properties (bootstrap servers, security settings, etc.).
- The broker forwards the request to the active controller.
- The controller saves the properties into the metadata log as ConfigRecord entries with type MIRROR.
- If this is the first mirror being created, the controller also auto creates the __mirror_state internal topic.
- All brokers receive the metadata update and the MirrorMetadataManager registers the new mirror configuration.
...
Delete Mirror
- The user sends a DeleteMirror DeleteClusterMirror request with the mirror name.
- The controller validates that the mirror is empty (no topics assigned) or all its partitions are in STOPPED state.
- If valid, the controller tombstones the mirror configuration in the cluster metadata log, removing all ConfigRecord entries for the mirror.
- The mirror state records in __mirror_state internal topic are also tombstoned.
- Any remaining coordinator state is shut down, source cluster connections are closed, and the mirror name becomes available for reuse.
- After deletion, failback using this mirror configuration is no longer possible.
List Mirrors
The user sends ListMirrorsRequest ListClusterMirrors request to any broker (no parameters required).
- The broker handler gets all configured mirror partitions from, which reads from the in memory metadata cache.
- For each authorized mirror, the broker returns: mirror name, source cluster ID, source bootstrap servers, and topic count.
- No metadata records are written. This is a read only operation against the local metadata cache.
Describe Mirrors
- The user sends DescribeMirrorsRequest DescribeClusterMirrors request with optional mirror names (empty means all mirrors).
The broker handler queries two sources:
The ReplicaManager which provides source offset, destination offset, and lag for each partition.
- The MirrorCoordinator which provides the current partition state from the metadata manager cache.
The request is forwarded to each broker that only reports partitions for which it has lag information or is the partition leader. This avoids duplicate reporting across brokers.
- For each partition, the response includes: mirror name, topic name, partition ID, source offset, destination offset, lag, current state, and LME.
- No metadata records are written. This is a read only operation.
...
A new command-line tool kafka-cluster-mirrors.sh provides administrative operations for managing cluster mirrors:
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --help
Create cluster mirrors and manage mirrored topics.
Option Description
------ -----------
--bootstrap-server <String: server to REQUIRED: The destination Kafka server
connect to> to connect to.
--command-config <String: command Property file containing configs to be
config property file> passed to Admin Client.
--create Create a new cluster mirror from a
source cluster.
--delete Delete a cluster mirror.
--describe Describe a cluster mirror including
partition lag and state.
--exclude <String: exclude patterns> Comma-separated list of topic names or
regex patterns to exclude from
mirroring. Only valid with --start.
--help Print usage information.
--json Output description in JSON format
--list List all cluster mirrors.
--mirror <String: mirror> The name of the cluster mirror.
--mirror-config <String: mirror config Property file containing source
property file> cluster configs for mirroring.
--pause Pause mirroring for topics matching
the given patterns.
--resume Resume mirroring for previously paused
topics matching the given patterns.
--start Start mirroring topics matching the
given patterns.
--stop Stop mirroring topics matching the
given patterns.
--topics <String: topics> Comma-separated list of topic names or
regex patterns (e.g., 'my-topic,
orders-.*,payments').
--version Display Kafka version. |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --start \
--topics 'orders-.*' --exclude 'orders-internal' --mirror my-mirror
Started 2 mirror topic(s) in mirror my-mirror: [orders-us, orders-eu] |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --stop --topics 'orders-us' --mirror my-mirror
Stopped mirroring for 1 topic(s) in mirror my-mirror: [orders-us] |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --delete --mirror my-mirror
Deleted mirror my-mirror |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --pause --topics my-topic --mirror my-mirror
Paused mirroring for 1 topic(s) in mirror my-mirror: [my-topic] |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --resume --topics my-topic --mirror my-mirror
Resumed mirroring for 1 topic(s) in mirror my-mirror: [my-topic] |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --list
MIRROR TOPICS CLUSTER-ID BOOTSTRAP-SERVER
my-mirror 2 lBq12jYZRp-9wF3M9MPopg localhost:9091,localhost:9092
new-mirror 1 lBq12jYZRp-9wF3M9MPopg localhost:9091,localhost:9092 |
...
| Code Block | ||
|---|---|---|
| ||
$ bin/kafka-cluster-mirrors.sh --bootstrap-server :9094 --describe
MIRROR TOPIC PARTITION SOURCE-OFFSET DESTINATION-OFFSET LAG STATE
my-mirror bar 0 - - - STOPPED
my-mirror foo 0 69 66 3 MIRRORING
my-mirror foo 1 94 84 10 MIRRORING
my-mirror foo 2 94 90 4 MIRRORING
new-mirror baz 0 - - - PAUSED
new-mirror baz 1 - - - PAUSED |
...
| Code Block | ||
|---|---|---|
| ||
/** * Create a new cluster mirror. * * @param mirrorName The name of the cluster mirror * @param configs Configuration for the cluster mirror, including bootstrap servers and security settings * @param options Options for the create mirror operation * @return The CreateMirrorResultCreateClusterMirrorResult */ CreateMirrorResultCreateClusterMirrorResult createMirrorcreateClusterMirror(String mirrorName, Map<String, String> configs, CreateMirrorOptionsCreateClusterMirrorOptions options);/** * Create a new cluster mirror. * * @param mirrorName The name of the cluster mirror * @param configs Configuration for the cluster mirror, including bootstrap servers and security settings * @param options Options for the create mirror operation * @return The CreateMirrorResultCreateClusterMirrorResult */ CreateMirrorResultCreateClusterMirrorResult createMirrorcreateClusterMirror(String mirrorName, Map<String, String> configs, CreateMirrorOptionsCreateClusterMirrorOptions options); /** * Start mirroring for the specified topics. * * When topics are started in a mirror, they become read-only on the destination cluster and start * replicating data from the source cluster. This operation marks the specified topics with the * mirror name, preventing local writes and enabling the MirrorFetcherThread to begin replication. * * @param mirrorName The cluster mirror name * @param topics Set of topic names to start mirroring * @param options Options for the start mirror topics operation * @return The StartMirrorTopicsResult containing futures for each topic */ StartMirrorTopicsResult startMirrorTopics(String mirrorName, Set<String> topics, StartMirrorTopicsOptions options); /** * Options for {@link Admin#startMirrorTopics(String, Set, StartMirrorTopicsOptions)}. */ public class StartMirrorTopicsOptions extends AbstractOptions<StartMirrorTopicsOptions> { private List<String> includePatterns = List.of(); private List<String> excludePatterns = List.of(); private Map<String, StartMirrorTopicsRequestData.TopicData> topicMetadata = Map.of(); public StartMirrorTopicsOptions includePatterns(List<String> patterns) { this.includePatterns = patterns; return this; } public StartMirrorTopicsOptions excludePatterns(List<String> patterns) { this.excludePatterns = patterns; return this; } public StartMirrorTopicsOptions topicMetadata(Map<String, StartMirrorTopicsRequestData.TopicData> metadata) { this.topicMetadata = metadata; return this; } public List<String> includePatterns() { return includePatterns; } public List<String> excludePatterns() { return excludePatterns; } public Map<String, StartMirrorTopicsRequestData.TopicData> topicMetadata() { return topicMetadata; } } /** * Stop mirroring for the specified topics. * * This operation is typically used during failover scenarios when the destination cluster needs to * be promoted from passive (read-only mirror) to active (accepting writes). Stopping mirror topics * clears the mirrorName field from partition metadata, which allows producers to write * to these partitions. * * @param mirrorName The cluster mirror name * @param topics Set of topic names to stop mirroring * @param options Options for the stop mirror topics operation * @return The StopMirrorTopicsResult containing futures for each topic */ StopMirrorTopicsResult stopMirrorTopics(String mirrorName, Set<String> topics, StopMirrorTopicsOptions options); /** * Options for {@link Admin#stopMirrorTopics(String, Set, StopMirrorTopicsOptions)}. */ public class StopMirrorTopicsOptions extends AbstractOptions<StopMirrorTopicsOptions> { private List<String> patterns = List.of(); public StopMirrorTopicsOptions patterns(List<String> patterns) { this.patterns = patterns; return this; } public List<String> patterns() { return patterns; } } /** * Pause mirroring for the specified topics. * * Paused topics remain read-only on the destination cluster but stop fetching new data from the * source cluster. The mirror fetcher threads are removed for these partitions, preserving the * current replicated state. Mirroring can be resumed later with {@link #resumeMirrorTopics}. * * @param mirrorName The cluster mirror name * @param topics Set of topic names to pause mirroring * @param options Options for the pause mirror topics operation * @return The PauseMirrorTopicsResult containing futures for each topic */ PauseMirrorTopicsResult pauseMirrorTopics(String mirrorName, Set<String> topics, PauseMirrorTopicsOptions options); /** * Resume mirroring for previously paused topics. * * Resumed topics restart fetching data from the source cluster, picking up from where they * left off. New mirror fetcher threads are created and the partitions transition back to the * MIRRORING state. * * @param mirrorName The cluster mirror name * @param topics Set of topic names to resume mirroring * @param options Options for the resume mirror topics operation * @return The ResumeMirrorTopicsResult containing futures for each topic */ ResumeMirrorTopicsResult resumeMirrorTopics(String mirrorName, Set<String> topics, ResumeMirrorTopicsOptions options); /** * Delete a cluster mirror including its configuration. * * The mirror must be empty (no topics) or all its topics must have been removed (in STOPPED * state). After deletion, all mirror metadata are tombstoned and failback is no longer possible. * * @param mirrorName The cluster mirror name * @param options Options for the delete mirror operation * @return The DeleteMirrorResultDeleteClusterMirrorResult */ DeleteMirrorResultDeleteClusterMirrorResult deleteMirrordeleteClusterMirror(String mirrorName, DeleteMirrorOptionsDeleteClusterMirrorOptions options); /** * List the cluster mirrors available in the cluster. * * @param options The options to use when listing the mirrors. * @return The ListMirrorsResultListClusterMirrorsResult. */ ListMirrorsResultListClusterMirrorsResult listMirrorslistClusterMirrors(ListMirrorsOptionsListClusterMirrorsOptions options); /** * Describe cluster mirrors. * * This operation retrieves detailed information about cluster mirrors including: * - Topics being mirrored * - Partition-level lag information * - Mirroring state for each partition * * @param mirrorNames The names of the mirrors to describe * @param options The options to use when describing mirrors * @return The DescribeMirrorsResultDescribeClusterMirrorsResult */ DescribeMirrorsResultDescribeClusterMirrorsResult describeMirrorsdescribeClusterMirrors(Collection<String> mirrorNames, DescribeMirrorsOptionsDescribeClusterMirrorsOptions options); |
Protocol Changes
...
| Code Block |
|---|
// new request field in FetchPartition type
{ "name": "MirrorLeaderEpoch", "type": "int32", "versions": "19+", "default": "-1", "taggedVersions": "19+", "tag": 2, "ignorable": true,
"about": "The latest known mirror leader epoch." }
// new response field in PartitionData type
{ "name": "MirrorLeaderEpoch", "type": "int32", "versions": "19+", "default": "-1", "taggedVersions": "19+", "tag": 3, "ignorable": true,
"about": "The latest known mirror leader epoch." },
|
...
CreateClusterMirror
Allows users to create a mirror and supply its configuration. The broker validates that the mirror name is not already in use, contains only permitted characters, and does not end with .stopped or .paused suffix. Once validated, the request is forwarded to the controller, which persists the configuration in the metadata log.
| Code Block |
|---|
{
"apiKey": TBD,
"type": "request",
"listeners": ["broker", "controller"],
"name": "CreateMirrorRequestCreateClusterMirrorRequest",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
"about": "The cluster mirror name."},
{ "name": "Config", "type": "[]MirrorConfigClusterMirrorConfig", "versions": "0+",
"about": "The cluster mirror configurations.", "fields": [
{ "name": "Name", "type": "string", "versions": "0+", "mapKey": true,
"about": "The configuration key name." },
{ "name": "Value", "type": "string", "versions": "0+", "nullableVersions": "0+",
"about": "The value to set for the configuration key."}
]}
]
}
{
"apiKey": TBD,
"type": "response",
"name": "CreateMirrorResponseCreateClusterMirrorResponse",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+",
"about": "The error message, or null if there was no error." }
]
} |
...
| Code Block |
|---|
{
"apiKey": TBD,
"type": "request",
"listeners": ["broker", "controller"],
"name": "ResumeMirrorTopicsRequest",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "MirrorName", "type": "string", "versions": "0+",
"about": "The cluster mirror name." },
{ "name": "Topics", "type": "[]TopicData", "versions": "0+", "about": "The data for the topics.",
"fields": [
{ "name": "TopicId", "type": "uuid", "versions": "0+", "about": "The unique topic ID."},
{ "name": "TopicName", "type": "string", "versions": "0+", "mapKey": true, "entityType": "topicName",
"about": "The topic name." }
]}
]
}
{
"apiKey": TBD,
"type": "response",
"name": "ResumeMirrorTopicsResponse",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
"about": "The top-level error message, or null if there was no error." },
{ "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
"about": "The cluster mirror name." },
{ "name": "Topics", "type": "[]TopicResult", "versions": "0",
"about": "The results for the topics.", "fields": [
{ "name": "Name", "type": "string", "versions": "0", "entityType": "topicName",
"about": "The topic name." },
{ "name": "ErrorCode", "type": "int16", "versions": "0",
"about": "The error code, or 0 if there was no error." }
]}
]
} |
...
DeleteClusterMirror
Permanently deletes a cluster mirror, including its configuration. The mirror must be empty (no topics) or all its partitions must be in STOPPED state. After deletion, all metadata are tombstoned, making failback impossible. This is an irreversible operation.
| Code Block |
|---|
{
"apiKey": TBD,
"type": "request",
"listeners": ["broker", "controller"],
"name": "DeleteMirrorRequestDeleteClusterMirrorRequest",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
"about": "The cluster mirror name to delete."}
]
}
{
"apiKey": TBD,
"type": "response",
"name": "DeleteMirrorResponseDeleteClusterMirrorResponse",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+",
"about": "The error message, or null if there was no error." }
]
} |
...
ListClusterMirrors
Returns the current mirror names and their associated topic counts in the cluster. It also includes source cluster ID and bootstrap server.
| Code Block |
|---|
{
"apiKey": TBD,
"type": "request",
"listeners": ["broker"],
"name": "ListMirrorsRequestListClusterMirrorsRequest",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": []
}
{
"apiKey": TBD,
"type": "response",
"name": "ListMirrorsResponseListClusterMirrorsResponse",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
"about": "The top-level error message, or null if there was no error." },
{ "name": "Mirrors", "type": "[]ListedMirror", "versions": "0+",
"about": "Each mirror in the response.", "fields": [
{ "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
"about": "The cluster mirror name." },
{ "name": "SourceBootstrap", "type": "string", "versions": "0+",
"about": "The source cluster bootstrap servers." },
{ "name": "SourceClusterId", "type": "string", "versions": "0+", "default": "",
"about": "The source cluster ID, or empty if not yet resolved." },
{ "name": "TopicCount", "type": "int32", "versions": "0+", "default": "0",
"about": "The number of topics configured for this mirror. 0 indicates an empty mirror with no topics." }
]}
]
} |
...
DescribeClusterMirrors
Returns the current mirroring status, state, and configuration for the specified mirror topics on the destination cluster. Allows destination cluster partition leaders to query the LME from the source cluster.
| Code Block |
|---|
{
"apiKey": TBD,
"type": "request",
"listeners": ["broker"],
"name": "DescribeMirrorsRequestDescribeClusterMirrorsRequest",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "MirrorNames", "type": "[]string", "versions": "0+", "entityType": "mirrorName",
"about": "The names of the mirrors to describe. Null or empty array means all mirrors." },
{ "name": "IncludeAuthorizedOperations", "type": "bool", "versions": "0+", "default": "false",
"about": "Whether to include authorized operations." }
]
}
{
"apiKey": TBD,
"type": "response",
"name": "DescribeMirrorsResponseDescribeClusterMirrorsResponse",
// Version 0 is the initial version.
"validVersions": "0",
"flexibleVersions": "0+",
"fields": [
{ "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
"about": "The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "ErrorMessage", "type": "string", "versions": "0+", "nullableVersions": "0+", "default": "null",
"about": "The top-level error message, or null if there was no error." },
{ "name": "Mirrors", "type": "[]DescribedMirror", "versions": "0+",
"about": "Each described mirror.", "fields": [
{ "name": "ErrorCode", "type": "int16", "versions": "0+",
"about": "The error code, or 0 if there was no error." },
{ "name": "MirrorName", "type": "string", "versions": "0+", "entityType": "mirrorName",
"about": "The cluster mirror name." },
{ "name": "AuthorizedOperations", "type": "int32", "versions": "0+", "default": "-2147483648",
"about": "32-bit bitfield to represent authorized operations for this mirror." },
{ "name": "Topics", "type": "[]TopicPartitions", "versions": "0+",
"about": "Each topic in the mirror.", "fields": [
{ "name": "TopicName", "type": "string", "versions": "0+",
"about": "The topic name." },
{ "name": "Partitions", "type": "[]PartitionDetail", "versions": "0+",
"about": "Each partition detail.", "fields": [
{ "name": "PartitionIndex", "type": "int32", "versions": "0+",
"about": "The partition index." },
{ "name": "SourceOffset", "type": "int64", "versions": "0+", "default": "-1",
"about": "The high watermark offset from the source cluster leader, or -1 if not yet available." },
{ "name": "DestinationOffset", "type": "int64", "versions": "0+", "default": "-1",
"about": "The log end offset on the destination cluster, or -1 if not yet available." },
{ "name": "Lag", "type": "int64", "versions": "0+", "default": "-1",
"about": "The lag (source offset - destination offset), or -1 if not yet available." },
{ "name": "State", "type": "string", "versions": "0+",
"about": "The partition state." },
{ "name": "", "type": "int32", "versions": "0+", "default": "-1",
"about": "The last mirror leader epoch, or -1 if not available." }
]}
]}
]}
]
} |
...
Key | Description | Default |
|---|---|---|
mirror.topic.num.partitions | Number of partitions for __mirror_state internal topic. | 50 |
mirror.topic.replication.factor | Replication factor for __mirror_state internal topic. | 3 |
mirror.num.replica.fetchers | Number of fetcher threads per mirrored source broker, | 1 |
mirror.metadata.refresh.interval.ms | The interval in milliseconds at which the coordinator refreshes metadata from source clusters. This controls how frequently the coordinator polls source clusters to detect new topics and metadata changes. | 30000 |
mirror.replication.throttled.rate | A long representing the upper bound (bytes/sec) on replication traffic for mirrored follower node enumerated in the property “mirror.replication.throttled.replicas” (for each topic). This property can be only set dynamically. It is suggested that the limit be kept above 1MB/s for accurate behaviour. | MAX_LONG |
request.timeout.ms | Maximum amount of time in milliseconds the client will wait for the response of a request. | 30000 |
socket.* | Socket connection configurations. | |
replica.* | Fetcher threads configurations. |
Mirror
Set via CreateMirror CreateClusterMirror or IncrementalAlterConfigs. Stored in cluster metadata records.
...
| Code | Name | Message | Used By |
|---|---|---|---|
| 3 | UNKNOWN_TOPIC_OR_PARTITION | The topic does not exist on the target cluster | StopMirrorTopics, PauseMirrorTopics, ResumeMirrorTopics |
| 15 | COORDINATOR_NOT_AVAILABLE | The mirror coordinator is not active | WriteMirrorStates, ReadMirrorStates |
| 31 | CLUSTER_AUTHORIZATION_FAILED | The client is not authorized to perform the mirror operation | WriteMirrorStates, ReadMirrorStates |
| 35 | UNSUPPORTED_VERSION | Cluster mirroring is disabled (mirror.version=0) | CreateMirrorCreateClusterMirror, StartMirrorTopics, StopMirrorTopics, PauseMirrorTopics, ResumeMirrorTopics, ListMirrorsListClusterMirrors, DescribeMirrorsDescribeClusterMirrors, DeleteMirrorDeleteClusterMirror |
| TBD | READ_ONLY_TOPIC | The topic is read-only because it is a mirror topic on the target cluster | Produce |
| TBD | INVALID_CLUSTER_MIRROR_NAME | The cluster mirror name does not meet the naming rules | CreateMirrorCreateClusterMirror |
| TBD | CLUSTER_MIRROR_ALREADY_EXISTS | The cluster mirror already exists | CreateMirrorCreateClusterMirror |
| TBD | UNKNOWN_CLUSTER_MIRROR | The topic is not assigned to any cluster mirror | StopMirrorTopics, PauseMirrorTopics, ResumeMirrorTopics |
| TBD | TOPIC_ALREADY_IN_CLUSTER_MIRROR | The topic is already assigned to a cluster mirror | StartMirrorTopics |
| TBD | TOPIC_NOT_IN_CLUSTER_MIRROR | The topic does not belong to the specified cluster mirror | StopMirrorTopics, PauseMirrorTopics, ResumeMirrorTopics |
| TBD | MIRROR_TOPIC_ALREADY_PAUSED | The mirror topic is already paused | PauseMirrorTopics |
| TBD | MIRROR_TOPIC_NOT_PAUSED | The mirror topic is not paused | ResumeMirrorTopics |
| TBD | MIRROR_TOPIC_BEING_STOPPED | The mirror topic is being stopped | ResumeMirrorTopics |
| TBD | CLUSTER_MIRROR_NOT_EMPTY | The cluster mirror still has active or non-removed topics | DeleteMirrorDeleteClusterMirror |
| TBD | CLUSTER_MIRROR_AUTHORIZATION_FAILED | Mirror Cluster mirror authorization failed | CreateMirrorCreateClusterMirror, StartMirrorTopics, StopMirrorTopics, PauseMirrorTopics, ResumeMirrorTopics, DeleteMirrorDeleteClusterMirror |
Compatibility, Deprecation, and Migration Plan
...
- CLI Workflow: Create mirror with kafka-cluster-mirrors.sh, start mirroring, verify log convergence
- Basic Replication: Create mirror via API, start mirroring, verify log convergence
- Metadata Sync: Modify topic config in source, verify automatic sync to destination
- Partition Expansion: Add partitions to source topic, verify destination expands
- Consumer Groups: Commit offsets in source, verify replication to destination
- ACL Replication: Create ACL in source, verify creation in destination
- Leader Changes: Trigger leader election in source, verify fetcher reconnects
- Broker Failures: Stop destination broker, verify replication continues after recovery
...
- Log Convergence: Check log convergence after a series of leader elections in source and destination clusters.
- Failover Test: Simulate source cluster failure, measure consumer recovery time.
- Security Validation: Test all authentication mechanisms (SASL PLAIN, SCRAM, Kerberos, mTLS) via kafka-mirrors.sh config files.
- Migration Test: Test migration from older Kafka versions.
- Scalability Test: Replicate 1000 topics with 100,000 partitions across clusters.
- Long-Running Stability: Run continuous replication for 7 days, verify no memory leaks or performance degradation.
- Performance Benchmark: Measure replication throughput and latency across WAN.
...