DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
| Code Block | ||
|---|---|---|
| ||
/**
* Create a new cluster mirror.
*
* @param mirrorName The name of the cluster mirror
* @param configs Configuration for the cluster mirror, including bootstrap servers and security settings
* @param options Options for the create mirror operation
* @return The CreateMirrorResult
*/
CreateMirrorResult createMirror(String mirrorName, Map<String, String> configs, CreateMirrorOptions options);
/**
* Add topics to an existing cluster mirror for cross-cluster replication.
*
* When topics are added to a mirror, they become read-only on the destination cluster and start
* replicating data from the source cluster. This operation marks the specified topics with the
* mirror name, preventing local writes and enabling the MirrorFetcherThread to begin replication.
*
* @param mirrorName The mirror name to add the topics to
* @param topics Set of topic names to add to mirroring
* @param options Options for the add topics to mirror operation
* @return The AddTopicsToMirrorResult containing futures for each topic addition
*/
AddTopicsToMirrorResult addTopicsToMirror(String mirrorName, Set<String> topics, AddTopicsToMirrorOptions options);
/**
* Remove topics from cluster mirror, making them writable on the destination cluster.
*
* This operation is typically used during failover scenarios when the destination cluster needs to
* be promoted from passive (read-only mirror) to active (accepting writes). Removing topics from
* the mirror clears the mirrorName field from partition metadata, which allows producers to write
* to these partitions.
*
* @param mirrorName The mirror name to remove the topics from
* @param topics Set of topic names to remove from mirroring
* @param options Options for the remove topics from mirror operation
* @return The RemoveTopicsFromMirrorResult containing futures for each topic removal
*/
RemoveTopicsFromMirrorResult removeTopicsFromMirror(String mirrorName, Set<String> topics, RemoveTopicsFromMirrorOptions options);
/**
* Pause mirroring for the specified topics.
*
* Paused topics remain read-only on the destination cluster but stop fetching new data from the
* source cluster. The mirror fetcher threads are removed for these partitions, preserving the
* current replicated state. Mirroring can be resumed later with {@link #resumeMirrorTopics}.
*
* @param mirrorName The mirror name to pause the topics for
* @param topics Set of topic names to pause mirroring for
* @param options Options for the pause mirror topics operation
* @return The PauseMirrorTopicsResult containing futures for each topic
*/
PauseMirrorTopicsResult pauseMirrorTopics(String mirrorName, Set<String> topics, PauseMirrorTopicsOptions options);
/**
* Resume mirroring for previously paused topics.
*
* Resumed topics restart fetching data from the source cluster, picking up from where they
* left off. New mirror fetcher threads are created and the partitions transition back to the
* MIRRORING state.
*
* @param mirrorName The mirror name to resume the topics for
* @param topics Set of topic names to resume mirroring for
* @param options Options for the resume mirror topics operation
* @return The ResumeMirrorTopicsResult containing futures for each topic
*/
ResumeMirrorTopicsResult resumeMirrorTopics(String mirrorName, Set<String> topics, ResumeMirrorTopicsOptions options);
/**
* Delete a cluster mirror including its configuration.
*
* The mirror must be empty (no topics) or all its topics must have been removed (in STOPPED
* state). After deletion, all mirror metadata are tombstoned and failback is no longer possible.
*
* @param mirrorName The name of the cluster mirror to delete
* @param options Options for the delete mirror operation
* @return The DeleteMirrorResult
*/
DeleteMirrorResult deleteMirror(String mirrorName, DeleteMirrorOptions options);
/**
* List the cluster mirrors available in the cluster.
*
* @param options The options to use when listing the mirrors.
* @return The ListMirrorsResult.
*/
ListMirrorsResult listMirrors(ListMirrorsOptions options);
/**
* Describe cluster mirrors.
*
* This operation retrieves detailed information about cluster mirrors including:
* - Topics being mirrored
* - Partition-level lag information (source offset vs destination offset)
* - Mirroring state for each partition (INITIALIZING, PREPARING, MIRRORING, etc.)
*
* @param mirrorNames The names of the mirrors to describe
* @param options The options to use when describing mirrors
* @return The DescribeMirrorsResult
*/
DescribeMirrorsResult describeMirrors(Collection<String> mirrorNames, DescribeMirrorsOptions options); |
Protocol Changes
This KIP extends CreateTopic API, but also introduces some new APIs and metadata records.
CreateTopic
The CreateTopic API is extended to add information required for mirror topic creation.
section describes all protocol level changes and new RPCs.
EntityType
A new entity type is added for the message generator to provide schema-level type validation for mirror name fields:
| Code Block |
|---|
public enum EntityType {
// ... existing types ...
@JsonProperty("mirrorName")
MIRROR_NAME(FieldType.StringFieldType.INSTANCE);
} |
ResourceType
A new resource type is added to the ResourceType enum to enable per-mirror authorization:
| Code Block |
|---|
public enum ResourceType {
// ... existing types ...
/**
* A cluster mirror.
*/
CLUSTER_MIRROR((byte) 8); |
CoordinatorType
The FindCoordinatorRequest object is extended to support a new coordinator type:
| Code Block | ||
|---|---|---|
| ||
public enum CoordinatorType {
// ... existing types ...
MIRROR((byte) 3);
} |
CreateTopic
The CreateTopic API is extended to add information required for mirror topic creation.
| Code Block |
|---|
| Code Block |
// new added { "name": "MirrorInfo", "type": "MirrorInfo", "versions": "8+", "nullableVersions": "8+", "ignorable": true, "about": "Mirror information for creating a mirror topic from a source cluster.", "fields": [ { "name": "TopicIdMirrorInfo", "type": "uuidMirrorInfo", "versions": "8+", "nullableVersions": "8+", "ignorable": true, "about": "TheMirror information for creating a mirror topic ID from thea source cluster.", "fields": } ]}[ { "name": "TopicId", "type": "uuid", "versions": "8+", "about": "The topic ID from the source cluster." } ]} |
The topic ID field ensures The topic ID field ensures mirror topics retain the same topic ID as the source cluster topic. This allows fetch requests to pass validation on the source broker, and enables the system to verify that a topic being mirrored to a same-named topic in the destination cluster is indeed the same logical topic, not a name collision.
In normal topic creation, the MirrorInfo field will be null. When receiving the CreateTopic request, the controller will check the new field. If it is not set, the topic ID will be generated with random UUID as usual. Otherwise, the controller will do the following validation:
- This topic ID is not used by other topics in the current cluster
- The replicas for the partition assignment are all active and not in fenced or controlled shutdown. This is to make sure when a topic gets deleted and re-created with the same topic ID, the stale offline log dir won’t be treated as the active log dir after it becomes online (KAFKA-16234).
EntityType
A new entity type is added for the message generator to provide schema-level type validation for mirror name fields:
| Code Block |
|---|
public enum EntityType {
// ... existing types ...
@JsonProperty("mirrorName")
MIRROR_NAME(FieldType.StringFieldType.INSTANCE); // New type
} |
ResourceType
A new resource type is added to the ResourceType enum to enable per-mirror authorization:
...
:
- This topic ID is not used by other topics in the current cluster
- The replicas for the partition assignment are all active and not in fenced or controlled shutdown. This is to make sure when a topic gets deleted and re-created with the same topic ID, the stale offline log dir won’t be treated as the active log dir after it becomes online (KAFKA-16234).
CreateMirror
Allows users to create a mirror and supply its configuration. When the broker receives the request, it validates that the mirror name is not already in use, contains only permitted characters, and does not end with ".removed" or ".paused" suffix. Once validated, the request is forwarded to the controller, which persists the configuration in the metadata log.
...
The FindCoordinatorRequest object is extended to support a new coordinator type:
...
| language | java |
|---|
...
Mirror Metadata Records
LastMirroredOffsets
...