DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
New methods are added to the Admin interface for programmatic cluster mirror management, along with their supporting classes:
| Code Block | ||
|---|---|---|
| ||
CreateMirrorResult createMirror(String mirrorName, Map<String, String> configs, CreateMirrorOptions options); AddTopicsToMirrorResult addTopicsToMirror(Map<String, String> topicToMirrorName, AddTopicsToMirrorOptions options); RemoveTopicsFromMirrorResult removeTopicsFromMirror(String mirrorName, Set<String> topics, RemoveTopicsFromMirrorOptions options); ListMirrorsResult listMirrors(ListMirrorsOptions options); /** * Create a new cluster mirror. * * @param mirrorName The name of the cluster mirror * @param configs Configuration for the cluster mirror, including bootstrap servers and security settings * @param options Options for the create mirror operation * @return The CreateMirrorResult */ CreateMirrorResult createMirror(String mirrorName, Map<String, String> configs, CreateMirrorOptions options); /** * Add topics to an existing cluster mirror for cross-cluster replication. * * When topics are added to a mirror, they become read-only on the destination cluster and start * replicating data from the source cluster. This operation marks the specified topics with the * mirror name, preventing local writes and enabling the MirrorFetcherThread to begin replication. * * @param topicToMirrorName Map of topic names to mirror names, allowing multiple topics to be * added to potentially different mirrors in a single operation * @param options Options for the add topics to mirror operation * @return The AddTopicsToMirrorResult containing futures for each topic addition */ AddTopicsToMirrorResult addTopicsToMirror(Map<String, String> topicToMirrorName, AddTopicsToMirrorOptions options); /** * Remove topics from cluster mirror, making them writable on the destination cluster. * * This operation is typically used during failover scenarios when the destination cluster needs to * be promoted from passive (read-only mirror) to active (accepting writes). Removing topics from * the mirror clears the mirrorName field from partition metadata, which allows producers to write * to these partitions. * * @param topics Set of topic names to remove from mirroring * @param options Options for the remove topics from mirror operation * @return The RemoveTopicsFromMirrorResult containing futures for each topic removal */ RemoveTopicsFromMirrorResult removeTopicsFromMirror(Set<String> topics, RemoveTopicsFromMirrorOptions options); /** * Pause mirroring for the specified topics. * * Paused topics remain read-only on the destination cluster but stop fetching new data from the * source cluster. The mirror fetcher threads are removed for these partitions, preserving the * current replicated state. Mirroring can be resumed later with {@link #resumeMirrorTopics}. * * @param topics Set of topic names to pause mirroring for * @param options Options for the pause mirror topics operation * @return The PauseMirrorTopicsResult containing futures for each topic */ PauseMirrorTopicsResult pauseMirrorTopics(Set<String> topics, PauseMirrorTopicsOptions options); /** * Resume mirroring for previously paused topics. * * Resumed topics restart fetching data from the source cluster, picking up from where they * left off. New mirror fetcher threads are created and the partitions transition back to the * MIRRORING state. * * @param topics Set of topic names to resume mirroring for * @param options Options for the resume mirror topics operation * @return The ResumeMirrorTopicsResult containing futures for each topic */ ResumeMirrorTopicsResult resumeMirrorTopics(Set<String> topics, ResumeMirrorTopicsOptions options); /** * List the cluster mirrors available in the cluster. * * @param options The options to use when listing the mirrors. * @return The ListMirrorsResult. */ ListMirrorsResult listMirrors(ListMirrorsOptions options); /** * Describe cluster mirrors. * * This operation retrieves detailed information about cluster mirrors including: * - Topics being mirrored * - Partition-level lag information (source offset vs destination offset) * - Mirroring state for each partition (INITIALIZING, PREPARING, MIRRORING, etc.) * * @param mirrorNames The names of the mirrors to describe * @param options The options to use when describing mirrors * @return The DescribeMirrorsResult */ DescribeMirrorsResult describeMirrors(Collection<String> mirrorNames, DescribeMirrorsOptions options); |
Protocol Changes
This KIP extends CreateTopic API, but also introduces some new APIs and metadata records.
...
Set via CreateMirror or IncrementalAlterConfigs. Stored in cluster metadata records.
Key | Description | Default |
|---|---|---|
bootstrap.servers | A list of host/port pairs to use for establishing the initial connection to the source cluster. | |
mirror.topic.properties.exclude | A comma-separated list of topic config property names to exclude from synchronization. Properties in this list will not be replicated from the source cluster. The mirror.name property is always excluded regardless of this setting. | follower.replication.throttled.replicas, leader.replication.throttled.replicas, message.timestamp.difference.max.ms, log.message.timestamp.before.max.ms, log.message.timestamp.after.max.ms, message.timestamp.type, unclean.leader.election.enable, min.insync.replicas, mirror.name |
mirror.groups.include | A comma-separated list of regex patterns for consumer group IDs to include in offset synchronization. Only consumer groups whose IDs match at least one of the patterns will have their offsets replicated from the source cluster. | .* |
mirror.acl.include | A comma-separated list of ACL include rules. Each rule uses semicolon-separated fields: resourceType;resourceName;operation;permissionType;principal. Use '*' as wildcard for any field. The resourceName field supports regex patterns. Trailing wildcard fields can be omitted. See AclRule javadoc for examples. | * |
security.protocol | Protocol for source cluster communication (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL). | |
sasl.* | SASL configuration properties. | |
ssl.* | SSL configuration properties. |
Topic Configuration
Set by topic creation or alter. Stored in topic config.
...