Page History

...

• UNKNOWN: The partition has no cached state (broker just became leader, state not loaded yet). Not an explicit API-driven state, just the absence of state.
• PREPARING: The coordinator for this partition detects via onMetadataUpdate that it leads a mirror partition. It fetches last mirrored offsets from the source cluster and truncate logs to align the local log with the source. Valid from: null, UNKNOWN, STOPPED, FAILED.
• MIRRORING: All ISR members have completed truncation. A MirrorFetcherThread is started to continuously replicate records from the source cluster. Valid from: PREPARING only.
• STOPPING: Triggered by RemoveTopicsFromMirror API (user wants to fail over) or topic deletion on the source. The system records the last mirrored offset to the internal topic. Valid from: PREPARING, MIRRORING.
• STOPPED: Last mirrored offsets have been persisted. The topic becomes writable on the destination cluster (the mirror fetcher is removed and the read-only flag is cleared). Valid from: STOPPING only.
• FAILED: An error occurred. Can transition back to PREPARING to retry. Valid from: any state.
• (TODO: pause/unpauseresume)

Scenarios:

Starting a mirror (UNKNOWN -> PREPARING -> MIRRORING): The addTopicsToMirror command sets mirror.name config via the controller. The metadata update propagates to brokers. The broker leading the partition finds out the partition state via the coordinator, and this might trigger readMirrorState RPC to query from the remote coordinator and transitions to PREPARING if it's in a valid transition state (e.g. UNKNOWN). After truncation completes, it moves to MIRRORING and starts the mirror fetcher to fetch data from the source cluster.

...

$ bin/kafka-mirrors.sh --bootstrap-server :9094 --describe
MIRROR                         TOPIC                                    PARTITION  SOURCE-OFFSET   DESTINATION-OFFSET LAG      STATE       
my-mirror                      bar                                      0          2324            2324               0        MIRRORING   
my-mirror                      foo                                      0          69              66                 3        MIRRORING   
my-mirror                      foo                                      1          94              84                 10       MIRRORING   
my-mirror                      foo                                      2          94              90                 4        MIRRORING   
new-mirror                     baz                                      0          189             189                0        MIRRORING   
new-mirror                     baz                                      1          859             859                0        MIRRORING

Pause/resume mirroring for a specific topic or set of topics (topics reamin read-only):

TODO

Remove a specific topic or set of topics from a mirror (failover: all partitions become writable):

...