DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
- UNKNOWN: The partition has no cached state (broker just became leader, state not loaded yet). Not an explicit API-driven state, just the absence of state.
- PREPARING: The coordinator for this partition detects via onMetadataUpdate that it leads a mirror partition. It fetches last mirror epochs from the source cluster and truncates logs to align the local log with the source.
- MIRRORING: All ISR set members have completed truncation. A mirror fetcher thread is started to continuously replicate records from the source cluster.
- PAUSING: Triggered by the pause operation. The system removes fetchers for the affected partitions. Valid from: MIRRORING only.
- PAUSED: Fetchers have been removed. The partition stays read-only with no active fetchers and no metadata sync (configs, consumer groups, ACLs). On resume, transitions directly to MIRRORING.
- STOPPING: When a mirror partition enters the STOPPING state, the following operations execute sequentially.
- Remove mirror fetcher threads to stop cross-cluster replication.
- Bump leader epoch by sending a request to the controller. The destination's leader epoch must exceed the source cluster's last known epoch, fencing any stale producers.
- Append ABORT markers for all ongoing transactions.
- Update last mirror epochs in the __mirror_state topic, recording the latest leader epoch from each partition for future failback. Write The mirror fetcher is removed, the leader epoch is bumped, ABORT markers are appended for all ongoing transactions, last mirror epochs are persisted, and a MIRROR_PID_RESET barrier per partition, which clears all producer state entries from the ProducerStateManageris written.
- STOPPED: The topic becomes writable on the destination cluster. The mirror fetcher is removed and the read-only flag is cleared.
- FAILED: An error occurred. Valid from: any state. The operator that wants to restart a failed mirror partition can remove the topic from the mirror and add it back again. More restart topic mirroring. More sophisticated recovery strategies can be added later with a follow-up KIP.
...
In this example, source cluster log at the time of failure:
Offset | Type | PID | Content |
0 | DATA | 4001 | key=A, value=1 |
1 | DATA | 4001 | key=B, value=2 |
2 | DATA | 4002 | key=X, value=9 |
3 | COMMIT | 4001 | |
4 | DATA | 4003 | key=Y, value=5 |
5 | DATA | none | key=Z, value=10 |
Destination cluster log at failover (replication reached offset 2):
Offset | Type | PID | Content |
0 | DATA | 4001 | key=A, value=1 |
1 | DATA | 4001 | key=B, value=2 |
2 | DATA | 4002 | key=X, value=9 |
After the STOPPING transition appends abort markers:
Offset | Type | PID | Content |
0 | DATA | 4001 | key=A, value=1 |
1 | DATA | 4001 | key=B, value=2 |
2 | DATA | 4002 | key=X, value=9 |
3 | ABORT | 4001 | |
4 | ABORT | 4002 |
Transaction 4001 was committed at the source but aborted at the destination because the COMMIT marker (offset 3) had not yet been replicated. Transaction 4002 was correctly aborted at both clusters. Applications that require strict transactional guarantees across clusters should implement should implement deduplication or reconciliation logic after failover. Additionally, the kafka-transactions tool can only abort transactions originated from the local cluster. It cannot abort transactions replicated via mirroring because the __transaction_state topic is not mirrored. Hanging transactions from mirrored data are resolved exclusively by the STOPPING transition flow described above.
...