...

While electing a leader, if none of the in-sync replicas is alive, the controller elects a replica that was not a part of the in-sync replica set. Such a leader is called an unclean leader. Since this replica was not a part of the last known in-sync replica set, it is associated with data loss in a partition.
Currently, there is no way for brokers to tell whether a given leader was an unclean leader at the time of its election. This information can be useful at the broker in order to invoke the appropriate data loss handling routine. One such scenario can be handling of

LeaderAndIsr

data loss in a partition that is part of a transaction.

Another scenario is when a partition holds some kind of metadata and any data loss in this partition further requires an automated or manual intervention.

Changed Interfaces

...

• LeaderAndIsr data to be saved at zoo keeper will add a boolean flag to indicate whether the current leader is unclean or not.

case class LeaderAndIsr(leader: Int,
                        leaderEpoch: Int,
                        isr: List[Int],
                        zkVersion: Int,
                        isUnclean: Boolean)

• This boolean information will be sent to the brokers in the LeaderAndIsrRequest under LeaderAndIsrPartitionState commonStructs array, like so:

{ "name": "IsUncleanLeader", "type": "bool", "versions": "4+", "default": "false", "tag": 100020,
        "taggedVersions": "4+", "ignorable": true, "about": "Whether the elected leader is unclean." },

...

• Leader replica will use the AlterISR RPC to set the IsUncleanLeader state to false once it has completed the desired recovery.

Proposed Changes

This KIP proposes to append a boolean state to the LeaderAndIsr state maintained at zookeeper. The new boolean state will be called IsUncleanLeader. When set to true, it will signify that the current leader was elected as an unclean leader, false otherwise. The flag can be maintained according to the following rules:

...