...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-595 introduced KRaft topic partitions. These are partitions with replicas that can achive consensus on the Kafka log without relying on the Controller or ZK. The KRaft Controllers in KIP-631 use one of this topic partiton (called cluster metadata topic partition) to order operations on the cluster, commit them and replicate them to other controllers and brokers.

Consensus on the cluster metadata partition was achieved by the voters (Controllers). If the operator of a KRaft cluster wanted to make changes to the set of voters, they would have to  shutdown all of the controllers nodes and manually make changes to the on disk state of the old controllers and new controllers. If the operator wanted to replace an existing voter because of a disk failure or general hardward failure, they would have to make sure that the new voter node has a superset of the the previous voter state. Both of these solutions are manual and error prone.

This KIP describes a protocol for extending KIP-595 and KIP-630 so that the operator can programatically update the voter set in a way that is safe and is available. There are two important use cases that this KIP supports. One is that the operator wants to change the number of controllers by adding or removing a controller.  The other is that the operation wants to replace a controller because of a disk or hardware failureDescribe the problems you are trying to solve.

Proposed Changes

Replica UUID

...