Page History

...

This KIP provides a simple solution to an availability issue exposed by the auto-join feature proposed in KIP-853 and how AddRaftVoterRequest  RPC is currently handled. The problem is because the active controller does not send a response to complete the AddRaftVoterRequest until after the new voter set is committed, and that KRaft (and Kafka in general) only support one in-flight request to a node. Consider the following scenario:

Some controller A that is automatically joining by sending the AddRaftVoterRequest  RPC is the same controller who's whose Fetch is needed to commit the new voter set. A clear example of this is when bootstrapping with --standalone and having controllers auto-join, as the first controller to auto-join will increase the voter set size from 1 to 2. The active controller needs controller A to complete a Fetch  RPC to complete the AddRaftVoterRequest  RPC. However, this case also applies when going from a voter set of size X to size X + 1, where a minority of X + 1 nodes from the new voter set are unavailable.

...

The proposed change is to update the AddRaftVoterRequest  RPC with a boolean flag that tells the active controller whether the request was sent as a part of auto-join (i.e. sent from another controller), or not (e.g. sent via the AdminClient)when to send a response for the RPC: after the new voter set is committed, or after the new voter set is written locally.

When the request is coming from another controller as a part of auto-join, the active controller will send a response after it appends the new voter set to its local log, rather than after that voter set is committed. This allows the "joining" replica to actually fetch the new voter set in time.

This change is sufficient because the main motivation behind not completing the RPC until the new voter set was committed was for an intuitive UX for the operator. The inactive controllers that send AddRaftVoterRequest  as a part of auto-join do not care if the new voter set was committed, since they will retry the request on a timer until it completes successfully.

...