Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The general goal of this KIP is to allow the user to dynamically change the set of voters (also known as controllers) for the KRaft cluster metadata partition. This is achieved by storing the set of voters and their known endpoints in the log instead of the controller.quorum.voters properties. Because the set of voters is stored in the log it allows the leader to replicate this information to all of the fetching replicas (voters and observers). Since old log segments (records) can be deleted once a snapshot has been created, the KRaft snapshots will also contain the set of voters up to the included offset.

Following The fetching (following) voters will discover the set of voters and their endpoints by fetching the latest log from the leader but how . How do new voters discovery the leader's endpoint? The leader will push this information to new voters (or voters that were offline for a long time) using the BeginQuorumEpoch request. The active leader sends BeginQuorumEpoch to all of the voters in the voter set when it becomes leader for an epoch.

The leader doesn't send BeginQuorumEpoch to observers since this are dynamic and are not included in the KRaft partition log. Observer will instead discover the leader using the controller.quorum.bootstrap.servers. It is important that this property includes at least one of the available voters, else brokers (observers) will not be able to discover the leader of the KRaft cluster metadata partition.

To make changes to the voter set safe it is required that the majority the competing voter sets commit the voter changes. In this design the competing voter sets are the current voter set and new voter set. Since this design only allows one voter change at a time the majority of the new configuration always overlaps (intercepts) the majority of the old configuration. This done by the leader committing the current epoch when it becomes leader and committing single voter changes with the new voter set before accepting another voter change.

The rest of these section and subsection goes into the detail changes required to implement this new functionality. 

...