Page History

...

This KIP proposes changing the ISR expansion logic on the leader and and the ISR validation logic on the controller to avoid bringing back fenced or shutting down replicas in the ISR. The leader will consider only active (not fenced nor in controlled shutdown) replicas to be eligible to join the ISR. It will rely on the metadata cache to get this information via the metadata log. The controlled shutdown state is not persisted to the metadata log at the moment. The information is only available as a soft state in the BrokerHeartbeatManager. This means that the information is not available to the leaders and not recovered when the controller fails over. This KIP proposes to persist it for the leader to leverage it.

As the metadata cache is eventually consistent, the leader might try to add a replica - which was just removed by the controller - back to the ISR because it does not know that the replica's latest state yet. In order to avoid this, the controller will validate the new ISR and reject any AlterPartition request containing an ineligible replica with the newly introduced INELIGIBLE_REPLICA error code. The leader will consider only active (not fenced nor in controlled shutdown) replicas to be eligible. When the leader receives an INELIGIBLE_REPLICA error code, it is expected to revert back its state to the last committed state - assuming that the state did not change in the mean time. When a broker is unfenced by the controller, the leader does nothing because subsequent fetch requests from the followers will try to get them back into the ISR if they are caught-up.

...