...

1. UnregisterControllerRequest is sent to the active controller
2. The active controller writes an UnregisterControllerRecord to the metadata log
3. When this record is committed, return a response to the user for unregistering the controller
4. The active controller's state machine removes the registration for controller 9990, meaning feature upgrades no longer consider node 9990's supported features
5. The registration from controller 9990 is removed from the metadata image

kafka-cluster unregister-controller  is a command for users when they want to unregister a controller from the cluster. This command should only be run after a controller is stopped, and the operator does not intend to bring it back. kafka-cluster unregister-controller  works irrespective of the quorum mode. 

This command supports unregistering any controller registration in any quorum mode, such as:

• Any voter in a dynamic quorum
• Any observer in a dynamic quorum
• Any stale "voter" in a static quorum as a result of a misconfiguration of controller.quorum.voters 

In any of these cases, it is expected that the controller process has stopped before the operator unregisters it via these tools. If the controller process has not stopped, it will attempt to re-register with the cluster.

kafka-metadata-quorum 

Add the --unregister flag to the kafka-metadata-quorum remove-controller command. When this flag is set, invoking this command with --unregister set will remove the controller as a KRaft voter and unregister it. Below is an example invocation.

...

When the user executes this command, kafka tries to remove controller 9990 as a voter AND unregister it:

1. RemoveRaftVoterRequest  is sent to the active controller
2. The KRaft leader writes a VotersRecord without voter 9990 to the metadata log
3. When this record is committed, return a response to the user for removing the voter
4. UnregisterControllerRequest is sent to the active controller if steps 1-3 were successful
   1. If steps 1-3 were not successful, return the error and direct the user to use kafka-cluster unregister-controller instead.
5. The active controller writes an UnregisterControllerRecord to the metadata log
6. When this record is committed, return a response to the user for unregistering the controller
7. The active controller's state machine removes the registration for controller 9990, meaning feature upgrades no longer consider node 9990's supported features
8. The registration from controller 9990 is removed from the metadata image

The main use case for this command is to remove a node from the voter set in a dynamic KRaft quorum, AND unregister it from the cluster all within the same CLI command. Since this is a common usage pattern, running this command with --unregister can be thought of as a "built-in" script that provides a smooth UX for decommissioning voters in a dynamic quorum. Running the command with --unregister will still fail when the cluster does not support dynamic quorum to be consistent with the behavior of the command when --unregister is not set.

Proposed Changes

Controller Changes

...

This new flag means that the remove-controller command could send two RPCs to the active controller, one to remove the node from the KRaft voter set, and another to remove the node's registration. The implementation of this command should be able to handle cases where:

...

To maintain consistency with this command only being supported with dynamic quorum, running remove-controller --unregister will fail if the cluster does not support dynamic quorum reconfiguration. Instead, the user should be directed to use the kafka-cluster unregister-controller  command.

Compatibility, Deprecation, and Migration Plan

...