Page History

...

One possibility that we've discussed is whether we could have controller IDs and broker IDs share the same ID space.  For example, could there be both a broker 1 and a controller 1?  This is not a good idea because NetworkClient assumes a single ID space.  So if there is both a controller 1 and a broker 1, we don't have a way of picking the "right" one.  We would have to add a concept of node types.  That is a fair amount of extra work.  It also is a bit conceptually messy for the network layer to understand node types, rather than just treating them all as endpoints.

Another reason not to share node IDs is that it would make migrating from a combined broker+controller node to separate nodes more difficult.  For example, let's say you have three combined nodes but the controller load is getting too high and you now want a separate controller node.  If you also have to change the controller ID, the migration becomes more difficult.  Since controller node IDs make their way into the Raft log itself, changing the ID later on is non-trivial.

External metrics and management systems often make use of the concept of node ID.  For example, someone may want to aggregate all the socket server metrics from node 1.  But if there are actually two socket server sharing this same node ID (controller and broker) this may create problems and require changes to the external system.

A related question is whether a broker and a controller could share the same port, if they are co-located in the same JVM.  We discussed this as part of KIP-590 and agreed that we would not share ports.  A shared port would make it impossible to have separate RPC handlers, complicating the code.  It would also make it impossible to have separate, stricter authentication for controllers.  Sharing a port with the broker opens us up to denial of service attacks, including unintentional ones.