Page History

...

A broker only appears in MetadataResponse if it is in the "registered and active" state.  If it is in the unreigstered state, or the "registered and fenced" state, it will not appear in MetadataResponse.

Essentially, registration relates to the permanent state of the cluster: the number of brokers we expect to have, what racks we expect them to be in, and so on.  Fencing refers to the transitory state of the cluster: which brokers are currently available.

By separating the permanent state from the transitory state, we can more effectively handle transitory issues.  For example, if you have a 3 node cluster that is undergoing rolling upgrade, one of the nodes might be down because it is rolling.  However, we should still allow users to create new topics with replication factor 3. Currently, that is not possible, because the node's information gets wiped the moment its ZK registration goes away.  With KIP-631, the registration remains, although the node becomes fenced.  Another example is doing reassignment on a cluster where one or more nodes is down.  Currently, when a node is down, all of its ZK registration information is gone.  But  we need this information in order to understand things like whether the replicas of a particular partition are well-balanced across racks.

Broker Registration

Every distributed system needs a way of managing cluster membership.  Prior to KIP-500, Kafka brokers registered ephemeral znodes in order to register themselves as part of the cluster.  The Kafka controller passively consumed the registration information from Zookeeper.

...

Broker Leases

Once a broker is registeredhas completed log recovery and registered with the controller, its next goal is to transition from registered+fenced to registered+active.  It can do this by sending a broker heartbeat request to the active controller .  When it receives a the heartbeat request, the active controller must decide whether to give the broker a lease, or whether it should deny the lease.  Brokers that don't have a broker ID lease are said to be "fenced."with ShouldFence = false.

Once the broker sets ShouldFence = false, it becomes eligible for unfencing.  However, the controller will not actually unfence it unless its metadata is also reasonably current.  It will determine this by checking the metadata offset in the heartbeat request.

As mentioned earlier, brokers which are fenced will not appear in MetadataResponses.  So clients that have up-to-date metadata will not try to contact fenced brokers.

...