Page History

...

Metadata snapshots will need to include the the latest Producer ID block that was committed to the metadata log. Since we only need the latest block, the impact on the size of the snapshots is trivial.

Broker

The broker code becomes quite simple with this change. Rather than implementing the logic to allocate the next block of PIDs and dealing with semantics of ZooKeeper, the broker will simply will now defer to the controller . The broker will perform a request to the controller to fetch the next block of IDsfor allocating blocks of PIDs. This is done by sending an AllocateProducerIdBlockRequest to the controller using an inter-broker channel. If the request fails, the broker will retry for certain transient errors. The broker should use the ID block returned by the RPC (as opposed to waiting to replicate the resulting ProducerIdBlockRecord metadata record).

Since there is now a Kafka network request involved in the ID block generation, we should consider pre-fetching blocks so a client is never waiting on an InitProducerIdRequest for too long. We will likely share an existing broker-controller channel for this RPC, so we cannot guarantee low or consistent latencies.

...