Page History

...

When creating topics or partitions, the Kafka controller has to pick brokers to host the new partitions. The current assignment logic is based on a round robin algorithm and supports rack awareness. While this works relatively well in many scenarios as it's not aware of the state of the clusters, in a few cases the assignments it generates is not optimal. Many cluster administrators then rely on tools like Cruise Control to move partitions to better brokers but this . This proocess is expensive as often data has to be copied between brokers.

It would be great to allow custom logic for the assignor to hopefully better understand the state of the cluster and minimize the number of partition reassignments necessary. It would enable administrators to build assignment goals (similar to Cruise Control goals) for their clusters.

...

2) New broker configuration:

Name: replica.assignor.class.name

Type: class

Doc: The fully qualified class name that implements ReplicaAssignor. This is used by the broker to determine replicas when topics or partiions are created. This default defaults to DefaultReplicaAssignor.

...

The existing assignment logic will be extracted into a class, DefaultReplicaAssignment, that implements the ReplicaAssignor interface. It will stay the default implementation and a private class.

AdminManager will create an instance of the specified ReplicaAssignor implementation or if none were set of DefaultReplicaAssignor. When creating topics or partitions, for each topic, it will call assignReplicasToBrokers(). If multiple topics are present in the request, AdminManager will update the Cluster object so the ReplicaAssignor class has access to the up to date cluster metadata.

...