...

Previously, KIP-660: Pluggable ReplicaPlacer proposed making the ReplicaPlacer pluggable, but the proposal was rejected. The main concern was that an intelligent placement strategy might require a full view of the cluster, which could be prohibitively expensive in large deployments. Furthermore, such implementations could become overly complex and might introduce latency that impacts the controller thread, thus affecting system stability.

The issue of KAFKA-19507 shows a real-world need for solving custom replica assignment problems. It reinforces the point that different usage scenarios often come with unique assignment needs.

Nevertheless, we can make the ReplicaPlacer an internal pluggable component. This means the mechanism would not be exposed to external users, but could be utilized in internal testing, special-purpose scenarios, or controlled experiments. By allowing different implementations to be injected internally, we can make the placement logic more flexible for testing purposes, without compromising the stability of the controller.

...

• Class: ReplicationConfigs
• Name: internal.replica.placer.class.name
• Type: string
• Default: StripedReplicaPlacer
• Valid Values: non-empty string
• Doc: The fully qualified class name of the ReplicaPlacer implementation. This class is used by the broker to determine the placement of replicas when topics or partitions are created.

...

    public static final String INTERNAL_REPLICA_PLACER_CLASS_NAME = "internal.replica.placer.class.name";
    public static final String INTERNAL_REPLICA_PLACER_CLASS_NAME_DEFAULT = StripedReplicaPlacer.class.getName();
    public static final String INTERNAL_REPLICA_PLACER_CLASS_NAME_DOC = "The fully qualified class name of the ReplicaPlacer implementation. This class is used by the broker to determine the placement of replicas when topics or partitions are created.";

...

Add new test for new internal config internal.replica.placer.class.name

Rejected Alternatives

...

KIP-660: Pluggable ReplicaPlacer, but the proposal was ultimately rejected due to the following concerns:

• Implementing an intelligent placement strategy may require access to the full cluster state, which could be prohibitively expensive in large deployments.

• Such strategies could become overly complex and harder to maintain.

• Slow or inefficient placement logic could block the controller thread, negatively impacting Kafka's overall stability.