THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
We would keep leader.replication.throttled.replicas and follower.replication.throttled.replicas because there are still scenarios where generic replication throttling is needed. One such case is a bootstrapping broker where a lot of follower partitions need to catch up with their leader but they're using up the bandwidth from the leader partitions on that broker. In this case it is still useful to set follower replication throttling so the leaders can stay in-sync. (Furthermore as a possible improvement to bootstrapping it might be useful to allow only a subset of follower partitions to replicate at a time but this falls out of the scope of this KIP.)
New Configs
We will introduce two new configs in order to eventually replace *.replication.throttled.rate.
The new configs are:
| Config name | Type | Default | Valid values | Importance | Dynamic update mode |
|---|---|---|---|---|---|
| leader.reassignment.throttled.rate | Long | Long.MAX_VALUE | [1,...] | medium | per-broker |
| follower.reassignment.throttled.rate | Long | Long.MAX_VALUE | [1,...] | medium | per-broker |
...
Behavior-wise they'd throttle the addingReplicas of the LeaderAndIsrRequest during reassignment. Also leader.replication.throttled.replicas and replicas and follower.replication.throttled.replicas wouldn't be applied to these replicas as long as they're not in ISR. Once they enter it, they'd be allowed to have the replication bandwidth. would take precedence over these new reassignment configs to ensure backward compatibility.
To change these configs, the user must have ALTER_CONFIG privilege on the given cluster config as imposed by the incrementalAlterConfigs API which will be the medium for applying the configuration. By adding the new configs we'd also like to remove the related zookeeper dependencies in the kafka-reassign-partitions.sh command, so applying the quota would happen through the AdminClient API.
...
The only change which needs to be mentioned is the tooling change. With this we'll change the --throttle option's behavior. If for some reason the old behavior is needed it can be reproduced by calling kafka-configs.sh manually before and after the reassignment with the intended parameters.
The old configs (leader.replication.throttled.replicas and follower.replication.throttled.replicas) would take precedence over these new reassignment configs to ensure backward compatibility.