Page History

...

KIP-73 introduced a method of throttling replication traffic between brokers such that a predictable level of service can be maintained during data intensive administrative operations (e.g. rebalancing partitions). This method works by enabling replication throttling on partition-replicas via dynamic or static configuration at the topic level (leader.replication.throttled.replicas & follower.replication.throttled.replicas), and setting a throttle rate via dynamic configuration at the broker level (leader.replication.throttled.rate & follower.replication.throttled.rate). This works very well for planned operations, however, is not conducive with unplanned events that trigger a spike in replication traffic between brokers, for example a broker automatically rejoining the cluster after an extended period of downtime. This KIP intends to address this case by allowing the throttle rate variables (leader.replication.throttled.rate & follower.replication.throttled.rate) to be configured both dynamically and statically. With this improvement, an administrator can preemptively set a maximum throttling rate to ensure that, in the event of a spike in replication traffic, there is still some bandwidth reserved for client traffic. In this case, the administrator would enable throttling on all partition-replicas using the wildcard '*' - alternatively, if KIP-1009 accepted, they could enable throttling at the broker level.

...