THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Config | Use case |
|---|---|
| fetch.max.bytes | Maximum bytes expected for the entire fetch response. |
| failed.authentication.delay | Connection close delay on failed authentication. |
| replica.fetch.response.max.bytes | Maximum bytes expected for the entire fetch response. |
| replica.fetch.wait.max.ms | Max wait time for each fetcher request issued by follower replicas. |
| follower.replication.throttled (new config to add) | Should all replication follower throttle |
| leader.replication.throttled (new config to add) | Should all replication leader throttle |
fetch.max.bytes: ReplicaManager will multi-inherit from BrokerReconfigurable but no reconfigurable interfaces need to be implemented since the fetcher request will calculate fetchMaxBytes as the minimum of fetchRequest.maxBytes and config.fetchMaxBytes
failed.authentication.delay: SocketServer already implemented the Reconfigurable interface. Setter method will be added in both Processor and Selector and update the Selector::failedAuthenticationDelayMsProp variable in the Reconfigure callback.
replica.fetch.response.max.bytes: The two class ReplicaAlterLogDirsThread and ReplicaFetcherThread read this config as their class variable when they initialize. So they will multi-inherit from BrokerReconfigurable and implemented setter method to modify the class variable in the Reconfigure callback.
replica.fetch.wait.max.ms: The changes are similar to replica.fetch.response.max.bytes
follower.replication.throttled: A new class variable LeaderReplicationThrottledProp will be added to ReplicationQuotaManager. Similar to leader.replication.throttled.rate, the BrokerConfigHandler will call updateBrokerThrottle() and update this class variable upon receiving the config change notification from ZooKeeper. ReplicationQuotaManager::isThrottled() will have its logic modified and judge based on if all the replication follower is throttled.
leader.replication.throttled: Similar to follower.replication.throttled
Public Interfaces
This KIP introduces two new broker level configs related to replication throttle:
- follower.replication.throttled - Should all replication follower throttle
- leader.replication.throttled - Should all replication leader throttle
...
- There won't be any impact on existing users.
- There won't be any change of current behavior.
- No migration tool required
To discuss
- What should be the type / format of follower.replication.throttled and leader.replication.throttled.replicas, a boolean or a list such as "[partitionId]:[brokerId], [partitionId]:[brokerId]"?
- For the two new broker level configurations follower.replication.throttled and leader.replication.throttled.replicas, we should think about their precedence compared to the existing topic level configs leader.replication.throttled.replicas and leader.replication.throttled.replicas. When the broker level throttle and topic level throttle both are both set, which will dominate?