THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Current state: Under Discussion
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
| Config | Use case | |||
|---|---|---|---|---|
| fetch.max.bytes | Maximum bytes expected for the entire fetch response. | |||
| failed.authentication.delay | Connection close delay on failed authentication. | |||
| replica.fetch.response.max.bytes | Maximum bytes expected for the entire fetch response. | |||
| replica.fetch.wait.max.ms | Max wait time for each fetcher request issued by follower replicas. | |||
| followerreplica.fetch.replication.throttled (new config to add) | Should all replication follower throttle | leader.replication.throttled (new config to add) | backoff.ms | The amount of time to sleep when fetch partition error occurs. |
| replica.fetch.response.max.bytes | Maximum bytes expected for the entire fetch response. Should all replication leader throttle |
fetch.max.bytes: ReplicaManager will multi-inherit from BrokerReconfigurable but no reconfigurable interfaces need to be implemented since the fetcher request will calculate fetchMaxBytes as the minimum of fetchRequest.maxBytes and config.fetchMaxBytes
...
replica.fetch.wait.max.ms: The changes are similar to replica.fetch.response.max.bytes
follower.replication.throttled: A new class variable LeaderReplicationThrottledProp will be added to ReplicationQuotaManager. Similar to leader.replication.throttled.rate, the BrokerConfigHandler will call updateBrokerThrottle() and update this class variable upon receiving the config change notification from ZooKeeper. ReplicationQuotaManager::isThrottled() will have its logic modified and judge based on if all the replication follower is throttled.
leader.replication.throttled: Similar to follower.replication.throttled
Public Interfaces
This KIP introduces two new broker level configs related to replication throttle:
follower.replication.throttled - Should all replication follower throttle
leader.replication.throttled - Should all replication leader throttle
Currently, we have two similar topic level configs:
leader.replication.throttled.replicas: A list of replicas for which log replication should be throttled on the leader side. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic.
follower.replication.throttled.replicas: A list of replicas for which log replication should be throttled on the follower side. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic.
...
Compatibility, Deprecation, and Migration Plan
- There won't be any impact on existing users.
- There won't be any change of in current behavior.
- No migration tool required
To discuss
...