THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Current state: Under Discussion
Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
| Config | Use case | |||
|---|---|---|---|---|
| fetch.max.bytes | Maximum bytes expected for the entire fetch response. | |||
| failed.authentication.delay | Connection close delay on failed authentication. | |||
| replica.fetch.response.max.bytes | Maximum bytes expected for the entire fetch response. | |||
| replica.fetch.wait.max.ms | Max wait time for each fetcher request issued by follower replicas. | |||
| followerreplica.fetch.replication.throttled (new config to add) | Should all replication follower throttle | leader.replication.throttled (new config to add) | backoff.ms | The amount of time to sleep when fetch partition error occurs. |
| replica.fetch.response.max.bytes | Maximum bytes expected for the entire fetch response. Should all replication leader throttle |
fetch.max.bytes: ReplicaManager will multi-inherit from BrokerReconfigurable but no reconfigurable interfaces need to be implemented since the fetcher request will calculate fetchMaxBytes as the minimum of fetchRequest.maxBytes and config.fetchMaxBytes
failed.authentication.delay: SocketServer already implemented the Reconfigurable interface. Setter method will be added in both Processor and Selector and update the Selector::failedAuthenticationDelayMsProp variable in the Reconfigure callback.
replica.fetch.response.max.bytes: The two class ReplicaAlterLogDirsThread and ReplicaFetcherThread read this config as their class variable when they initialize. So they will multi-inherit from BrokerReconfigurable and implemented setter method to modify the class variable in the Reconfigure callback.
Public Interfaces
This KIP introduces two new broker level configs related to replication throttle:
- follower.replication.throttled - Should all replication follower throttle
- leader.replication.throttled - Should all replication leader throttle
Currently, we have two similar topic level configs:
leader.replication.throttled.replicas: A list of replicas for which log replication should be throttled on the leader side. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic.
follower.replication.throttled.replicas: A list of replicas for which log replication should be throttled on the follower side. The list should describe a set of replicas in the form [PartitionId]:[BrokerId],[PartitionId]:[BrokerId]:... or alternatively the wildcard '*' can be used to throttle all replicas for this topic.
However, sometimes operational people may want to throttle all the replication leader / follower. It's hard for them to achieve this using the topic level replication throttles.
replica.fetch.wait.max.ms: The changes are similar to replica.fetch.response.max.bytes
Public Interfaces
Compatibility, Deprecation, and Migration Plan
- There won't be any impact on existing users.
- There won't be any change of in current behavior.
- No migration tool required
...