THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
All other requests may be throttled if the rate exceeds the configured quota. Produce/Fetch requests will return the total throttling time reflecting both bandwidth and utilization based throttling in the response. All other requests that may be throttled will have an additional field throttle_time_ms to indicate to the client that the request was throttled. The versions of these requests will be incremented.
...
Clients will expose average and maximum request throttle time as JMX metrics similar to the current produce/fetch throttle time metrics. These The existing metrics will reflect the total throttle time for produce and fetch requests including both byte-rate throttling and processing time throttling. Two new metrics request-throttle-time-max and request-throttle-time-min will be added to reflect total request processing time based throttling for all requestsrequest types including produce/fetch.
Tools
kafka-configs.sh will be extended to support request quotas. A new quota property will be added, which can be applied to <client-id>, <user> or <user, client-id>:
...
The maximum throttle time for any single request will be the quota window size (one second by default). This ensures that timing-sensitive requests like heartbeats are not delayed for extended durations. For example, if a user has a quota of 0.001 and a stop-the-world GC pause takes 100ms during the processing of the user's request, we don't want all the requests from the user to be delayed by 100 seconds. By limiting the maximum delay, we reduce the impact of GC pauses and single large requests. To exploit this limit to bypass quota limits, clients would need to generate requests that take significantly longer than the quota limit. If R is the amount of time taken process one request and the user has C active connections, the maximum amount of time a user/client can use in the long term use per quota window is max(quota, C * R). In practice, quotas are expected to be much larger than the time taken to process individual requests and hence this limit is sufficient. Byte rate quotas will also additionally help to increase throttling in the case where large produce/fetch requests result in larger per-request time. DoS attacks using large numbers of connections is not addressed in this KIP.
...
Two new metrics and corresponding sensors will be added to track request-time and throttle-time of each quota entity for quota type IOThread. The request-time sensor will be configured with the quota for the user/client so that quota violations can be used to add delays to the response. Quota window configuration (quota.window.size.seconds) will be the same as the existing configuration for produce/fetch quotas: 1 second window with 11 samples retained in memory by default. A new delay queue sensor will also be added for quota type IOThread. All the new sensor names (IOThread-<quota-entity>, IOThreadThrottleTime-<quota-entity> and IOThread-delayQueue) are prefixed by the quota type, making these sensors consistent with existing sensors for Produce/Fetch. The new metrics will be in the metrics group IOThread, distinguishing these from similar metrics for Produce/Fetch byte rates.
...
On the client side, a new sensor name named request-throttle-time will be added to track total request throttle time returned in all responses. This is in addition to the sensor used to track produce/fetch throttle times, which will continue to be supported. Existing These existing produce/fetch throttle times will include total throttling time for both bandwidth and utilization for produce/fetch requests. Maximum and average throttle times for request time based throttling will be exposed as metrics in addition to throttle time metrics for New metrics request-throttle-time-avg and will be added and these will include throttle times across all requests including produce/fetch.request-throttle-time-max
| Anchor | ||||
|---|---|---|---|---|
|
...