...

Set quota to a very low value to verify that the clients are neither timing out or sending more requests to the broker before throttle time has passed.

Amendments

Instead of bumping up the protocol version of all requests/responses that may be throttled, we decided to bump up the protocol version of ApiVersionsRequest and ApiVersionsResponse only, to indicate clients whether or not brokers perform throttling before sending out responses. We think this is sufficient given that network client exchanges ApiVersionsRequest/Response with each broker when establishing connection to it and thus it can detect the broker's throttling behavior just by examining the ApiVersionsResponse version.

Rejected Alternatives

One potential solution to the above problem is to set a very high request.timeout.ms on the client side. This is not ideal because the request timeout is not supposed to be set in response to a throttling condition.