...

The maximum amount of time in milliseconds to wait when retrying a request to the broker that has repeatedly failed. If provided, the backoff per client will increase exponentially for each failed request, up to this maximum. To prevent all clients from being synchronized upon retry, a randomization factor of 0.2 will be applied to the backoff, resulting in a random range between 20% below and 20% above the computed value. If retry.backoff.ms is set to be higher than retry.backoff.max.ms, then retry.backoff.max.ms would be used as the backoff.

retry.backoff.max.ms would default to 1000 ms, and the starting backoff will be derived from retry.backoff.ms (which defaults to 100 ms). If This config would default to 1000 ms if retry.backoff.ms is not set explicitly. Otherwise, it will default to the same value as set to be greater than retry.backoff.max.ms, then retry.backoff.max.ms will be used as the backoff. The formula to calculate the backoff is as follows:

MIN(

...

retry.backoff.max.ms

...

, (retry.backoff.ms * 2**(failures - 1)) * random(0.8, 1.2))

"random" in this case is the random function that will randomly factor in a "jitter" that is 20% higher or lower to the computed value. This will keep increasing until it hits the retry.backoff.max.ms value.

...

Compatibility, Deprecation, and Migration Plan

For users who have not set retry.backoff.ms explicitly, the default behavior will change so that the backoff will grow up to 1000 ms. For users who have set retry.backoff.ms explicitly, the behavior will remain the same as they could have specific requirementsThis KIP will replace the old static retry backoff behavior with this more dynamic retry backoff behavior, and as a result there will be no forward/backwards compatibility to consider.

Rejected Alternatives

1. Default retry.backoff.max.ms to the same value as retry.backoff.ms so that existing behavior is always maintained: for reasons explained in the compatibility section.
2. Default retry.backoff.max.ms to be 1000 ms unconditionally: for reasons explained in the compatibility section.