Status

Current state: Under Discussion Accepted

Discussion thread: here

JIRA: here

...

This config must be set to at least one1.

The default value is set to 10. We ran experiments showing the impact of zookeeper.max.in.flight.requests on completion times for various ZooKeeper-intensive controller protocols. The default was chosen to be the smallest number beyond which the experiment results have found diminishing returns.

...

1. Set the default value to be unbounded (Integer.MAX_VALUE), effectively stating that no such limit be applied and to pipeline as aggressively as possible. This was rejected because a config should not default to removing control over the system.
2. Set the default value to one1, effectively disabling pipelining and maintaining Kafka's synchronous requests to ZooKeeper. This was rejected because we want users to see the benefits of pipelining.