Page History

...

The client left as is risks imposing heavy load on ZooKeeper. ZooKeeper itself only has a coarse-grained throttling mechanism in place through its zookeeper.globalOutstandingLimit config which defaults to 1000. This config is insufficient for several reasons:1.

1. the limit is meant to protect ZooKeeper from memory pressure associated with a backlog of requests.

...

1. the limit is applied across all connections. Even with this config, one misbehaved client will affect the other clients.

We need a throttling mechanism in the client-side to give administrators control over Kafka's impact on ZooKeeper, and for this we propose a new broker config called zookeeper.max.in.flight.requests.

...