Page History

...

Current state: "Under Discussion"

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

...

• High-load or unstable-networking clusters: Where metadata topic writing or consumer offset reads might experience transient delays exceeding 30 seconds.
• Large cluster with many connector tasks: For a large Kafka Connect cluster with more than 500 thousands connector tasks, it will take some time to write each task’s configuration and then read back .  The configuration topic is designed to be a single partition topic which further reduces the parallelism.  When a worker starts up and needs to read a compacted configuration topic with a significant history, the readToEnd operation might time out.

We have expressed the need to increase this timeout to avoid ConnectException  during these spikes (see discussion thread here).  This This KIP proposes making this timeout a configurable worker parameter to allow operators to tune it according to their environment's performance characteristics.

...