Page History

...

An additional feature for producer warmup would be for the producer to automatically detect when its performance has stabilized then switch modes into steady state. The producer could compare variance between windows then choose to switch to steady state once the between-window variance had reached a threshold at a specified minimum. Statistics such as average latency, p99 latency, or maximum latency could be used to determine when variance has reached steady state, depending on user requirements Mechanisms for automatic detection of producer stability at end of warmup include simple checks such as time-based warmup or latency-based warmup, but monitored performance indicators could also include DNS queries, broker SSL connections, Producer Throttler actions, response intervals from brokers, and even JVM compilation activity. While this could dramatically simplify the use of the producer performance tool, automatic determination of the correct limits at which the producer should switch modes is a complex tuning problem, since that would effectively be defining steady state for all deployments. Due to this additional complexity, we believe that auto-detection of the producer’s warmup phase should be discussed in a subsequent KIP.

...