Status

Current state: "Voting in Progress" Accepted

Vote thread: https://lists.apache.org/thread/pxc1sx0kb35kd8dsccc10c0pp51kp8cj

Discussion thread: DISCUSS KIP-1052 

JIRA: KAFKA-17645

Motivation

When analyzing performance of long-running applications like Kafka, it is useful to separate the application's lifecycle into phases such as initialization, steady-state, transient-states due to changes in system load, and shut-down. Of these states, steady-state and transient-response-states are of special interest since these represent the normal operating states of the application. To measure the steady state performance of Kafka, one would typically set up a cluster, then create topics and proceed to measure the performance of various load points on their cluster. 
 
A problem arises, however, when using the producer performance test to measure steady-state performance. In our observations, the producer performance test has high latency during the initialization phase as producers are coming online and brokers are beginning to exchange replicated messages. Currently, all measurements with the producer performance test will include measurements of the high latency that occurs during startup in any evaluation of steady-state performance, especially when test durations are not sufficiently long. 

...

An additional feature for producer warmup would be for the producer to automatically detect when its performance has stabilized then switch modes into steady state. The producer could compare variance between windows then choose to switch to steady state once the between-window variance had reached a threshold at a specified minimum. Statistics such as average latency, p99 latency, or maximum latency could be used to determine when variance has reached steady state, depending on user requirements Mechanisms for automatic detection of producer stability at end of warmup include simple checks such as time-based warmup or latency-based warmup, but monitored performance indicators could also include DNS queries, broker SSL connections, Producer Throttler actions, response intervals from brokers, and even JVM compilation activity. While this could dramatically simplify the use of the producer performance tool, automatic determination of the correct limits at which the producer should switch modes is a complex tuning problem, since that would effectively be defining steady state for all deployments. Due to this additional complexity, we believe that auto-detection of the producer’s warmup phase should be discussed in a subsequent KIP.

...