...
It would be beneficial to add a metric to record the average/max time between calls to poll metrics to track poll calls as it can be used by both Kafka application owners and operators to:
...
We will add the following metrics:
Metric Name | Mbean Name | Description |
---|---|---|
time-between-poll-avg | kafka.consumer:type=consumer-metrics,client-id=([-.\w]+) | The average delay between invocations of poll(). |
time-between-poll-max | kafka.consumer:type=consumer-metrics,client-id=([-.\w]+) | The max delay between invocations of poll(). |
poll-idle-ratio | kafka.consumer:type=consumer-metrics,client-id=([-.\w]+) | The fraction of time the consumer spent waiting for the user to process records from poll. |
Proposed Changes
time-between-pool
As we want this metric to measure the time that the user takes to call poll()
, we will store a long lastPollMs
in KafkaConsumer. We will calculate the elapsed time (and update lastPollMs)
on every call to poll.
On each call to poll()
, we will calculate the elapsed time since lastPollMs
, record it in the Sensor
which has Avg
& Max
, and update lastPollMs
.
poll-idle-ratio
There will be an additional metric that measures:
LaTeX Formatting |
---|
$\textrm{poll-idle-ratio} = \frac{\textrm{time-between-poll}}{\textrm{time-inside-poll}}$ |
Compatibility, Deprecation, and Migration Plan
...