Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Renamed a few metrics and added two more to follow the new pattern of `-rate` and `-total` metric pairs defined in KIP-187

...

Metric/Attribute NameDescription
source-record-poll-rateThe average per-second number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker.
source-record-poll-counttotalThe total number of records produced/polled (before transformation) by this task belonging to the named source connector in this worker, since the task was last restarted.
source-record-write-rateThe average per-second number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker. This is after transformations are applied and excludes any records filtered out by the transformations.
source-record-write-counttotalThe total number of records output from the transformations and written to Kafka for this task belonging to the named source connector in this worker, since the task was last restarted.
poll-batch-max-time-msThe maximum time in milliseconds taken by this task to poll for a batch of source records
poll-batch-99p-time-msThe 99th percentile time in milliseconds spent by this task to poll for a batch of source records
poll-batch-95p-time-msThe 95th percentile time in milliseconds spent by this task to poll for a batch of source records
poll-batch-90p-time-msThe 90th percentile time in milliseconds spent by this task to poll for a batch of source records
poll-batch-75p-time-msThe 75th percentile time in milliseconds spent by this task to poll for a batch of source records
poll-batch-50p-time-msThe 50th percentile (average) time in milliseconds spent by this task to poll for a batch of source records

...

Metric/Attribute NameDescription
sink-record-read-rateThe average per-second number of records read from Kafka (before transformations are applied) for this task belonging to the named sink connector in this worker.
sink-record-read-counttotalThe total number of records read from Kafka (before transformations are applied) for this task belonging to the named sink connector in this worker, since the task was last restarted.
sink-record-send-rateThe average per-second numbrer of records output from the transformations and sent to this task belonging to the named sink connector in this worker. This is after transformations are applied and excludes any records filtered out by the transformations.
sink-record-send-counttotalThe numbrer total number of records output from the transformations and sent to this task belonging to the named sink connector in this worker, since the task was last restarted.
sink-record-lag-maxThe maximum lag in terms of number of records behind the consumer the offset commits are for any topic partitions.
partition-countThe number of topic partitions assigned to this task belonging to the named sink connector in this worker.
offset-commit-seq-noThe current sequence number for offset commits
offset-commit-completion-rateThe average per-second number of offset commit completions that were completed successfully
offset-commit-completion-totalThe total number of offset commit completions that were completed successfully
offset-commit-completion-skip-rateThe average per-second number of offset commit completions that were received too late and skipped/ignored
offset-commit-completion-skip-totalThe total number of offset commit completions that were received too late and skipped/ignored
put-batch-max-time-msThe maximum time taken by this task to put a batch of sinks records
put-batch-99p-time-msThe 99th percentile time in milliseconds spent by this task to put a batch of sinks records
put-batch-95p-time-msThe 95th percentile time in milliseconds spent by this task to put a batch of sinks records
put-batch-90p-time-msThe 90th percentile time in milliseconds spent by this task to put a batch of sinks records
put-batch-75p-time-msThe 75th percentile time in milliseconds spent by this task to put a batch of sinks records
put-batch-50p-time-msThe 50th percentile (average) time in milliseconds spent by this task to put a batch of sinks records
flush-max-time-msThe maximum time in milliseconds taken by this sink task to pre-commit/flush
flush-99p-time-msThe 99th percentile time in milliseconds spent by this sink task to pre-commit/flush
flush-95p-time-msThe 95th percentile time in milliseconds spent by this sink task to pre-commit/flush
flush-90p-time-msThe 90th percentile time in milliseconds spent by this sink task to pre-commit/flush
flush-75p-time-msThe 75th percentile time in milliseconds spent by this sink task to pre-commit/flush
flush-50p-time-msThe 50th percentile (average) time in milliseconds spent by this sink task to pre-commit/flush

...

Metric/Attribute NameDescription

task-count

The number of tasks run in this worker
connector-countThe number of connectors run in this worker
leader-nameThe name of the group leader
epochThe epoch or generation number of this worker
statusThe state of this worker, one of: rebalancing, running
status-rebalancingSignals whether the worker is in the rebalancing state
status-runningSignals whether the worker is in the running state
rest-request-rateThe average per second number of requests handled by the REST endpoints in this worker
rest-request-totalThe total number of requests handled by the REST endpoints in this worker, since the worker was last restarted.


Worker Rebalance Metrics

MBean namekafka.connect:type=connect-worker-rebalance-metrics

...

Metric/Attribute NameDescription
rebalance-success-totalThe total number of this worker's successful rebalances
rebalance-success-percentageThe average percentage of this worker's rebalances that succeeded
rebalance-failure-totalThe total number of this worker's failed rebalances
rebalance-failure-percentageThe average percentage of this worker's rebalances that failed
rebalance-max-time-msThe maximum time in milliseconds spent by this worker to rebalance
rebalance-99p-time-msThe 99th percentile time in milliseconds spent by this worker to rebalance during the last window (defaults to an hour)
rebalance-95p-time-msThe 95th percentile time in milliseconds spent by this worker to rebalance during the last window (defaults to an hour)
rebalance-90p-time-msThe 90th percentile time in milliseconds spent by this worker to rebalance during the last window (defaults to an hour)
rebalance-75p-time-msThe 75th percentile time in milliseconds spent by this worker to rebalance during the last window (defaults to an hour)
rebalance-50p-time-msThe 50th percentile (average) time in milliseconds spent by this worker to rebalance during the last window (defaults to an hour)
time-ms-since-last-rebalanceThe time in milliseconds since the most recent rebalance in this worker
task-failure-rateThe average per second number of tasks that failed in this worker
task-failure-totalThe total number of tasks that failed in this worker

...