Status

Current state: Under Discussion"Accepted

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

Public Interfaces

New Metrics

Kafka Streams

blocked-time-total
tags: thread-id, application-id
group: stream-thread-metrics
level: INFO. The proposed metrics should be collectible at INFO level without adding meaningful overhead. They require sampling the time twice during the corresponding API calls, which nowadays is very cheap.
description: the total time the Kafka Streams thread spent blocked on Kafka.

thread-start-time
tags: thread-id
group: stream-thread-metrics
level: INFO
description: the epoch time the Kafka Streams thread was started. This is useful for computing the processing ratio during the first interval after the thread starts. 

Producer

flush-time-ns-total
tags: client-id
group: producer-idmetrics
level: INFO
description: the total time the Producer spent in `Producer.flush` in nanoseconds.

txn-init-time-ns-total
tags: client-id
group: producer-metrics
level: INFO
description: the total time the Producer spent initializing transactions in nanoseconds (for EOS).

txn-begin-time-ns-total
tags: client-id
group: producer-metrics
level: INFO
description: the total time the Producer spent in beginTransaction in nanoseconds (for EOS).

txn-send-offsets-time-ns-total
tags: client-id
group: producer-idmetrics
level: INFO
description: the total time the Producer spent sending offsets to transactions transactions in nanoseconds (for EOS).

txn-commit-time-ns-total
tags: client-id
group: producer-idmetrics
level: INFO
description: the total time the Producer spent committing transactions in nanoseconds (for EOS).

txn-abort-time-ns-total
tags: client-id
group: producer-idmetrics
level: INFO
description: the total time the Producer spent aborting transactions transactions in nanoseconds (for EOS).

Consumer

commitcommited-synctime-timens-total
tags: client-id
group: consumer-idmetrics
level: INFO
description: the total time the Consumer spent committing offsets (for AOS)in committed in nanoseconds.

threadcommit-startsync-time-ns-total
tags: threadclient-id, application-id
group: consumer-metrics
level: INFO
description: the epoch time the Kafka Streams thread was started. This is useful for computing the processing ratio during the first interval after the thread starts. total time the Consumer spent committing offsets in nanoseconds (for AOS).

Proposed Changes

flush-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.flush.

txn-init-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.initTransactions.

txn-begin-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.beginTransaction.

txn-send-offsets-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.sendOffsetsToTransaction.

txn-commit-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.commitTransaction.

txn-abort-time-ns-total: this will be a Producer metric computed as the cumulative sum of time elapsed during calls to Producer.abortTransaction.

commited-time-ns-total: this will be a Consumer metric computed as the cumulative sum of time elapsed during calls to Consumer.committed.

commit-sync-time-ns-total: this will be a Consumer metric computed as the cumulative sum of time elapsed during calls to Consumer.commitSync.

blocked-time-ns-total: this will be a Value that returns the sum of the following metrics:

• consumer’s io-waittime-total
• consumer’s iotime-total
• consumer’s committed-time-ns-total
• consumer’s commit-sync-time-ns-total
• restore consumer’s io-waittime-total
• restore consumer’s iotime-total
• admin client’s io-waittime-total
• admin client’s iotime-total
• producer’s bufferpool-wait-time-total
• producer's flush-time-ns-total
• producer's txn-init-time-ns-total
• producer's txn-begin-time-ns-total
• producer's txn-send-offsets-time-ns-total
• producer's txn-commit-time-timens-total
• producer's txn-abort-time-ns-total

Compatibility, Deprecation, and Migration Plan

...