Status

Current state: Under Discussion

Discussion thread: TODO

JIRA: TODO

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Kafka has a family of metrics consisting of:

• org.apache.kafka.common.metrics.stats.Count
• org.apache.kafka.common.metrics.stats.Sum
• org.apache.kafka.common.metrics.stats.Total
• org.apache.kafka.common.metrics.stats.Rate.SampledTotal
• org.apache.kafka.streams.processor.internals.metrics.CumulativeCount

These metrics are all related to each other, but their relationship is obscure (and one is redundant) (and another is internal).

I've recently been involved in a third  recapitulation of trying to work out which metric does what. It seems like it's time to clean up the mess and save everyone from having to work out the mystery for themselves.

Public Interfaces

The affected public interfaces are:

sampled count metric:

• (deprecated) org.apache.kafka.common.metrics.stats.Count
• (new) org.apache.kafka.common.metrics.stats.SampledCount

sampled sum metric:

• (deprecated) org.apache.kafka.common.metrics.stats.Rate.SampledTotal
• (deprecated) org.apache.kafka.common.metrics.stats.Sum
• (new) org.apache.kafka.common.metrics.stats.SampledSum

non-sampled count metric:

• (internal: removed) org.apache.kafka.streams.processor.internals.metrics.CumulativeCount
• (new) org.apache.kafka.common.metrics.stats.CumulativeCount

non-sampled sum metric:

• (deprecated) org.apache.kafka.common.metrics.stats.Total
• (new) org.apache.kafka.common.metrics.stats.CumulativeSum

Proposed Changes

The existing metrics cover four quadrants of a matrix:

It's immediately apparent that there's no consistency in naming, that there's a missing quadrant, and that one quadrant is redundantly covered.

The proposal is simple:

Under this proposal, the metrics are clearly and regularly named and all quadrants are covered uniquely. There is no ambiguity in the names, and the structure of the names also indicate a pattern that would guide users to select the correct metric for their needs.

Compatibility, Deprecation, and Migration Plan

Existing metrics are deprecated in favor of unambigously named ones. They will be made to subclass the new metrics to avoid code duplication, but this won't cause any code compatibility issues, since they'll still inherit the same interfaces.

Rejected Alternatives

none