Status

Current state: "Under Discussion" Accepted

Discussion thread: here

Vote Discussion thread: here

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

With this KIP, these metrics will also emit topic names without any modification to periods, matching the producer naming scheme. New consumer-fetch-manager-metrics will not have replaced dot (.) in topic names. "However for backward compatibility, the period replaced metrics will also be emitted. They are now deprecated and shall will be deprecated removed in a future major Kafka version (5.0)."

Proposed Changes

• Add additional metric without periods replacement in Consumer Metrics Topic Names:

  • If topic contains period (.), add metrics with same name in the Kafka consumer metrics to preserve the topic names as they are, without replacing periods with underscores. Keep emitting the existing period replaced metrics as well for backward compatibility.
  • If topic doesn't contain period (.), no change to existing metrics. 

Note: A drawback of the approach is that metrics for topics containing periods will be emitted twice: once with the actual topic name and once with periods replaced. Users must monitor either metric and may need to discard duplicate metrics for accurate reporting. But this is necessary to maintain backward compatibility and prevent disruption to existing user reporting.

Example:

Kafka 4.x:

• Topic Name: topic.a.b
  • Deprecated Metric: A metric with the tag topic_a_b will continue to be emitted, but it will be marked as deprecated.
  • New Metric: A new metric with the topic tag topic.a.b and same name has been introduced. Users should transition to this new metric.
• Topic Name: topic_c_d
  • Metric: A metric with the tag topic_c_d will continue to be emitted without any changes.
  • No Additional Metrics: No additional metrics are needed as the existing metric functions correctly.

Kafka 5.0:

The period replacement policy will be completely removed.

Compatibility, Deprecation, and Migration Plan

...