Status
Current state: Under Discussion
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
KIP-664: Provide tooling to detect and abort hanging transactions provided tooling to get visibility into transactional and idempotent producers that the broker keeps track of. This KIP proposes to add ProducerCount
metrics that enable easy monitoring of transactional and idempotent producer counts on the broker.
Public Interfaces
We propose adding 2 new broker metrics
Name | Description |
---|---|
kafka.cluster:type=Partition,name=ProducerIdCount,topic=<topic>,partition=<N> | The number of active transactional / idempotent producers that produced to topic <topic> partition <N>. |
kafka.server:type=ReplicaManager,name=ProducerIdCount | The total number of active transactional / idempotent producers in the broker. |
Proposed Changes
Add the new metrics to the ReplicaManager
and Partition
classes correspondingly.
Compatibility, Deprecation, and Migration Plan
- No migration plan is needed because these metrics are new
Rejected Alternatives
Name the metric ProducerCount
- may be misleading as the producers without producer ids are not counted.
Have 2 metrics IdempotentProducerCount
and TransactionalProducerCount
- currently we don't keep track which producer id is idempotent and which is transactional, adding that would add some complexity and potential runtime overhead, currently there doesn't seem to be a monitoring scenario that requires distinguishing between the two.