This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Status
Current state: Under Discussion
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
Broker merges the metrics of paritiotns in the same topic. That obstructs us from monitoring the metrics for each partition. If we consider the data In and data Out are based on “partition”, merging partition metrics is not good to data monitoring. For instance, our production has a monitor tool offers visible data flow and traffic. Recently, we plan to make data balance visiable so developers are capable to see the trend around partitions generated by custom data dispatcher (i.e partitioner). The data dispatcher manages to get load balance by calculating cost of data moving from/to paritions (of course, broker is a cost function as well) and the balance plan is visible. Unfortunately, we can’t filter out specify partition metrics from broker since all of them are squashed.
Public Interfaces
N/A
Proposed Changes
this KIP is to add a tag formed as “partition=xxx” to BrokerTopicMetrics so we can see more precise metrics for specify partition.
type=BrokerTopicMetrics,name=BytesInPerSec,topic=xxx,parition=0
Compatibility, Deprecation, and Migration Plan
The downside of this change are :
- breaking the metrics compatibility
- Broker needs more space to keep metrics
Rejected Alternatives
In order to eliminate the concerns of above downsided, the alternative, which is more complicated, is to add an new config to enable/disable this new metrics and offers enough document to explain the benefit and cost of this new metrics.