...

• Recording metrics is expensive. There are scenarios where, for debug purposes, tens or hundreds of metrics need to be recorded. The performance impact of always recording can be as high as 50% at times (see graph below showing some Kafka Streams microbenchmarks. This graph is also part of PR for the JIRA for this KIP. Compare the yellow bar which is with all metrics enabled, with any other bar). We would like a way to control which metrics are recorded to address this performance problem.
  
  
  
  
• This KIP proposes the addition of a RecordLevel in Sensor.java. This associates each sensor with a record level and the metrics and computations associated with the sensor are recorded or calculated only if the metric config of the client requires these metrics to be recorded. For example, a sensor associated with a RecordLevel.DEBUG record level will only record metrics, (and correspondingly compute the time.milliseconds or nanoseconds if required by its metrics), if the client config for metric record levels is set to DEBUG. Note that the sensor will still be registered and polled by any reporters, however the expensive part of updating its metrics will not be done. This provides a mechanism for clients to toggle metric granularity without a change to individual sensors, and we do not perform unnecessary intensive computations associated on metrics which must not be recorded.
• An objective of this KIP is to leave open the possibility of changing the record level at runtime, although it is not in the scope of this KIP to implement that right now.

Public Interfaces

• Changes to public class Sensor.java to include Sensor.RecordLevel
• Addition of static configuration string in CommonClientConfigs.java
• Exposing that configuration string in StreamsConfig.java and KafkaConfig.scala since both use CommonClientConfigs.

...