Page History

...

• This KIP proposes the addition of latency and throughput metrics for Kafka Streams at the granularity of each processor node and the addition of count metrics at the granularity of each task. This is in addition to the global rate (which already exists). The idea is to allow users to toggle the recording of these metrics when needed for debugging. The RecordLevel for these granular metrics is debug DEBUG, and a client can toggle the record level by changing the  “metrics.record.level” in the client config. (The introduction of RecordLevel and client config changes are covered in the separate KIP-105).
• This KIP also proposes exposing the metrics registry and several helper functions so that Kafka Streams users can register their own metrics. 

...

• A StreamsMetrics class with the following methods:
  

  public interface StreamsMetrics {
      
  /**
   * @return The base registry where all the metrics are recorded.
   */
   Metrics registry();
  
   /**
   * Add a latency sensor. This is equivalent to adding a sensor with metrics on latency and rate.
   *
   * @param scopeName Name of the scope, could be the type of the state store, etc.
   * @param entityName Name of the entity, could be the name of the state store instance, etc.
   * @param recordLevel The recording level (e.g., INFO or DEBUG) for this sensor.
   * @param operationName Name of the operation, could be get / put / delete / etc.
   * @param tags Additional tags of the sensor.
   * @return The added sensor.
   */
   Sensor addLatencySensor(String scopeName, String entityName, String operationName, Sensor.RecordLevel recordLevel, String... tags);
  
   /**
   * Record the given latency value of the sensor.
   * @param sensor sensor whose latency we are recording.
   * @param startNs start of measurement time in nanoseconds.
   * @param endNs end of measurement time in nanoseconds.
   */
   void recordLatency(Sensor sensor, long startNs, long endNs);
  
   /**
   * Add a throughput sensor. This is equivalent to adding a sensor with metrics rate.
   *
   * @param scopeName Name of the scope, could be the type of the state store, etc.
   * @param entityName Name of the entity, could be the name of the state store instance, etc.
   * @param recordLevel The recording level (e.g., INFO or DEBUG) for this sensor.
   * @param operationName Name of the operation, could be get / put / delete / etc.
   * @param tags Additional tags of the sensor.
   * @return The added sensor.
   */
   Sensor addThroughputSensor(String scopeName, String entityName, String operationName, Sensor.RecordLevel recordLevel, String... tags);
  
   /**
   * Records the throughput value of a sensor.
   * @param sensor sensor whose throughput we are recording.
   * @param value throughput value.
   */
   void recordThroughput(Sensor sensor, long value);
  
   /**
   * Remove a sensor with the given name.
   * @param name Sensor name to be removed.
   */
   void removeSensor(String name);
  }

...