THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
If you are a Kafka user and you want to track the progress a client is making you can use the returned committed offset. However Streams does not have that capability as several Kafka clients are embedded in Streams client. This KIP proposes to add metrics methods to Kafka Streams that repost the progress Kafka exposes already.
Public Interfaces
Each added metric will be on task-level and have the following tags:
- type = task-processing-metrics
- sub-topology = [sub-topology ID]
- thread-id = [thread ID]
- task-id = [task ID]
The following metrics will be exposed in the KafkaStreams' metrics:
- latest-committed-offset
- is-task-idling
- last-offset-produced
Proposed Changes
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
package org.apache.kafka.streams;
KafkaStreams.java
/**
* Returns the last committed offset for each task.
*/
public Map<TaskId, Integer> committedOffsets();
/**
* Returns if a task is idling.
*
* @param taskId the id for the task that is being checked.
*/
public boolean isTaskIdling(TaskId taskId) |
Proposed Changes
committedOffsets
This should have the latest committed off set of each task. Each poll will update this.
...
isTaskIdling
If the task is intentionally not pulling because it is idling this metric method will report that
last-offset-produced
This metric will report that last metric produced by the task.
Compatibility, Deprecation, and Migration Plan
Since metrics methods are only added and no other metrics methods are modified, this KIP should not
- affect backward-compatibility
- deprecate public interfaces
- need a migration plan other than adding the new metrics to its own monitoring component
Rejected Alternatives
...
- reporting these as metrics