This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Under Discussion"

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here and here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

If you are a Kafka user and you want to track the progress a client is making you can use the returned committed offset. However Streams does not have that capability as several Kafka clients are embedded in Streams client. This KIP proposes to add methods to Kafka Streams that report the progress Kafka exposes already.

Public Interfaces

package org.apache.kafka.streams;
  
KafkaStreams.java
/**
* Returns the last committed offset for each TopicPartition in a task assigned to this client.
*/
public Map<TopicPartition, Long> committedOffsets();

/**
* Returns a map telling if each task is idling.
*
*/
public Map<TaskId, boolean> tasksIdling()

Proposed Changes

committedOffsets

This should have the latest committed off set of each task assigned to the client. Each poll will update this. The offsets committed by the task manger will be saved. Each time a new commit is made the map containing the offsets will update. When committedOffsets() is called then it will return this map.

isTaskIdling

If a task assigned to this instance is intentionally not pulling because it is idling this method will report that

Compatibility, Deprecation, and Migration Plan

Since methods are only added and no other methods are modified, this KIP should not

• affect backward-compatibility
• deprecate public interfaces
• need a migration plan other than adding the new metrics to its own monitoring component

Rejected Alternatives

• reporting these as metrics