Status

Current state: Under Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The TaskMetadata class in KafkaStreams is used for encoding information about a particular task, such as its offsets, topic partitions, and taskId. For some reason, the taskId is stored and exposed as a String object, rather than using the actual TaskId class. We should move towards using TaskId herereturning this as a TaskId object, since that's literally what it is, and because if/when we add additional fields to the TaskId it will become more and more unwieldy to parse the string encoding in order to extract the actual information about the task.

At the same time, the existing TaskId class is a bit awkward, as it's clearly intended to be a simple metadata class to expose the topicGroupId (subtopology) and partition number fields. Yet it has plenty of totally irrelevant (to a user) utility methods, such as de/serialization. It also has completely public fields, and no actual getters for them. We should take this opportunity to migrate to a new public interface with only the getters for the relevant fields, and move all the implementation details and utilities to an internal implementation class.

Public Interfaces

New TaskIdMetadata TaskInfo interface added to the public API:

package org.apache.kafka.streams.processor;

/**
 * A basic wrapper class for basic task id metadatainfo
 */
public interface TaskIdMetadataTaskInfo {
    /**
     * @return The ID of the topic group, ie the subtopology that this task executes, ie the topicGroupId
     */
    int topicGroupIdsubtopologyId();

    /**
     * @return The ID number of the partition.
     */
    int partition();
}

"New" internal TaskId TaskIdImpl class that implements the public TaskIdMetadata TaskId interface:

package org.apache.streams.processor.internals;

class TaskId implements TaskIdMetadataTaskInfo {
    // This is just the existing TaskId class & implementation, but
    // moved to the internals package
}

...

Deprecate TaskId getter on the StateStoreContext and ProcessorContext interfaces, and replace with a getter that returns TaskIdMetadatathe TaskInfo interface:

public interface ProcessorContext {

    /**
     * Returns the task id metadatainfo.
     *
     * @return the task id metadatainfo
     */
    TaskIdMetadataTaskInfo taskIdMetadatataskInfo();
    
    /**
     * Returns the task id.
     *
     * @return the task id
     * @deprecated use {@link #taskIdMetadata#taskInfo()} instead.
     */
    @Deprecated
    TaskId taskId();
}

public interface StateStoreContext {
    /**
     * Returns the task id metadatainfo.
     *
     * @return the task id metadatainfo
     */
    TaskIdMetadataTaskInfo taskIdMetadatataskInfo();
    
    /**
     * Returns the task id.
     *
     * @return the task id
     * @deprecated use {@link #taskIdMetadata#taskInfo()} instead.
     */
    @Deprecated
    TaskId taskId();
}

Deprecate the taskId() getter that returns the task id as a string, and add an API to return the actual TaskIdMetadata objectTaskInfo:

package org.apache.kafka.streams.processor;

class TaskMetadata {

    private final TaskIdMetadataTaskId taskId;

    /**
     * @return the TaskId with additionalbasic task metadata such as partitionsubtopology and grouppartition id
     */
    public TaskIdMetadataTaskInfo idtaskInfo() {
        return taskIdMetadatataskId;
    }

    /**
     * @return athe string representing the TaskIdbasic task metadata such as partitionsubtopology and grouppartition id
     * @deprecated please use {@link #id#taskInfo()} instead.
     */
    @Deprecated
    public String taskId() {
        return taskId.toString();
    }
}

Proposed Changes

To separate the implementation from the the public API, we'll deprecate the existing o.a.k.streams.processor.TaskId class and replace it with a public interface o.a.k.streams.processor.TaskIdMetadataTaskInfo. The existing TaskId class implementation will be moved to a new o.a.k.streams.processor.internals.TaskId class, which will implement the added TaskIdMetadata interface.We'll introduce a new TaskIdMetadata interface in the public API that's implemented by the current TaskId class, and move the TaskId class to an internal package to clarify that it's not a publicTaskInfo interface.

To migrate from the String to TaskIdMetadata TaskInfo in TaskMetadata, we'll need to deprecate the existing taskId() getter method and add a taskIdMetadatataskInfo() getter in its place. Similarly, we will need to deprecate the existing taskId() getters on the ProcessorContext and StateStoreContext interfaces, and replace them with taskIdMetadatataskInfo() getters that return the new public interface instead of the now-deprecated class.

Compatibility, Deprecation, and Migration Plan

The TaskMetadata#taskId(), ProcessorContext#taskId(), and StateStoreContext#taskId() methods will be deprecated and removed in a future release. The entire o.a.k.streams.processor.TaskId class will also be deprecated and later removed.

Rejected Alternatives

N/A