Status

Current state: Under Discussion

Discussion thread:

JIRA: KAFKA-6690

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

To support use-cases where there’s a need to prioritize source topics. If a consumer is subscribed to more than one topic, i.e., a high priority topic High and a low priority topic Low.  The consumer should consume events from topic Low only when all the events from topic High are consumed.

Public Interfaces

The addition of new subscribe API method that allows caller to prioritize topics.  

public TopicPriority(java.lang.String topic, int priority);

public void subscribe(java.util.List<TopicPriority> topicPriorities);

Proposed Changes

The new behavior is only in effect when the user specifies priorities for topics using the above new API.  In this case topics are checked starting from highest priority first and then in descending order based on priority.  All events must be consumed from a higher priority topic before consumption is performed on a lower priority topic. 

Compatibility, Deprecation, and Migration Plan

This feature preserves backward compatibility by treating topics without priorities as having the same priority.  

Rejected Alternatives

None