Status

Current state: Under Discussion

Discussion thread:

JIRA: JIRA 6657

Motivation

Kafka Streams allows to pass in different configs for different clients by prefixing the corresponding parameter with `producer.` or `consumer.`.

However, Kafka Streams internally uses multiple consumers, (1) the main consumer (2) the restore consumer and (3) the global consumer

For some use cases, it's required to set different configs for different consumers. Thus, we should add two new prefix for restore and global consumer. 

We might also consider to extend `KafkaClientSupplier` and add a `getGlobalConsumer()` method.

Public Interfaces

The changes affected classes including StreamsConfig and KafkaClientSupplier. Existing APIs look like below:

public static final String CONSUMER_PREFIX = "consumer.";


public Map<String, Object> getConsumerConfigs(final String groupId, final String clientId);
public Map<String, Object> getRestoreConsumerConfigs(final String clientId); 

// Current KafkaClientSupplier consumer APIs: 
Consumer<byte[], byte[]> getConsumer(final Map<String, Object> config);
Consumer<byte[], byte[]> getRestoreConsumer(final Map<String, Object> config);

Proposed Changes

We shall add a new public API in the KafkaClientSupplier class to get global consumer during init. 

// New API for global consumer:
Consumer<byte[], byte[]> getGlobalConsumer(final Map<String, Object> config); 

Compatibility, Deprecation, and Migration Plan

There is no backward compatibility issue as we are not deprecating any public API, and the underlying change should be transparent to the user.

Rejected Alternatives

Non.