Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-813 introduced "read-only state stores" in Kafka Streams. These state stores use their source topic as a changelog for recovery rather than creating a dedicated changelog topic. While this functionality was added to the Processor API (Topology.addReadOnlyStateStore), it is not currently accessible through the high-level DSL API. This improvement would expose this functionality through the StreamsBuilder class.

Read-only state stores are useful for efficiently sharing state across applications, especially with the introduction of tiered storage support, where compacted topics can grow very large. By using the source topic as a changelog, applications can avoid duplicating large amounts of data. Making this functionality available in the high-level DSL would provide a more user-friendly interface to this feature.

Public Interfaces

org/apache/kafka/streams/StreamsBuilder will be extended with an addReadOnlyStateStore method, allowing the changelog topic to be passed in.

Proposed Changes

The proposed DSL method has a simplified signature compared to the Processor API method, maintaining consistency with other DSL methods like addGlobalStore, which use the same parameter pattern:

StreamsBuilder addReadOnlyStateStore(final StoreBuilder<?> storeBuilder,
									 final String topic,
    								 final Consumed<KIn, VIn> consumed,
									 final ProcessorSupplier<KIn, VIn, Void, Void> stateUpdateSupplier)

Compatibility, Deprecation, and Migration Plan

This change is fully backwards compatible as it only adds new methods
No migration is needed
No features are being deprecated

Test Plan

Integration tests will be added to verify:

Basic read-only state store functionality in the DSL
State recovery after restart
Thread shutdown behaviour

Rejected Alternatives

None. This is a straightforward exposure of existing functionality to a higher-level API.