Status

Current state: Adopted (2.0)

Discussion thread: here 

Voting thread: here

JIRA: KAFKA-6874

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

With the release of 2.0 Kafka Streams will introduce topology optimization, the goal of which is to reduce the footprint Kafka Streams creates especially with repartition and changelog topics. To that end, we would like to introduce a new config "topology.optimization". The default value for this new configuration will be "none". Kafka Streams needs this configuration to ensure backward compatibility.

Public Interfaces

There are no changes planned to public interfaces with this change.

Proposed Changes

This KIP will add a new config "topology.optimization" to the StreamsConfig class.  Setting TOPOLOGY_OPTIMIZATION to "all" will enable optimization, setting this configuration to "none" disables optimization.  The default setting is "none".

public static final String TOPOLOGY_OPTIMIZATION = "topology.optimization" 
private static final String TOPOLOGY_OPTIMIZATION_DOC = "A configuration telling Kafka Streams if it should optimize the topology, none by default"

Compatibility, Deprecation, and Migration Plan

There is no backward compatibility issue as we are not deprecating or changing any public API. 

Since the default value is false, the underlying change should be transparent to the user, with no impact if optimization is turned off. 

The impact of turning on optimization may result in a different topology than as initially written.   As a result, users electing to use optimization should plan for shutting down all instances, cleaning up any local state and internal topics using the Streams-reset tool then starting the application again with optimization turned on via configs.  

Rejected Alternatives

None