You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Status

Current state: "DISCUSS"

Discussion thread: 

JIRA: KAFKA-12313

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).


Motivation

  • KIP-659 introduced a config windowSize to TimeWindowedDeserializer which discourages the setting of windowSize to Long.MAX_VALUE as the default value. However, there is still a possibility of setting the config only for innerClassSerde or only for windowSize. This KIP aims at standardising the way the deserializer object can be created for the usage in console consumer and via the DSL. 
  • The KIP also introduces changes around the default.windowed.serde.inner.class configs. These are:
    • Rename the default.windowed.key.serde.inner to windowed.deserializer.inner.class config. This way, the config comes closer to the window.size.ms config introduced in KIP-659 and it also emphasises that the config isn't really a default one. Also, as per the proposed changes below, this config is to be used really from the console consumer so we can use Deserialiser class here directly instead of a Serde.
    • Deprecate the default.windowed.value.serde.inner config. This is because there's really no concept of a "WindowedValue" in Kafka Streams, unlike the "WindowedKey" which appears in the DSL. The Windowed class in Streams itself assumes that the inner class is a key. If a user really does want a WindowedValue in some custom processor for some reason, they would have to write the WindowedValue class themselves. In that case I think it's reasonable to assert that they should be responsible for writing the WindowedValueSerializer class as well, and pass this to the console consumer should they need to use it.

Proposed Changes

Here are the changes being proposed in the KIP:

  • Rename the default.windowed.key.serde.inner to windowed.deserializer.inner.class in StreamConfig.
  • Deprecate default.windowed.value.serde.inner in StreamConfig.
  • If the users want to run Console consumer for testing purposes, then it will invoke the default constructor of TimeWindowedDeserializer. We will ensure that both the configs (windowSize and deserialiser class) are set and if they aren't then an error would be thrown. 
  • For any other plain consumer client, the user can pass them in as configs OR pass the parameters to the TimeWindowedDeserializer constructor, and then pass that object to the consumer. It is ok for the user to pass both as long as there is no conflict between the params passed via constructor and via the configs.

Public Interfaces

  • StreamsConfig:
    • Rename default.windowed.key.serde.inner to windowed.key.serde.inner.
    • Deprecate default.windowed.value.serde.inner




  • No labels