Page History

...

We propose to move the creation of the internal topics to an initialization method that users can call before they start their application for the first time. That way, users have full control over when the internal topics are created and Kafka Streams can reliably notify the users about possible data loss by throwing an exception when internal topics do unexpectedly not exist or are misconfigured. Additionally, we propose to add a configuration that allows to turn automatic creation of internal topics on or off. The configuration is needed for backward compatibility but also to keep the first steps with Kafka Streams simple for new users.

...

We propose to add two new exceptions –  namely MissingInternalTopicsException and MisconfiguredInternalTopicException – that help to discriminate errors originating from missing or misconfigured internal topics from other errors in the uncaught exception handler. For example, reacting on a missing source topic (i.e., MissingSourceTopicException) might be different from reacting on a missing or misconfigured internal topic, because the process for re-creating source topics might differ from the process for re-creating internal topics. Furthermore, source topics might be owned by a different team than the internal topics, consequently different people need to be paged. Misconfigured internal topics are internal topics whose configuration differs from the intended one. For example, a changelog topic for a non-windowed state store is expected to have a infinite retention. If such a changelog topic does not have infinite retention when it is verified, exception MisconfiguredInternalTopicException should be thrown. Exceptions MissingInternalTopicsException and MisconfiguredInternalTopicException will be thrown during explicit initialization and during a rebalance when the internal topics are verified. Both exceptions are fatal. 

Additionally, we propose to add an exception InternalTopicsAlreadySetupException that is thrown if users attempt to intialize an already initialized application. Exception InternalTopicsAlreadySetupException will be only thrown during explicit initialization. Such a behavior will ensure that users explicitly initialize the application only when:

...