This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Under Discussion"

Discussion thread: here 

JIRA: here 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Kafka Streams has a set of configs with `default.` prefix. The intent for the default-prefix is to make a distinction between, well the default, and in-place overwrites in the code. Eg, users can specify ts-extractors on a per-topic basis.

However, for the deserialization- and production-exception handlers, no such overwrites are possible, and thus, `default.` does not really make sense, because there is just one handler overall. Via KIP-1033 we added a new processing-exception handler w/o a default-prefix, too.

Thus, we should consider to deprecate the two existing configs names and add them back w/o the `default.` prefix.

Public Interfaces

Changes to org.apache.kafka.streams.StreamsConfig

  1. The below configs in org.apache.kafka.streams.StreamsConfig will be deprecated

      2. Configs to be added to org.apache.kafka.streams.StreamsConfig

Proposed Changes

As such there are no functional changes at all. The main changes look like below.

         

		@Deprecated
        public static final String DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG = "default.deserialization.exception.handler";
        
        @Deprecated
        public static final String DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_DOC = "Exception handling class that implements the <code>org.apache.kafka.streams.errors.DeserializationExceptionHandler</code> interface.";

        /** {@code default.production.exception.handler} */
        @SuppressWarnings("WeakerAccess")
        @Deprecated
        public static final String DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG = "default.production.exception.handler";
        
        private static final String PRODUCTION_EXCEPTION_HANDLER_CLASS_DOC = "Exception handling class that implements the <code>org.apache.kafka.streams.errors.ProductionExceptionHandler</code> interface.";
        
        private static final String DESERIALIZATION_EXCEPTION_HANDLER_CLASS_DOC = "Exception handling class that implements the <code>org.apache.kafka.streams.errors.DeserializationExceptionHandler</code> interface.";

         /** {@code default.production.exception.handler} */
        @SuppressWarnings("WeakerAccess") 
        public static final String DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG = "deserialization.exception.handler";

        /** {@code default.production.exception.handler} */
        @SuppressWarnings("WeakerAccess")
        public static final String PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG = "production.exception.handler";

The below changes arise when we deprecate above 2 configs like above.

  1. Deprecate the below two configs in org.apache.kafka.streams.StreamsConfig
    1. default.deserialization.exception.handler
    2. default.production.exception.handler
  2. Configs to be added to org.apache.kafka.streams.StreamsConfig
    1. deserialization.exception.handler
    2. production.exception.handler
  3. Rename the below template configs
    1. DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_DOC to DESERIALIZATION_EXCEPTION_HANDLER_CLASS_DOC
    2. DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_DOC to PRODUCTION_EXCEPTION_HANDLER_CLASS_DOC
  4. Update all the relevant dependency classes, so that both old and new configs are supported
    1. Update ConfigDef in TopologyConfig and code blocks like isTopologyOverride(DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG, topologyOverrides)) with new configs, so that code behaves the same for both the old and new configs
    2. RecordDeserializer update handleDeserializationFailure method
  5. Mark the tests depending on old configs as deprecated and introduce new tests in the below
    1. InternalTopologyBuilderTest
    2. StreamTaskTest
    3. StreamThreadTest
  6. If user sets both old and new configs, a ConfigException to be thrown.

Compatibility, Deprecation, and Migration Plan

The below configs will be marked for deprecation, and new configs will be introduced, and in later further releases, they could be removed

Basically we have to make sure that after deprecation, the functionality for StreamsConfig, TopologyConfig and RecordDeserializer remains the same and should behave exactly same for old and new configs.

But if user sets both old and new configs, a ConfigException to be thrown.

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

The test plan for this change includes the following making sure functionality for StreamsConfig, TopologyCongfig and RecordDeserializer remains the same and should behave exactly same for old and new configs.

Rejected Alternatives

None