...
This KIP is aimed at improving the error-handling semantics in Kafka Streams when Kafka Steams fails to serialize a message to the downstream sink by providing an interface that can provide custom messaging of the error (e.g. report to a custom metrics system) and indicate to Streams whether or not it should re-throw the Exception, thus causing the application to fall over.
Status
Current state: inactive Accepted
Discussion thread: here
JIRA: KAFKA-7499
...
To accept different types of records from multiple topologies, ProducerRecord
is defined without generics. The above interface method will have a default
implementation which returns ProductionExceptionHandlerResponse.FAIL
Proposed Proposed Changes
This implementation will override the new method, handleSerializationException, in the following class:
...
We'll implement the following error handling logic to the send in RecordCollectorImpl. The new method, handleSerializationException
, in ProductionExceptionHandler will not be invoked whenfor
ClassCastException
is thrown while serializing record key / value. Today, we are throwingStreamsException
on hitting this exception. Whether to throw the
exception to the user will be decided based on the response received from ProductionExceptionHandlerhandleSerializationException
method.- If the result is
CONTINUE
, log a note atDEBUG
that we received that result and are not failing Streams as a result. If the result is
FAIL
, log a message at ERROR
that we received that result and throw StreamsException
so Streams will fail.We will continue to throw this exception and not invoke the new method. This will allow the current behavior to continue as this can help identify misconfigured serdes It will be invoked for
- Any other unchecked exceptions, that thrown during record key / value serialization.
- If the result is
CONTINUE
, log a note atDEBUG
WARN
that we received that result and are not failing Streams as a result. - If the result is
FAIL
, log a message atERROR
that we received that result and setsendException
so Streams will fail.
- If the result is
...