Status

Current state: Under Discussion

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here

Motivation

Currently there are two error handling options in Kafka Connect, “none” and “all”. Option “none” will config the connector to fail fast, and option “all” will ignore broken records.

...

Therefore, it would be idea if Kafka Connect provides an additional option that sends the broken raw bytes to SinkTask directly. 

Public Interfaces

In Kafka Connect, the configuration

...

will have a third option "continue" besides "none" and "all"

Proposed Changes

Add a third option to error handling, which should behave like “continue” when error occurs at Converter or SMT. The infrastructure should send the broken byte message directly to SinkTask.

...

• Being user friendly. Connectors can handle broken record and hide that from clients.
• Providing more flexibility to SinkTask in terms of broken record handling.

Compatibility, Deprecation, and Migration Plan

There is no compatibility issue. Behavior of Kafka Connect does not change unless user explicitly specify the error handling method to be "continue". 

Rejected Alternatives

None