Page History

...

Discussion thread: Here

JIRA: HereTBD

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

As stated in rejected alternatives, ”Write records that fail in the put() step of a sink connector to the dead letter queue: since sink connectors can chose to batch records in a put() method, it is not clear what errors are caused by what records (they might be because of records that were immediately written to put(), or by some previous records that were processed later). Also, there might be connection issues that are not handled by the connector, and simply bubbled up as IOException (for example). Effectively, errors sent back to the framework from the put() method currently do not have sufficient context to determine the problematic records (if any). Addressing these issues would need a separate KIP. “Beyond this, we currently have a proprietary connect-reporter library with a Reporter class. However, beyond the fact that we cannot use the proprietary library in the public sink connectors, using this library adds redundancy to configuration, as a user would have to configure this library in addition to the configuration properties already provided by the KIP-298 error handling functionality. 

Thus, this proposal aims to extend KIP-298 and add error reporting functionality within the context of put(...) without adding redundancy in configuration.

...