Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Motivation

Currently, the error handling mechanism in Kafka Connect supports only predefined error reporter implementations from the runtime module. The `org.apache.kafka.connect.runtime.errors.ErrorReporter` interface is located in the runtime module, which restricts the ability to define custom error reporting mechanisms.  Current error reporter implementations: 

...

In addition to these implementations, custom error reporters can be beneficial for implementing custom logic to report errors to specific storage systems, as well as defining custom metrics and logs based on the error context. This can be particularly useful for source connectors, where the log reporter is the only option to understand the root cause of failures. This proposal aims to introduce the capability to define custom error reporting functionality.

Public Interfaces

We propose the following new public interface:

...

Code Block
languagetext
errors.reporters=example

errors.reporters.example.type=com.example.SimpleErrorRecordReporter

errors.reporters.example.param=testValue

Proposed Changes

We propose a new pluggable interface (`ErrorRecordReporter`) which enables the creation of custom error record reporters. These custom reporters will be executed when `errors.tolerance=all`. The draft pull request: https://github.com/apache/kafka/pull/17493

Compatibility, Deprecation, and Migration Plan

There are no backward compatibility concerns.

Rejected Alternatives

1. Exposing `ErrorReporter` and `ProcessingContext` as public APIs: The main issue with this design is that it exposes packages with `runtime` in their names, which cannot be changed.
2. Naming as ErrorReporter: This name is already used for error reporters in the runtime module. To avoid name conflicts, `ErrorRecordReporter` was chosen.