DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Motivation
Currently, the error handling mechanism in Kafka Connect supports only predefined error reporter implementations from the runtime module. The `org.apache.kafka.connect.runtime.errors.ErrorReporter` interface is located in the runtime module, which restricts the ability to define custom error reporting mechanisms. Current error reporter implementations:
...
In addition to these implementations, custom error reporters can be beneficial for implementing custom logic to report errors to specific storage systems, as well as defining custom metrics and logs based on the error context. This can be particularly useful for source connectors, where the log reporter is the only option to understand the root cause of failures. This proposal aims to introduce the capability to define custom error reporting functionality.
Public Interfaces
We propose the following new public interface:
...
| Code Block | ||
|---|---|---|
| ||
errors.reporters=example errors.reporters.example.type=com.example.SimpleErrorRecordReporter errors.reporters.example.param=testValue |
Proposed Changes
We propose a new pluggable interface (`ErrorRecordReporter`) which enables the creation of custom error record reporters. These custom reporters will be executed when `errors.tolerance=all`. The draft pull request: https://github.com/apache/kafka/pull/17493
Compatibility, Deprecation, and Migration Plan
There are no backward compatibility concerns.
Rejected Alternatives
1. Exposing `ErrorReporter` and `ProcessingContext` as public APIs: The main issue with this design is that it exposes packages with `runtime` in their names, which cannot be changed.
2. Naming as ErrorReporter: This name is already used for error reporters in the runtime module. To avoid name conflicts, `ErrorRecordReporter` was chosen.