Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: temporary changes to include DLQ.

...

For sink connectors, we will write the original record (from the Kafka topic the sink connector is consuming from) which caused the failure to a configurable Kafka topic.

Config OptionDescriptionDefault ValueDomain
errors.deadletterqueue.topic.nameThe name of the dead letter queue topic. If not set, this feature will be disabled.""A valid Kafka topic name
errors.deadletterqueue.context.headers.enableIf true, multiple headers will be added to annotate the record with the error contextfalseBoolean

If the property errors.deadletterqueue.context.headers.enable is set to true, the following headers will be added to the produced raw message (only if they don't already exist in the message). All values will be Strings.

Header NameDescription
__connect.errors.topicName of the topic that contained the message.
__connect.errors.partitionThe original partition that contained the message (encoded as a String).
__connect.errors.offsetThe original offset that contained this message (encoded as a String).
__connect.errors.connector.nameThe name of the connector which encountered the error.
__connect.errors.task.idThe id of the task which encountered the error (encoded as a String).
__connect.errors.stageThe name of the stage where the error occurred.
__connect.errors.class.nameThe fully qualified name of the class that caused the error.
__connect.errors.exception.class.nameThe fully qualified classname of the exception that was thrown during the execution.
__connect.errors.exception.messageThe message in the exception.
__connect.errors.exception.stacktraceThe stacktrace of the exception.

Metrics

The following new metrics will monitor the number of failures, and the behavior of the response handler. Specifically, the following set of counters:

...