Versions Compared

Old Version 5

changes.mady.by.user Unknown User (schofielaj)

Saved on

compared with

New Version 6

changes.mady.by.user Unknown User (schofielaj)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Tidy up headers, configs, add metrics

...

Header nameDescription
__dlq.errors.topic 

The name of the topic that contained the original record.

__dlq.errors.partition 

The partition that contains the original record (encoded as a UTF-8 string).

__dlq.errors.offset 

The offset of the original record (encoded as a UTF-8 string).

__dlq.errors.group 

If known, the group ID of the delivery of the original record.

__dlq.errors.delivery.count 

If known, the delivery count of the original record (encoded as a UTF-8 string).

__dlq.errors.exception.class.name 

If known, the fully-qualified class name of the exception that was thrown during the final delivery attempt.

__dlq.errors.exception.message 

If known, the message of the exception that was thrown during the final delivery attempt.

The "__dlq.errors.exception." headers are defined for situations in which the cause of the failure could be identified and included in the DLQ record, such as a RecordDeserializationException .

Additionally, the broker can include the header, key and value of the original record in the DLQ record, but that takes additional configuration because copying the original record duplicates its content which may be undesirable in performance or security terms.

...

ConfigurationDescriptionValues
errors.deadletterqueue.auto.create.topics.enable Whether to create DLQ topics automatically. Note that automatic creation of the DLQ topics is independent of the auto.create.topics.enable  configuration.Type: boolean, default: false 
errors.deadletterqueue.copy.record.enable Whether to allow copying the original record onto the dead-letter queue topic for failed record delivery.Type: boolean, default: true errors.deadletterqueue.topic.name.prefix The prefix of permitted dead-letter queue topic names. If this is set to "", there is no restriction on the names used for dead-letter queue topics.Type: string, default: "dlq." 

...

ConfigurationDescriptionValues
errors.deadletterqueue.topic.name The name of the topic to be used as the dead-letter queue (DLQ) for undeliverable records. The topic name is blank by default, which means that there is no DLQ topic for this group.Type: string, default: ""
errors.deadletterqueue.copy.record.enable Whether to copy the record onto the dead-letter queue topic, or just to write a record containing the context information headers.Type: boolean, default: false 

Records

This KIP defines a new DeliveryState  of Archiving  with the value of 3 which indicates a record which is in the process of being written to the dead-letter queue topic. As the details of the implementation develop, it will become clear whether this value is used. The KIP reserves it for potential use in the existing share coordinator record schemas.

Metrics

Broker metrics

The following new broker metrics are added:

Metric NameTypeGroupTagsDescriptionJMX Bean
DeadLetterQueueRecordCountMeterShareGroupMetricsgroup: <group_id> The number of records written to the dead-letter queue topic.

kafka.server:type=ShareGroupMetrics,name=DeadLetterQueueRecordCount

kafka.server:type=ShareGroupMetrics,name=DeadLetterQueueRecordCount,group={group_id} 

DeadLetterQueueProduceRequestsPerSecMeterShareGroupMetricsgroup: <group_id> The rate of produce requests writing records to the dead-letter queue topic.

kafka.server:type=ShareGroupMetrics,name=DeadLetterQueueProduceRequestsPerSec

kafka.server:type=ShareGroupMetrics,name=DeadLetterQueueProduceRequestsPerSec,group={group_id} 

Compatibility, Deprecation, and Migration Plan

...