Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
titleProducerConfig
.
.
.

public static final String CUSTOM_EXCEPTION_HANDLER_CLASS_CONFIG = "custom.exception.handler";
private static final String CUSTOM_EXCEPTION_HANDLER_CLASS_DOC = "Exception handling class that implements the <code>org.apache.kafka.common.errors.ProducerExceptionHandler</code> interface.";.
.
.
static {
public static final String DROP_INVALID_LARGE_RECORDS_CONFIG = new ConfigDef().define(
.....
.
.
.
"drop.invalid.large.records";
private static final String DROP_INVALID_LARGE_RECORDS_DOC = "Exception handling class that implements the <code>org.apache.kafka.common.errors.ProducerExceptionHandler</code> interface.";.

public static final String RETRY_UNKNOWN_TOPIC_PARTITION_CONFIG = "retry.unknown.topic.partition";
private static final String RETRY_UNKNOWN_TOPIC_PARTITION_DOC = "Exception handling class that implements the <code>org.apache.kafka.common.errors.ProducerExceptionHandler</code> interface.";.
. .
.
static {
CONFIG = new ConfigDef().define(
.....
.
.
.
                         .define(CUSTOM_EXCEPTION_HANDLER_CLASS_CONFIG,
                                 Type.CLASS,
                                 null,
                                 Importance.MEDIUM,
                                 CUSTOM_EXCEPTION_HANDLER_CLASS_DOC)
                         .define(DROP_INVALID_LARGE_RECORDS_CONFIG,
                                 Type.CLASS,
                                 false,
                                 Importance.MEDIUM,
                                 DROP_INVALID_LARGE_RECORDS_DOC)
                        .define(CUSTOMRETRY_EXCEPTIONUNKNOWN_HANDLERTOPIC_CLASSPARTITION_CONFIG,
                                 Type.CLASS,
                                  null true,
                                 Importance.MEDIUM,
                                  CUSTOMRETRY_EXCEPTIONUNKNOWN_HANDLERTOPIC_CLASSPARTITION_DOC);
 }


Proposed Changes

The RecordTooLargeException can be thrown by broker, producer and consumer. Of course, the ProducerExceptionHandler interface is introduced to affect ONLY the exceptions thrown from the producer.

With the changes made
here (because of

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-9279
) the producer send() method throws a RecordTooLargeException facing too large records in transactions. The user can bring the custom handler to bear to avoid the entire batch failing by dropping the poisoning too large record (SWALLOW the error). This way, this record does not get included in the batch.

...