Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We aim at enabling users to ignore the `send(ProducerRecord)` errors by adding an input parameter to the `send` method so that the user is able to determine not going to the error state by a poison pill record. 
The following list categorizes all types of errors that cause a transaction to fail. The category with a
(tick) beside specifies the cases that the new `send` API is going to cover. 

  • producer-side errors
    • recoverable, such as `RecordTooLargeException` (tick) 
    • irrecoverable, such as `ProducerFencedException` (error)
  • broker-side errors (error)

Currently, producer-side recoverable errors (the KIP's target category) prevent a record from being added to a batch. They additionally make a transition to an `error` state, which ends up transaction to fail. This KIP provides the possibility to commit a transaction successfully in presence of such errors. Obviously, the problematic records are not added to the batch, but the transition to the `error` state is not done.

Public Interfaces

If the user 1) is performing a transaction and 2) passes the `TxnSendOption` with the value `IGNORE_SEND_ERRORS` to the `send` method, any poison pill record is excluded from the batch, and the transaction is committed successfully. Note that if the user sets the `TxnSendOption` to `IGNORE_SEND_ERRORS` outside of a transaction, the overloaded `send` method , throws an `IllegalStateException`.


Code Block
languagejava
titleKafkaProducer
     public Future<RecordMetadata> send(ProducerRecord<K, V> record, Callback callback, TxnSendOption option) {}


     public enum TxnSendOption {
        /**
         * The irrecoverable {@link #send(ProducerRecord)} errors lead the transaction to the error state, which ends up an unsuccessful commit. 
         */
        NONE,
        /**
         * The records causing irrecoverable errors are excluded from the batch and the transaction is committed successfully. 	
         * Note to use this, only in transactions. Otherwise {@link #send(ProducerRecord, Callback, TxnSendOption)} throws exception.
         */
        IGNORE_SEND_ERRORS
    }

...