Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Update "Producer Recoverable" to "Producer Abortable"

...

We would new error types which will be extended by existing exceptions mentioned in below table.

Code Block
// Producer-Recoverable Abortable Transaction
public class AbortableTransactionException extends ApiException {
    public AbortableTransactionException(String message) {
        super(message);
    }
    ...
}

//Producer- Retriable 
public class ProducerRetriableTransactionException extends ApiException {
    public ProducerRetriableTransactionException(String message) {
        super(message);
    }
 	...
}

//Producer- Refresh and Retriable
public class ProducerRefreshMetadataRetriableTransactionExceptionProducerRefreshRetriableTransactionException extends ApiException {
    public ProducerRefreshMetadataRetriableTransactionExceptionProducerRefreshRetriableTransactionException(String message) {
        super(message);
    }
	...
}

//Application-Recoverable
public class ApplicationRecoverableTransactionException extends ApiException {
    public ApplicationRecoverableTransactionException(String message) {
        super(message);
    }
    ...
}

// Invalid-Configuration
public class InvalidConfiguationTransactionException extends ApiException {
    public InvalidConfiguationTransactionException(String message) {
        super(message);
    }
 	...
}

// ExtendingExample of how exceptions will be extended by one of above exception types example
public class InvalidProducerEpochException extends AbortableTransactionException {
    private static final long serialVersionUID = 1L;
    public InvalidProducerEpochException(String message) {
        super(message);
    }
}

...

  1. Producer-Retriable: mostly maps onto current retriable errors. The producer can handle the retry on its own and the failure is invisible to the application. Some of these errors may indicate that a metadata update at the producer level is required. Because of that we introduce a few subclasses to the retriable error

    1. retry only (just send the request again – after some period of backoff)

    2. refresh metadata and retry (request metadata and maybe modify the request before resending)

  2. Producer-RecoverableAbortable: mostly maps onto abortable errors. The error is bubbled to the application layer, and it can choose to roll back any state from the ongoing transaction and abort the transaction. The producer does not need to restart, as after aborting, it can be confident that the state was as it was before the transaction started. The application can also close the producer and react as it does for application-recoverable cases, but it doesn’t need to.

  3. Application-Recoverable: maps on to some fatal errors. The error is bubbled to the application layer, and it may need to do a bit more to roll back and clean up state. The producer can not simply abort and know the current state of the partitions. Applications can handle this in different ways – streams may rebalance a task and/or close the task and restart it. Another application may read from their own checkpoint to continue. In any case, the producer must restart and will be unusable after encountering this error.

  4. Invalid-Configuration: maps to some fatal errors. The error is bubbled up to the application layer. The application can decide what to do. The producer doesn’t need to restart, but the application may chose to close it.

...

AbortableTransactionException

Exception/Error Names

Current handling

Expected

New Handling


Producer API

Transaction API

Producer API

Transaction API

TransactionAbortableException
** note – Added via KIP-890

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

TransactionAbortableException
** note – Added via KIP-890

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

Producer

Recoverable

Abortable

ProducerRetriableTransactionException

Exception/Error Names

Current handling

New Handling


Producer API

Transaction API

Producer API

Transaction API

CorruptRecordException

NotEnoughReplicasAfterAppendException

NotEnoughReplicasException

TimeoutException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Producer Retriable Image Modified

Producer Retriable Image Modified

ConcurrentTransactionsException

Retriable

Retriable

Producer Retriable (KIP-890 may see this)

Producer Retriable

CoordinatorLoadInProgressException

Retriable

Retriable

Producer Retriable

Producer Retriable

OutOfOrderSequenceException

Sometimes retriable or abortable

N/A

Producer Retriable Image Modified

N/A

...


ProducerRefreshRetriableTransactionException

Exception/Error Names

Current handling

New Handling


Producer API

Transaction API

Producer API

Transaction API

UnknownTopicOrPartitionException

NotLeaderOrFollowerException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Refresh + Retriable Image Modified

Refresh + Retriable Image Modified

NotCoordinatorException

CoordinatorNotAvailableException

Retriable

Retriable

Refresh + Retriable

Refresh + Retriable


ApplicationRecoverableTransactionException

Exception/Error Names

Current handling

Expected

New Handling


Producer API

Transaction API

Producer API

Transaction API

IllegalStateException

Abortable

Sometimes Fatal depending on whether application or Sender caused issue. See: kafka: KAFKA-14831: Illegal state errors should be fatal in transactional producerCLOSED

Application Recoverable (probably not expected) Image Modified

Application RecoverableImage Modified

UnknownProducerIdException

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Modified

Application Recoverable Image Modified

ProducerFencedException

Fatal

Fatal

Application Recoverable Image Modified

Application Recoverable Image Modified

InvalidProducerEpochException

Abortable

Fatal

Application Recoverable Image Modified

Application Recoverable Image Modified

ProducerFencedException

Fatal

Fatal

Application Recoverable Image Modified

Application Recoverable Image Modified

InvalidPidMappingException

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Modified

Application Recoverable Image Modified

UnknownProducerIdException

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Modified

Application Recoverable Image Modified

FencedInstanceIdException

CommitFailedException

UnknownMemberIdException

IllegalGenerationExceiption

N/A

Abortable (TxnOffsetCommit Only)

N/A

Application Recoverable Image Modified

KafkaException

Abortable (default seems to be abortable)

Fatal in most cases, but abortable when there are partition errors

Application Recoverable (not expected) Image Modified

Application Recoverable (not expected) Image Modified

RuntimeException

Abortable (default seems to be abortable)

Fatal – only thrown as this generic type when correlation ID is wrong. This should be updated as KIP-691 suggests

Application Recoverable (not expected) Image Modified

Application Recoverable (not expected) Image Modified

CorrelationIdMismatchException



Application Recoverable

Application Recoverable


InvalidConfiguationTransactionException

...

InvalidTxnStateException requires special handling where producer API and Transactional API has different handling:

Exception/Error Names

Current handling

Expected Handling


Producer API

Transaction API

Producer API

Transaction API

InvalidTxnStateException

Abortable

Fatal

Producer

Recoverable

Abortable (KIP-890 relies on this – note this is the only one that differs with Produce API) Image Modified

Application Recoverable Image Modified


Compatibility, Deprecation, and Migration Plan

...