Versions Compared

Old Version 45

changes.mady.by.user Kaushik Raina

Saved on

compared with

New Version Current

changes.mady.by.user Kaushik Raina

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Status

Current state: Under Voting  Complete 

Discussion thread: here

JIRA: here 

...

Cross → Exception handling doesn't match as mentioned in KIP-691

RetriableException

Exception/Error Names

Current handling

New Handling


Producer API

Transaction API

Producer API

Transaction API

Comments

CorruptRecordException

NotEnoughReplicasAfterAppendException

NotEnoughReplicasException

TimeoutException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Producer Retriable Image Modified

Producer Retriable Image Modified


ConcurrentTransactionsException

Retriable

Retriable

Producer Retriable (KIP-890 may see this)

Producer Retriable


CoordinatorLoadInProgressException

Retriable

Retriable

Producer Retriable

Producer Retriable


RefreshRetriableException

Exception/Error Names

Current handling

New Handling


Producer API

Transaction API

Producer API

Transaction API

Comments

UnknownTopicOrPartitionException

NotLeaderOrFollowerException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Refresh + Retriable Image Modified

Refresh + Retriable Image Modified

Both UnknownTopicOrPartitionException and NotLeaderOrFollowerException  extends

AuthorizationException 

InvalidMetadataException 


Current class hierarchy in code is:

UnknownTopicOrPartitionException <

InvalidMetadataException < RetriableException < ApiException



New class hierarchy in code will be:

UnknownTopicOrPartitionException < 

InvalidMetadataException < RefreshRetriableException < RetriableException < ApiException

NotCoordinatorException

CoordinatorNotAvailableException

Retriable

Retriable

Refresh + Retriable

Refresh + Retriable


TransactionAbortableException

Exception/Error Names

Current handling

New Handling



Producer API

Transaction API

Producer API

Transaction API

Comments

TransactionAbortableException
** note – Added via KIP-890

Producer Abortable

Producer Abortable

Producer Abortable

Producer Abortable


ApplicationRecoverableException

Exception/Error Names

Current handling

New Handling



Producer API

Transaction API

Producer API

Transaction API

Comments

IllegalStateException

Abortable

Sometimes Fatal depending on whether application or Sender caused issue. See: kafka: KAFKA-14831: Illegal state errors should be fatal in transactional producerCLOSED

Application Recoverable (probably not expected) Image Modified

Application RecoverableImage Modified


We will handle all default exceptions as generic unknown errors, which will be application recoverable


KafkaException

Abortable (default seems to be abortable)

Fatal in most cases, but abortable when there are partition errors

Application Recoverable (not expected) Image Modified

Application Recoverable (not expected) Image Modified

RuntimeException

Abortable (default seems to be abortable)

Fatal – only thrown as this generic type when correlation ID is wrong. This should be updated as KIP-691 suggests

Application Recoverable (not expected) Image Modified

Application Recoverable (not expected) Image Modified

InvalidProducerEpochException

Abortable

Fatal

Application Recoverable Image Modified

Application Recoverable Image Modified


ProducerFencedException

Fatal

Fatal

Application Recoverable Image Modified

Application Recoverable Image Modified


InvalidPidMappingException

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Modified

Application Recoverable Image Modified


FencedInstanceIdException

CommitFailedException

UnknownMemberIdException

IllegalGenerationExceiption

N/A

Abortable (TxnOffsetCommit Only)

N/A

Application Recoverable Image Modified

CommitFailedException extends KafkaException, which is default exception handled as Application Recoverable. This class will not be extended by ApplicationRecoverableException

CorrelationIdMismatchException



Application Recoverable

Application Recoverable

CorrelationIdMismatchException extends IllegalStateException, which is default exception handled as Application Recoverable. This class will not be extended by ApplicationRecoverableException

InvalidTxnStateException

Abortable

Fatal

Producer Abortable (KIP-890 relies on this – note this is the only one that differs with Produce API) Image Modified

Application Recoverable Image Modified

InvalidTxnStateException requires special handling where producer API and Transactional API has different handling


InvalidConfigurationException

Exception/Error Names

Current handling

Expected Handling


Producer API

Transaction API

Producer API

Transaction API

Comments

AuthenticationException

Abortable

Fatal

Invalid Configuration Image Modified

Invalid Configuration (not expected) Image Modified


ClusterAuthorizationException

TransactionalIdAuthorizationException

UnsupportedVersionException

UnsupportedForMessageFormatException

Fatal, except UnsupportedForMessageFormatException which is Abortable

Cluster/Transaction Auth → abortable on InitProducerId, other errors abortable

TransactionAuth → fatal, others abortable for AddPartitions, Find Coordinator, EndTxn, AddOffsets

Transaction Auth, UnssupportedForMessageFormat → fatal for OffsetCommit

Invalid Configuration Image Modified

Invalid Configuration Image Modified


InvalidRecordException

InvalidRequiredAcksException

RecordBatchTooLargeException

InvalidTopicException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Invalid Configuration Image Modified

Invalid Configuration Image Modified


TopicAuthorizationException

GroupAuthorizationException

Abortable

Abortable

Invalid Configuration Image Modified

Invalid Configuration Image Modified

Both TopicAuthorizationException and GroupAuthorizationException extends AuthorizationException 


Current class hierarchy in code is:

TopicAuthorizationException < AuthorizationException < ApiException

New class hierarchy in code will be TopicAuthorizationException < AuthorizationException < InvalidConfigurationException < ApiException`


Public Interfaces

We have below exception classes available in current Kafka code

...

Code Block
//Producer Refresh and Retriable
public abstract class RefreshRetriableException extends RetriableException {
	...
}

//Application-Recoverable new
public abstract class ApplicationRecoverableException extends ApiException {
    ...
}


We will extend below existing exceptions types to maintain class hierarchy 

Code Block
// UnknownTopicOrPartitionException and NotLeaderOrFollowerException  extends InvalidMetadataException 
public class InvalidMetadataException extends RefreshRetriableException {
	...
}

// TopicAuthorizationException and GroupAuthorizationException extends AuthorizationException 
public class AuthorizationException extends InvalidConfigurationException {
	...
}


Client side code example

...

Currently, the transactional producer.send returns retriable exception types, such as TimeoutException , which poses a risk of duplicates in Kafka. In this KIP, we will update the transactional producer.send path such that all retriable exceptions will be translated to TransactionAbortableException in transaction producer code path. Older clients that are using the transactional producer and handling TimeoutException  by retrying the produce operation can update to handle TransactionAbortableException .

...