Table of Contents

Status

Current state: Under Voting Complete

Discussion thread: here

JIRA: here

...

Cross → Exception handling doesn't match as mentioned in KIP-691

RetriableException

Exception/Error Names	Current handling		New Handling
	Producer API	Transaction API	Producer API	Transaction API	Comments
CorruptRecordException NotEnoughReplicasAfterAppendException NotEnoughReplicasException TimeoutException	Retriable if the error is retriable. Otherwise abortable	Retriable if the error is retriable otherwise abortable	Producer Retriable Image Modified	Producer Retriable Image Modified
ConcurrentTransactionsException	Retriable	Retriable	Producer Retriable (KIP-890 may see this)	Producer Retriable
CoordinatorLoadInProgressException	Retriable	Retriable	Producer Retriable	Producer Retriable

RefreshRetriableException

Exception/Error Names

Current handling

New Handling

Producer API

Transaction API

Producer API

Transaction API

Comments

UnknownTopicOrPartitionException

NotLeaderOrFollowerException

Retriable if the error is retriable. Otherwise abortable

Retriable if the error is retriable otherwise abortable

Refresh + Retriable Image Modified

Both UnknownTopicOrPartitionException and NotLeaderOrFollowerException extends InvalidMetadataException

Current class hierarchy in code is:

InvalidMetadataException < RetriableException < ApiException

New class hierarchy in code will be:

InvalidMetadataException < RefreshRetriableException < RetriableException < ApiException

NotCoordinatorException

CoordinatorNotAvailableException

Retriable

Refresh + Retriable

TransactionAbortableException

Exception/Error Names	Current handling		New Handling
	Producer API	Transaction API	Producer API	Transaction API	Comments
TransactionAbortableException ** note – Added via KIP-890	Producer Abortable	Producer Abortable	Producer Abortable	Producer Abortable

ApplicationRecoverableException

Exception/Error Names	Current handling		New Handling
	Producer API	Transaction API	Producer API	Transaction API	Comments

InvalidProducerEpochException

IllegalStateException

Abortable

Fatal

Application Recoverable

Image Removed

Sometimes Fatal depending on whether application or Sender caused issue. See: kafka: KAFKA-14831: Illegal state errors should be fatal in transactional producerCLOSED

Application Recoverable

Image Removed

ProducerFencedException

Fatal

(probably not expected) Image Added

Application RecoverableImage Modified

Application Recoverable Image Removed

InvalidPidMappingException

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Removed

FencedInstanceIdException

CommitFailedException

UnknownMemberIdException

IllegalGenerationExceiption

N/A

Abortable (TxnOffsetCommit Only)

N/A

Application Recoverable Image Removed

CorrelationIdMismatchException

Application Recoverable

We will handle all default exceptions as generic unknown errors, which will be application recoverable. Below are few such exceptions:

Exception/Error Names

Current handling

New Handling

Producer API

Transaction API

Producer API

Transaction API

IllegalStateException

Abortable

Sometimes Fatal depending on whether application or Sender caused issue. See: kafka: KAFKA-14831: Illegal state errors should be fatal in transactional producerCLOSED

Application Recoverable (probably not expected)

We will handle all default exceptions as generic unknown errors, which will be application recoverable
	KafkaException	Abortable (default seems to be abortable)	Fatal in most cases, but abortable when there are partition errors	Application Recoverable (not expected) Image Added	Application Recoverable (not expected) Image Added
	RuntimeException	Abortable (default seems to be abortable)	Fatal – only thrown as this generic type when correlation ID is wrong. This should be updated as KIP-691 suggests	Application Recoverable (not expected) Image Added	Application Recoverable (not expected) Image Added
InvalidProducerEpochException	Abortable	Fatal	Application Recoverable Image Added	Application Recoverable Image Added
ProducerFencedException	Fatal	Fatal	Application Recoverable

Image Modified

Application Recoverable Image Modified

KafkaException

InvalidPidMappingException

Abortable (default seems to be abortable)

Fatal in most cases, but abortable when there are partition errors

Application Recoverable (not expected)

Sometimes retriable (no longer returned) or abortable

Abortable

Application Recoverable Image Modified

Application Recoverable

(not expected)

Image RemovedRuntimeException

Exception/Error Names

Current handling

Expected Handling

Producer API

Transaction API

Producer API

Transaction API

Image Added

FencedInstanceIdException

CommitFailedException

UnknownMemberIdException

IllegalGenerationExceiption

N/A

Abortable (

default seems to be abortable)

Fatal – only thrown as this generic type when correlation ID is wrong. This should be updated as KIP-691 suggests

Application Recoverable (not expected) Image Removed

InvalidTxnStateException requires special handling where producer API and Transactional API has different handling:

TxnOffsetCommit Only)	N/A	Application Recoverable Image Added	CommitFailedException extends KafkaException, which is default exception handled as Application Recoverable. This class will not be extended by ApplicationRecoverableException
CorrelationIdMismatchException			Application Recoverable	Application Recoverable	CorrelationIdMismatchException extends IllegalStateException, which is default exception handled as Application Recoverable. This class will not be extended by ApplicationRecoverableException

InvalidTxnStateException

Abortable

Fatal

Producer Abortable (KIP-890 relies on this – note this is the only one that differs with Produce API) Image Modified

Application Recoverable Image Modified

InvalidTxnStateException requires special handling where producer API and Transactional API has different handling

InvalidConfigurationException

Exception/Error Names	Current handling		Expected Handling
	Producer API	Transaction API	Producer API	Transaction API	Comments
AuthenticationException	Abortable	Fatal	Invalid Configuration Image Modified	Invalid Configuration (not expected) Image Modified
ClusterAuthorizationException TransactionalIdAuthorizationException UnsupportedVersionException UnsupportedForMessageFormatException	Fatal, except UnsupportedForMessageFormatException which is Abortable	Cluster/Transaction Auth → abortable on InitProducerId, other errors abortable TransactionAuth → fatal, others abortable for AddPartitions, Find Coordinator, EndTxn, AddOffsets Transaction Auth, UnssupportedForMessageFormat → fatal for OffsetCommit	Invalid Configuration Image Modified	Invalid Configuration Image Modified
InvalidRecordException InvalidRequiredAcksException RecordBatchTooLargeException InvalidTopicException	Retriable if the error is retriable. Otherwise abortable	Retriable if the error is retriable otherwise abortable	Invalid Configuration Image Modified	Invalid Configuration Image Modified
TopicAuthorizationException

GroupAuthorizationException

Abortable

Invalid Configuration Image Removed

GroupAuthorizationException

Abortable

Invalid Configuration Image Added

Both TopicAuthorizationException and GroupAuthorizationException extends AuthorizationException

Current class hierarchy in code is:

TopicAuthorizationException < AuthorizationException < ApiException

New class hierarchy in code will be TopicAuthorizationException < AuthorizationException < InvalidConfigurationException < ApiException`

Public Interfaces

We have below exception classes available in current Kafka code

Code Block

// Producer Abortable Transaction
public class TransactionAbortableException extends ApiException {
    ...
}

// Producer Abortable TransactionProducer Retriable  
public abstract class RetriableException extends ApiException {
   ...
}


// Invalid-Configuration
public class TransactionAbortableExceptionInvalidConfigurationException extends ApiException {
    ...
}

//Producer Retriable  
 	...
}

We will add new exception types, as listed in the table below, that extend the existing exceptions.

Code Block

//Producer Refresh and Retriable
public abstract class RetriableExceptionRefreshRetriableException extends ApiExceptionRetriableException {
   	...
}


// Invalid-ConfigurationApplication-Recoverable new
public abstract class InvalidConfigurationExceptionApplicationRecoverableException extends ApiException {
   	 ...
}

We will add new exception types, as listed in the table below, that extend the existing exceptions.extend below existing exceptions types to maintain class hierarchy

Code Block

//Producer RefreshUnknownTopicOrPartitionException and Retriable NotLeaderOrFollowerException  extends InvalidMetadataException 
public abstract class RefreshRetriableExceptionInvalidMetadataException extends RetriableExceptionRefreshRetriableException {
	...
}

//Application-Recoverable new
public abstract TopicAuthorizationException and GroupAuthorizationException extends AuthorizationException 
public class ApplicationRecoverableExceptionAuthorizationException extends ApiExceptionInvalidConfigurationException {
    	...
}

Client side code example

...

Note: The producer.abortTransaction() method should will not throw an abortable exception to avoid creating a loop.

...

Currently, the transactional producer.send returns retriable exception types, such as TimeoutException , which poses a risk of duplicates in Kafka. In this KIP, we will update the transactional producer.send path such that all retriable exceptions will be translated to TransactionAbortableException in transaction producer code path. Older clients that are using the transactional producer and handling TimeoutException by retrying the produce operation can update to handle TransactionAbortableException .

...

Space shortcuts

Child pages

Versions Compared

Old Version 43

New Version Current

Key

Status

RetriableException

RefreshRetriableException

TransactionAbortableException

ApplicationRecoverableException

InvalidConfigurationException

Public Interfaces

Client side code example

Space shortcuts

Child pages

Page History

Versions Compared

Old Version 43

New Version Current

Key

Status

RetriableException

RefreshRetriableException

TransactionAbortableException

ApplicationRecoverableException

InvalidConfigurationException

Public Interfaces

Client side code example