You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Status

Current state: Under Discussion

Discussion thread: TBD

JIRA:  Unable to render Jira issues macro, execution error.

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Kafka Producer supports transactional semantics since 0.11, including the following APIs:

  • InitTransaction for transactional producer identity initialization
  • beginTransaction to start a new transaction 
  • sendOffsetsToTransaction to commit consumer offsets advanced within the current transaction
  • commitTransaction commit the ongoing transaction
  • abortTransaction abort the ongoing transaction

Public Interfaces

We should add a new exception type called TransactionStateCorruptedException, which wraps non-fatal retriable producer txn exception.


TransactionStateCorruptedException.java
public class TransactionStateCorruptedException extends ApiException {

    public TransactionStateCorruptedException(String message, Throwable cause) {
        super(msg, cause);
    }
}

Proposed Changes

Below is a full list of exception types that could be thrown from producer API as of today, and we flagged those that should be thrown as fatal exception by themselves, vs exceptions that should be non-fatal: 

  1. ProducerFencedException (Fatal)
  2. InvalidProducerEpochException (Non-fatal)
  3. KafkaException, which potentially wraps the following exceptions:
    1. IllegalStateException
    2. InvalidPidMappingException
    3. TransactionAbortedException
    4. ClusterAuthorizationException
    5. UnknownProducerIdException
    6. TransactionalIdAuthorizationException
    7. UnsupportedVersionException
    8. AuthenticationException
    9. UnsupportedForMessageFormatException
    10. RuntimeException for detecting more than one inflight request
    11. InvalidRecordException
    12. InvalidRequiredAcksException
    13. NotEnoughReplicasAfterAppendException
    14. NotEnoughReplicasException
    15. RecordBatchTooLargeException
    16. InvalidTopicException
    17. CorruptRecordException
    18. UnknownTopicOrPartitionException
    19. NotLeaderOrFollowerException
    20. GroupAuthorizationException
    21. KafkaException
      1. indicates retriable idempotent sequence
      2. indicates fatal transactional sequence
      3. indicates Producer closed
      4. unknown reason
    22. TimeoutException for expired batch 

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?
  • If we are changing behavior how will we phase out the older behavior?
  • If we need special migration tools, describe them here.
  • When will we remove the existing behavior?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.

  • No labels