Page History

...

We will introduce a new exception type, which will be raised from KafkaConsumer.poll(Duration) as described above. This exception extends from `OffsetOutOfRangeException` for compatibility. If a user's offset reset policy is set to "none," they will still be able to catch `OffsetOutOfRangeException`. For new usage, users can catch the more specific exception type and use the new `seekToNearest()` API defined below.`seek()` API to resume consumption. This exception is raised by the consumer after using the OffsetForLeaderEpoch API to find the offset of divergence. This offset is included as a field in the exception. Typically users handling this exception will seek to this offset.

/**
 * In the even of unclean leader election, the log will be truncated,
 * previously committed data will be lost, and new data will be written
 * over these offsets. When this happens, the consumer will detect the 
 * truncation and raise this exception (if no automatic reset policy 
 * has been defined) with the first offset to diverge from what the 
 * consumer read.
 */ 
class LogTruncationException extends OffsetOutOfRangeException {
  /**
   * Get the truncation offset if one could be found. This is the first
   * offset which is known to diverge from what the consumer read.
   */	
  Optional<OffsetAndMetadata> truncationOffset();

class LogTruncationException extends OffsetOutOfRangeException {}

We will also add a new retriable error code for the UNKNOWN_LEADER_EPOCH error code:

...

If the current leader epoch does not match that of the leader, then we will send either FENCED_REPLICA or UNKNOWN_LEADER_EPOCH as we do for the Fetch API.Note that consumers will send a sentinel value (-1) for the current epoch and the broker will simply disregard that validation. 

Metadata

The metadata response will be extended to include the leader epoch. This enables stronger fencing for consumers. We can enable similar protection in producers in the future, but that is out of the scope of this KIP.

...