Versions Compared

Old Version 11

changes.mady.by.user Ben Stopford

Saved on

compared with

New Version 12

changes.mady.by.user Ben Stopford

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Changes added in response to Jun's feedback

...

To protect against this eventuality, if the controller receives a Broker Registration it will increment the Leader Epoch and propagate that value to the broker via the Leader and ISR Request (in the normal way). This mechanism must will also be sensitive to affected by the bug: 
Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-1120
 which is not addressed directly in the initial version of this KIP. 

Maintaining the LeaderEpochSequenceFile

...

After an unclean shutdown the Leader Epoch Sequence File could be ahead of the log (as the log is flushed asynchronously by default). Thus, if an unclean shutdown is detected on broker start (regardless of whether it's a leader or follower), all entries are removed from the Leader Epoch Sequence File, where the offset is greater than the log end offset.

Public Interfaces

Add API for

...

OffsetForLeaderEpochRequest/Response

Offset For Leader Epoch Request V0

LeaderEpochRequest OffsetForLeaderEpochRequest (Version: 0) => [partition_leader_epoch] topics] 

topics => topic [partitions] 

topic => string

partitions
 partition_leader_epoch => partition_id, leader_epoch

   partition     partition_id => INT32

   leader    leader_epoch => INT32

Offset For Leader Epoch Response V0

LeaderEpoch Response OffsetForLeaderEpochResponse (Version: 0) => [topics]

topics => topic [epoch_and_offset]partitions] 

topic => string

partitions
 epoch_and_offset => error_id, partition_id, leader_epoch, start_offset

   error    error_id => Int16

   leader_epoch => INT32

   partition  partition_id => INT32

   start    start_offset => INT64

Error Codes:

-1 -> Unknown Error

0 -> No Error

1 -> Not leader for partition

2 -> No epoch information for partition

...

MessageSet

MessageSet => [Offset MessageSize Message]

 Offset => int64

 MessageSize => int32

 leader_epoch => int32 <--------[NEW]

We bump up ProduceRequest/FetchRequest (and responses) versions to indicate the broker that this client supports new message format.

Add Leader Epoch Sequence File per Partition

...