Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Code Block
titlePurgeRequest
PurgeRequest => topics
  topics => [PurgeRequestTopic]
  timeout => int32
 
PurgeRequestTopic => topic partitions
  topic => str
  partitions => [PurgeRequestPartition]
 
PurgeRequestPartition => partition offset
  partition => int32
  offset => int64  // offset -1L will be translated into high_watermark of the partition when leader receives the request.

 

Create PurgeReponse

 

Code Block
titlePurgeReponse
PurgeReponse => topics
  topics => [PurgeResponseTopic]
 
PurgeResponseTopic => topic partitions
  topic => str
  partitions => [PurgeResponsePartition]
 
PurgeResponsePartition => partition low_watermark error_code
  partition => int32
  low_watermark => int32
  error_code => int16

 

...

2) Admin Client builds PurgeRequest using the offsetToPurge from purgeDataBefore() parameter and the requestTimeoutMs is taken from the AdminClient constructor. OnePurgeRequest is sent to each broker that acts as leader of any partition in the request. The request should only include partitions which the broker leads.

3) After receiving the PurgeRequest, for each partition in the PurgeRequest, the leader sets  the leader first sets offsetToPurge to high_watermark if offsetToPurge is -1L. It then sets low_watermark of leader replica to min(leader's logEndOffset, to max(low_watermarkoffsetToPurge) ) for each partition in the PurgeRequestif offsetToPurge <= high_watermark. Those segments whose largest offset < low_watermark will watermark will be deleted by the leader.

...

  • If low_watermark of this partition on all followers is larger than or equal to the offsetToPurge, the result of this partition will be the low_watermark of the leader replica.
  • If logEndOffset high_watermark of this partition is smaller than the offsetToPurge, the result of this partition will be OffsetOutOfRangeException.
  • If any replica of this partition goes offline, the result of this partition will be NotEnoughReplicasException
  • If the leadership of this partition moves to another broker, the result of this partition will be NotLeaderException
  • If the result of this partition is not available after RequestTimeoutMs, the result of this partition will be TimeoutException

...