Page History

...

It may be possible that log segments to truncate be truncated are in tiered storage when unclean leader election is enabled for a partition. It is possible that segments offloaded to the tiered storage (which were eligible as their end offset < LSO) need to be truncated when leadership of a topic-partition moved to a previously lagging follower. The following diagram illustrates a basic scenario that shows a situation where a remote segment would require truncation based on to exhibit the same replica lineage as local segments.todo: We will update this digram with other scenarios in handling log truncation. 

Image Removed

Local log segments are truncated in such a situation. We may not want to fetch and update the remote segment which may add a lot of complexity in handling the followers which started reading that segment. One way to address this is to update the respective remote log segment metadata with truncation offsets.  

todo: We will update with the scenarios in handling log truncation. 

Performance Test Results

We have tested the performance of the initial implementation of this proposal.

...