This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Draft

...

As per the Tiered Storage feature introduced in KIP-405, the follower fetch protocol was modified for topics enabled with tiered storage. For such topics, the empty follower finds the offset and leader epoch, up to which the auxiliary state (i.e. leader epoch sequence, producer snapshot state) needs to be built from the leader. The follower then starts fetching the data from the leader starting from that offset. In the KIP, it was decided to use the local-log-start-offset as the offset for this purpose. Instead of the local-log-start-offset, we could also last-tiered-offset which would help the followers quickly catch up with the leader as it only needs to fetch the segments that are not moved to tiered storage. The disadvantage of using the last-tiered-offset is that this new follower would possess only a limited number of locally stored segments. Should it ascend to the role of leader, there is a risk of needing to fetch these segments from the remote storage, potentially impacting broker performance.

...

Fetch remote segment info, and rebuild leader epoch sequence.

Step 2:

Continue fetching from the leader

Test Plan

Unit Tests and Integration Tests for the various follower fetch scenarios.