Page History

...

For follower fetch, the leader only returns the data that is still in the leader's local storage. LogSegments that exist only on remote storage are not replicated to followers as it is already present in remote storage. Instead, a follower will retrieve the information of the remote segment from RemoteMetadataManager. If a Replica becomes a leader, It can still locate and serve data from remote storage.We will discuss different cases of fetch offsets.

Followers fetch offsets and truncate their local logs if needed with the current mechanism.

Leaders will send a new error code `OFFSET_MOVED_TO_TIERED_STORAGE` to followers if the fetch offset is not available in the local log. In this case, the follower needs to get leader epochs (possibly producer-snapshot-ids too). This can be done in two ways.

• introduce new protocol (or API) to fetch these from leader partition.
• fetch these from remote storage. 

Latter is preferred here as remote storage can have this data and it is simpler without introducing a new protocol with the leader.  

This involves two steps in getting the required state of the respective log segment for the requested fetch offset.

• it should fetch the respective remote log segment metadata and
• it should fetch respective state like leader epochs from remote storage for the respective remote log segment metadata. 

So, we need to add respective ReplicaStates for both which can be called `FetchingRemoteLogMetadata` and `FetchingRemoteLogAuxiliaryState`. 

Consumer Fetch Requests

For any fetch requests, ReplicaManager will proceed with making a call to readFromLocalLog, if this method returns OffsetOutOfRange exception it will delegate the read call to RemoteLogManager. More details are explained in the RLM/RSM tasks section.

...