Page History

...

1. Unlike log index file, the time index file should not be rebuilt from local log when it crashes, but should always be fetched from the current leader, just the same as actual data. Otherwise we may have different time index on different replicas.
2. To ensure the log segments are identical on both leader and followers, we should always have a time index entry for the first message in a log segment.
3. In order to make the time based log retention work, we need the timestamp entry for the last message in a log segment.
4. When we truncate the messages in log segment files, we need to truncate entries in the time index files as well.

On One way to achieve the above goals is to treat the time index as a special partition that associated with the actual partition. When the leader receives a fetch request from the follower, it will include the data from this companion partition. More specifically, we may do the following:

...