...

Broker will do the followings during migration for log retention and searching by timestamp:

1. The broker will create a time index for each segment if the segment does not have one.
   1. For the inactive log segments, the broker will append an entry (last_modification_time_of_the_segment -> offset_of_the_first_message_in_the_segment) to each empty time index.
   2. For the active log segments, the time index file will be left empty.

After the entire cluster is migrated to use time based log index for log retention,

1.  The . The broker will enforce log retention using time index. Given what we do in step 1, the behavior is:
   1. For segments only having messages whose versions are before 0.10.0, the entry with last modification time in the time index will be used for retention.
   2. For segments having at least one message with version 0.10.0, the max timestamp of the messages will be used for log retention.

...

1. The broker will

...

1. enforce the

...

1. log rolling

...

1. On startup, broker will initially use the segment last modification time as the largest message timestamp.
2. If a new message whose version is after time when there is at least one message with format version 0.10.0 and its timestamp is greater than current max message timestamp. The broker updates the current largest message timestamp.Broker always use the difference between current time and largest timestamp of the segment to decide whether roll out a new log segment or notinserted, otherwise the log rolling is still based on segment create time.
3. Searching by timestamp behave the same as before because if there is no message with timestamp, the time index will only contain one entry with last_modification_time. If there are messages with timestamp inserted, those messages will be indexed for search.

Rejected Alternatives

Add a timestamp field to log index entry

...