...

Public Interfaces

This KIP does not introduce any public interface change.introduce a new configuration time.index.interval.ms on the broker side to control the granularity of time based index.

Besides thatHowever, there will be some behavioral changes to time based log retention and log rolling.

...

Add time.index.interval.ms configuration to broker

This configuration allows user to change granularity of indexing

...

1. When broker receives a message, if the message is not rejected due to timestamp exceeds threshold, the message will be appended to the log. 
2. The timestamp will either be LogAppendTime or CreateTime depending on the configuration.
3. If the message timestamp falls into a new time.index.interval.ms bucket which is greater than the previous appended index entry, the broker appends a new time index entry to the time index with the new time.index.interval.ms bucket timestamp.
4. When a log segment is closed, if the message with largest timestamp is in this closed segment, the broker will insert a time index entry to the time index. The time index entry points to that message with largest timestamp.
5. When a new log segment is created, the broker will insert a time index entry to the time index of the new log segment when the first message whose timestamp falls into a new time.index.interval.ms bucket is appended to the log segment. 
6. It is possible that a log segment does not have any time index entry if all the messages has smaller timestamp than the previous log segments. In that case the time based index would be empty.
7. If all the messages in a log segment have message.format.version before 0.10.0, broker will insert a time index entry (last_modification_time_of_the_segment -> offset_of_the_last_message_in_the_segment) to the time index.

...