Page History

...

Add time.index.interval configuration to broker

This configuration allows user to change granularity of indexing

Build the time index

Based on the proposal in KIP-32, the broker will build the time index in the following way:

1. When broker receives a message, it checks its timestamp to see if the time difference between the message timestamp and current local time is within max.message.time.difference.ms.The broker overwrites the timestamp with its current local time if the time difference is beyond the thresholdis not rejected due to timestamp exceeds threshold, the message will be appended to the log. 
2. The timestamp will either be LogAppendTime or CreateTime depending on the configuration.
3. The broker checks if the the message timestamp falls into a new minute time.index.interval bucket greater than the previous appended index entry. If it is, the broker appends a new time index entry to the time index with the new minute new time.index.interval timestamp.
4. When a log segment is closed, the broker will always insert a time index entry to the time index of the previous log segment. The entry points to the last message in the previous log segment.
5. If a new log segment is created, the broker will insert a time index entry to the time index of the new log segment when the first message is appended to the log segment. The time index entry points to the first message in the log segment.

...