Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. When broker receives a message, if the message is not rejected due to timestamp exceeds threshold, the message will be appended to the log. 
  2. The timestamp will either be LogAppendTime or CreateTime depending on the configuration.
  3. When a new log segment is created, the broker will create a time index file for the log segment.
  4. The time index is not globally monotonically increasing. Instead, it is only guaranteed to be monotonically increasing within each time index file. i.e. It is possible that the time index file for a later log segment contains smaller timestamp than some timestamp in the time index file of an earlier segment.
  5. We will insert a time index entry in the following scenarios:
    1. A time index file is empty and a message is appended to the log segment.
    2. If the timestamp of the appended message is greater than the timestamp of the last time index entry AND the broker has appended more than index.interval.bytes since last time index entry insertion.
    3. When a log segment is closed, the broker will write a time index entry to the time index file. That time index entry points to the message with the largest timestamp in this log segment.
  6. The default initial / max size of the time index files is the same as offset index files.
  7. If all the messages in a log segment have message.format.version before 0.10.0, the broker will insert (last_modification_time_of_the_segment -> offset_of_the_last_message_in_the_segment) for the first reserved entryinto the time index file.

Broker startup

On broker startup, The latest timestamp is needed for the next log index append. The broker will find the largest timestamp of the active segment by looking at the last inserted time index entry and scan from there till the log end.

...