Versions Compared

Old Version 33

changes.mady.by.user xiongqi wu

Saved on

compared with

New Version 34

changes.mady.by.user xiongqi wu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Adding topic level configuration "max.compaction.lag.ms",  and corresponding broker configuration "log.cleaner.max.compaction.lag.ms", which is set to MAX_LONG by default.  If both "max.compaction.lag.ms" and "min.compaction.lag.ms" are provided in topic creation, Kafka enforces "max.compaction.lag.ms" is no less than "min.compaction.lag.ms".  This configuration only applies to topics that have compaction enabled. 

  • Add two Metrics:  
    1) kafka.log:type=LogCleaner,name=num-logs-compacted-by-max-compaction-lag
    type: gauge
    value: the total number of  of logs which needs to be immediately compacted as determined by max.compaction.lag ; in the last cleaner run.

    2) kafka.log:type=LogCleaner,name=max-compaction-delay
    type: gauge
    value: the maximum value of "compaction_finish_time - earliest_timestamp_of_first_uncompacted_segment - max.compaction.lag.ms" among all logs that need to be compacted in the last cleaner run.
    This metric tells the max delay between the time when a log is required to be picked up for compaction and the time when the compaction is done for the log. 


...