Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Adding topic level configuration "max.compaction.lag.ms",  and corresponding broker configuration "log.cleaner.max.compaction.lag.ms", which is set to -1 0 (disabled) by default. Setting "max.compaction.lag.ms" to 0 means to perform compaction immediately after "min.compaction.lag.ms" has reached.  Since users might only specify one of "max.compaction.lag.ms" and "min.compaction.lag.ms" when creating/altering topic config in the zookeeper (and the default value will be used for the unspecified configuration), Kafka cannot enforce "min.compaction.lag.ms" is always less than or equal to "max.compaction.lag.ms".  If "max.compaction.lag.ms" conflict with  "min.compaction.lag.ms" in determining the eligibility of compaction,  "max.compaction.lag.ms" has higher priority than "min.compaction.lag.ms"
    -- Note that an alternative configuration is to use -1 as "disabled" and 0 as "immediate compaction". Because compaction lag is still determined based on min.compaction.lag and how long to roll an active segment,  the actual lag for compaction is undetermined if we use "0".  On the other hand, we can already set "min.cleanable.dirty.ratio" to achieve the same goal.  So here we choose "0" as "disabled".

  • "segment.ms" : no change in meaning.  The active segment is forced to roll when either "max.compaction.lag.ms" or "segment.ms" (log.roll.ms and log.roll.hours) has reached.  

  • min.cleanable.dirty.ratio : no change in meaning. However, the compaction decision that made based on "max.compaction.lag.ms" will override the compaction decision made based on "min.cleanable.dirty.ratio".

  • min.compaction.lag.ms :  When determining the eligibility of compaction, "max.compaction.lag.ms" has higher priority than "min.compaction.lag.ms".  

  • All above changes are only applicable for topics when compaction is enabled.

...