Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Adding topic level configuration "max.compaction.lag.ms",  and corresponding broker configuration "log.cleaner.max.compaction.lag.ms", which is set to 0 (disabled) by default.  Since users might only specify one of If both "max.compaction.lag.ms" and "min.compaction.lag.ms" when creating/altering topic config in the zookeeper (and the default value will be used for the unspecified configuration), Kafka cannot enforce "min are provided in topic creation, Kafka enforces "max.compaction.lag.ms" is always no less than or equal to "max.compaction.lag.ms".  If "max.compaction.lag.ms" conflict with  "min.compaction.lag.ms" in determining the eligibility of compaction,  "max.compaction. lag.ms" has higher priority than "min.compaction.lag.ms"
    -- Note that an alternative configuration is to use -1 as "disabled" and 0 as "immediate compaction". Because compaction lag is still determined based on min.compaction.lag and how long to roll an active segment,  the actual lag for compaction is undetermined if we use "0".  On the other hand, we can already set "min.cleanable.dirty.ratio" to achieve the same goal.  So here we choose "0" as "disabled".

  • "segment.ms" : no change in meaning.  The active segment is forced to roll when either "max.compaction.lag.ms" or "segment.ms" (log.roll.ms and log.roll.hours) has reached.  

  • min.cleanable.dirty.ratio : no change in meaning. However, the compaction decision that made based on "max.compaction.lag.ms" will override the compaction decision made based on "min.cleanable.dirty.ratio".

  • min.compaction.lag.msWhen no change in meaning. However, when determining the eligibility of compaction, "max.compaction.lag.ms" has higher priority than "min.compaction.lag.ms".  

  • All above changes are only applicable for topics when compaction is enabled.

...