Page History

...

• Compaction Strategies:
  • "offset"
    • The current behavior is active, compacting the logs purely based on offset;
    • Also used when the configuration is either empty or not present, making this the default strategy;
  • "timestamp"
    • The record timestamp will be used to determine which record to keep, in a 'keep-highest' approach;
    • When both records being compared contain an equal timestamp, then the record with the highest offset will be kept;
    • This requires caching also the timestamp field during compaction, in addition to the base offset, so each record being compacted will suffer a memory increase from 8 bytes to 16 bytes when using this strategy.
  • "header"
    • Searches the record for a header key that matches the configured value on "log.cleaner.compaction.strategy.header";
      • as the header can have duplicates, will pick the first last occurrence of the header key
    • If both records being compared do not have a matching header key, then the record with the highest offset will be kept;
    • If a header key that matches the configuration exists, then the header value (which must be of type "long" - 8 bytes) will be used to determine which record to keep, in a 'keep-highest' approach;
    • If both records being compared contain an equal header value, then the record with the highest offset will be kept;
    • If only one of the records being compared has a matching header, then this record is kept, as the other record is considered to be anomalous;
    • This requires caching also the header value during compaction, in addition to the base offset, so each record being compacted will suffer a memory increase from 8 bytes to 16 bytes when using this strategy.

...