THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- Enhance log compaction to support more than just offset comparison, so the insertion order isn't always dictating which records to keep (in effect, allowing for a form of OCC);
- The current behavior should remain as the default in order to minimize impact on already existing clients and avoid any migration efforts;
- Add new Kafka configuration "log.cleaner.compaction.strategy" to toggle the compaction strategy to this approach;
- Add new Topic configuration "compaction.strategy" representing the same as above, with reserved values "offset" and "timestamp";
- The default value of these configurations should be "offset", which represents the previous behavior;
- When this configuration is set to "timestamp", then the record timestamp will be used to determine which record to keep, in a 'keep-highest' approach;
- When this configuration is set to anything other than "offset" or "timestamp", then the record headers are scanned for a key matching this value. If this header is found then this value (which must be of type "long") will be used to determine which record to keep, in a 'keep-highest' approach;
- When both records being compared contain a "compaction value" that is considered equal, and the configuration is set to either "timestamp" , then both records will be kept;When both records being compared contain a "compaction value" that is considered equal, and the configuration is set to use a or using a custom header, then the record with the offset is used as a tie-breakerhighest offset will be kept;
When both records being compared do not have a "compaction value" at all, then the latest record will be kept;
When only one of the records being compared has a "compaction value", then this record is kept (as the other is considered to be anomalous);
...