...

PULL REQUEST: https://github.com/apache/kafka/pull/7368 (WIP) 

https://github.com/apache/kafka/pull/5149

Motivation

Current Kafka log compaction is based on server side view, which means records are compacted only based on records offset. For the same key, only the highest offset record is kept after compaction. Note that records are appended to logs based on the order when message is received on broker for the same topic partition. This default strategy insufficient in many scenarios. On the producer side, when multiple producers produce to the same topic partition, the producer side record order cannot be guaranteed on the server side. This is because the message transmitting over the network is determined by many factors out of the control of Kafka. On the server side, which message is reserved after compaction is random. The following is a motivating example:

...