...

• Tombstone feature will be backwards compatible
  • Message version migration would be handled as like in KIP-32
  • Bit flag would be default false 
  • Logic would base on current behavior of null value or if tombstone flag set to true, as such wouldn't impact any existing flows simply allow new producers to make use of the feature.

Rejected Alternatives

Do nothing KIP-82 would remove the immediate issue.

Originally it was proposed in in KIP-82 - Add Record Headers that by supporting headers would resolve this issue as then there would be no need for message wrappers the as one of the issues being flagged up in that discussion thread.

As per its discussion it got agreed that we should address the compaction based on null value seperately as discussed here, as it is cleaner to be explicit than still relying on a null payload even after header support.

Use Header field to be explicit based on KIP-82 Headers Proposal

Intent here is to use a header e.g. key = 5 (compaction tombstone) value type=boolean to make an explicit flag that can be used. 

...

• Risk of the issues this KIP being dependant on KIP-82, and as KIP-82 is more contentious risks not dealing with the immediate issue
  • This could always be re-wroked worked in future to use a header in a future release.

...