...

• Tombstone feature will be backwards compatible
  • Message version migration would be handled as like in KIP-32
  • Bit flag would be default false 
  • Logic would base on current behavior of null value or if tombstone flag set to true, as such wouldn't impact any existing flows simply allow new producers to make use of the feature.

Rejected Alternatives

...

Do nothing KIP-82 would remove the immediate issue.

Originally it was proposed in KIP-82 - Add Record Headers that by supporting headers would resolve this issue as then there would be no need for message wrappers the one of the issues being flagged up in that discussion thread. As per its discussion it got agreed that we should address the compaction based on null value seperately as discussed here, as it is cleaner to be explicit than still relying on a null payload even after header support.

Use Header field to be explicit based on KIP-82 Headers Proposal

Intent here is to use a header e.g. key = 5 (compaction tombstone) value type=boolean to make an explicit flag that can be used. 

Advantages

• Avoids any further changes to the core message protocol , this is one of the intents of KIP-82 to be able to add additional platform level information without constant protocol changes
• Avoids using the attributes flags which are finite is size

Disadvantages

• Risk of the issues this KIP being dependant on KIP-82, and as KIP-82 is more contentious risks not dealing with the immediate issue
  • This could always be re-wroked in future to use a header in a future release.