...

Currently the RecordMetadata contains details like topic, partition, offset, timestamp etc. But there is no information on for which record this RecordMetadata belongs to. Having headers information as part of RecordMetaData will bridge this gap.

It will make the  context of the record available to interceptors Interceptors and open up possibility for building more features.

...

As the Kafka usage is growing, we see Kafka being used for Payment transactional, Settlement  and reconciliation use cases. These kind of use cases are very sensitive to message loss.Current dataloss auditing systems like chaperone uses time window based aggregations, which is not sufficient when we need message level traceability. In order to build message level traceability, we need the capability to track message production and consumption.If we choose to track message production status based on acknowledgment the producer receives, we will need ability to relate acknowledgment to the record. Having this ability in Interceptors can help us to link the record to its acknowledgement. 

Public Interfaces

We will change RecordMetaData and FutureRecordMetaData to have headers information.

...