...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Besides the main payload, the majority of sources (if not all of the sources) expose additional information. It can simply be read-only metadata such as a Kafka read-offset or ingestion time. Additionally, users might want to read and write only parts of the record that contain data but additionally serve different purposes (partitioning, compaction etc.)

...