Status
Current state: Accepted
Discussion thread: here
JIRA: KAFKA-3846
Released: 0.10.1.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Timestamps were added to Kafka record types in the 0.10 release (KIP-32), however this does not get propagated automatically to Connect because it uses custom wrappers to add fields and rename some for clarity.
The addition of timestamps is trivial, but can be really useful:
New constructor variants and getters for the field on SourceRecord and SinkRecord.
public SourceRecord(Map<String, ?> sourcePartition, Map<String, ?> sourceOffset, String topic, Integer partition, Schema keySchema, Object key, Schema valueSchema, Object value, Long timestamp) { .. } public Long timestamp() { .. } |
public SinkRecord(String topic, int partition, Schema keySchema, Object key, Schema valueSchema, Object value, long kafkaOffset, Long timestamp, TimestampType timestampType) { .. } public TimestampType timestampType() { .. } public Long timestamp() { .. } |
Existing constructors for SourceRecord and SinkRecord will continue to exist so there are no compatibility concerns.
Unit tests that validate
In addition to timestamps, we could have also exposed record metadata that was added to Kafka's ConsumerRecord in KIP-42 (Interceptors), i.e. checksum and serialized size. There doesn't seem to be a use-case for exposing these fields to sink connectors.