Page History

...

In many cases, epoch is represented with different precisions within external systems : seconds, milliseconds, microseconds, nanoseconds.

When such case arise, Kafka Connect can't do anything expect pass along the Long and leave the conversion to another layer.

This issue was raised several times : 

...

Add a config property epoch.precision, which defaults tomillis for compatibility. Possible values: seconds, millis, micros, nanos.

...

Implementation details to be discussed :

• TimeUnit.MILLISECONDS.toMicros(epochMilis) and so on for the other conversions seems the easiest way. 

"transforms.TimestampConverter.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.TimestampConverter.field": "event_date_long",
"transforms.TimestampConverter.epoch.precision": "micros",
"transforms.TimestampConverter.target.type": "Timestamp"

java.util.Date and SimpleDateFormat limitations

Since these classes can only handle precisions down to the millisecond, it should be noted that:

• converting source Long microseconds or nanos into any target type leads to a precision loss (truncation after millis)
• converting any source type into target Long microseconds or nanos, the microseconds part after milliseconds will always be 000be 0
• A KIP that address Date vs Instant may be more appropriate but it impacts so much of the code that I believe this is a good first step.

int32 and seconds

Systems that produces int32 into Kafka should willingly chain Cast SMT and then TimestampConverter SMT if they want to use this feature.

...