Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Certain operations on KTables, such as print and foreach, are often confusing and redundant, since users can already print the values of a materialized view. Furthermore, users also have the option to use the KStream equivalent functions after converting a KTable to a KStream. So they have 3 ways of printing and doing foreach.

    Apart from the confusions, there is also a semantic gap with late arriving data. Because a changelog topic is compacted in Kafka, a late arriving record with timestamp T1 can over-write a newer record with timestamp T2. That is because log compaction in Kafka does not look at record timestamps.

Proposal

We propose to clean up the KTable API and make the KTable semantics clearer through API improvements and associated JavaDoc improvements.

...

3. Rename toStream() to toKStream() for consistency.

Changes to Kafka

Add the option to do log compaction based on timestamps:

log.cleanup.policy could now be [compact, delete, compact_timestamp]

Compatibility, Deprecation, and Migration Plan

  • No impact on existing users

...