Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Replication is aligned by batches, but up-conversion will happen independently on each broker for cases 3 and 4. For compressed records, v0 and v1 message formats support batching, so it's straightforward. Uncompressed records, however, are not batched in v0 and v1. To ensure alignment, we will convert them to single record batches in v2. It's worth noting that message format v2 is slightly less efficient than v0 and v1 when single record batches are used, but it's an acceptable cost for correctness and it only impacts older records. Over time, these records will either be deleted via retention or will be replaced by new versions for compacted topics.
  2. We need to make sure that once we write a record batch with a new format, we never write a record batch with an older format. Is this really true, it seems that we verify all record batches to decide whether we need to down-convert?

Apache Kafka 4.0

We will remove all support for message formats v0 and v1 in Apache Kafka 4.0.

...