...

Four years later, it's time to sunset message formats v0 and v1 to establish a new baseline in terms of supported broker behavior and to simplify the codebase (with all the benefits that brings), to improve maintainability and supportability. This also aligns with KIP-679, which will enable the idempotent producer by default in Apache Kafka 3.0 (and requires message format v2). We propose the deprecation of message formats v0 and v1 in Apache Kafka 3.0 and their removal in Apache Kafka 4.0.

...

Produce and fetch requests with v0 and v1 message formats would be supported via up-conversion and down-conversion. Up-conversion and (especially) down-conversion have measurable performance impact due to the increased CPU and memory usage, but the vast majority of Kafka clients have supported v2 for some time (even Spark, a notable late adopter, has supported it v2 since Spark 2.4, which was released in October 2018).

In order to remove saf The log.record.version.force.upgrade config TBD config 

Apache Kafka 4.0

...

1. Remove support for produce/consume with older message formats in AK 4.0. This would align well if we make 0.11 (or newer) the minimum client supported version, but it can also be done independently.

2. Maintain support for reading all message format versions when it comes to persisted data for the foreseeable future. We don't have a good mechanism to ensure data with the old formats doesn't exist on disk.

3. Note that Produce v3 and higher require message format v2. Fetch v4 or higher are required for message format v2.

Compatibility, Deprecation, and Migration Plan

...