...

Add the new transaction version and group version features. For every metadata version, we will map to the corresponding transaction and group version. For MVs that existed before these features, we map the new features to version 0.

Fixing KAFKA-17011

As part of KIP-584, brokers expose a range of supported versions for each feature. For example, metadata.version might be supported from 1 to 21. (Note that feature level ranges are always inclusive, so this would include both level 1 and 21.)

These supported ranges are supposed to be able to include 0. For example, it should be possible for a broker to support a kraft.version of [0, 1]. However, in older software versions, there is an assertion in org.apache.kafka.common.feature.SupportedVersionRange that prevents this. This causes problems when the older software attempts to deserialize an ApiVersionsResponse containing such a range.

In order to resolve this dilemma, we will create a new version 4 for ApiVersionsRequest. Clients which send v4 promise to be able to handle ranges including 0. Clients which send v3 will not be exposed to these ranges -- the feature will simply be omitted from the response.

Compatibility, Deprecation, and Migration Plan

...