...

• This is an optional interface and does not affect existing clients.
• Updated clients that add support for the new ApiVersionQueryRequest will be able to adapt their functionality based on supported broker functionality.

Scenarios

A core Kafka developer wants to add a new version of an existing API or a new API

1. Dev adds the new version of API or new API the way they do it today. Protocol documentation, which is auto generated, will automatically be updated with this new version or API in docs. Nothing else required to be done.

A core Kafka developer wants to deprecate a version of an existing API

1. Dev modifies the api description in code to add deprecation information. Protocol documentation, which is auto generated, will automatically be updated with the deprecation information. Note that it will be the client developers responsibility to check deprecation information before using or supporting an api version.

A client developer does not want to be dependent on API versions

1. They continue to operate the way they do without using changes proposed in this KIP.

A client developer wants to add support for a new feature

1. Client sends metadata request and gets to know what brokers it needs to talk to for producing, consuming, etc.
2. Client opens a connection to those brokers and sends ApiVersionQueryRequest. Cl
3. Client collects ApiVersionQueryResponse from all brokers it is interested in and builds a state that will look something like below.

   ApiVersionQueryResponse received:

   B1 -> (0, 0, 3), (1, 2, 3)
   B2-> (0, 1, 2), (1, 0, 3), (2, 0, 0)
   Versions state built by client.
   (0, 1, 2) // ApiKey, MinVersion across brokers, Max version across brokers
   (1, 2, 3)
4. Client is expected to have a map of feature to supported ApiVersions. Something like this.
   Feature1 -> {(0, 3, 3), (1, 2, 3)} // ApiKey, MinVersion supported, Max version supported
   Feature2 -> {(0, 0, 1), (1, 2, 3)}
   Note, that this KIP is not proposing to provide this information to clients. It is clients responsibility to maintain this information.
5. Using information from 3 and 4, client can see if it can use a feature or not. For the client in example, the result will be as below.
   Feature1 can not be used as version 3 of api_key 0 is not supported by all interested brokers.
   Feature 2 can be used.

Rejected Alternatives

• Include meta key-value tags in response ("broker.id=...", "message.max.bytes=...", ...)
• Include user-defined tags in response ("aws.zone=...", "rack=...", ..)
• Using a versionInt instead of ApiVersion's id as broker protocol version.

• Use a new protocol request and response, ProtocolVersionRequest and ProtocolVersionResponse. A client may use this new API to query the broker for its supported protocol request types and versions and use the information to select a suitable set of conforming API requests and versions for further communication with the broker.

• Return array of supported versions, instead of min and max of supported versions, in MetadataResponse.
• Return API versions in a new version of the MetadataRequest/Response

...