Page History

...

A core Kafka developer wants to add a new version of an existing API or a new API

1. Dev adds the new version of API or new API the way they do it today. Protocol documentation, which is auto generated, will automatically be updated with this new version or API in docs. Nothing else required to be done.

A core Kafka developer wants to deprecate a version of an existing API

A client developer does not want to be dependent on API versions

A client developer wants to add support for a new feature

...

1. Dev modifies the api description in code to add deprecation information. Protocol documentation, which is auto generated, will automatically be updated with the deprecation information. Note that it will be the client developers responsibility to check deprecation information before using or supporting an api version.

A client developer does not want to be dependent on API versions

1. If the clients are using MetadataRequest v0, they can continue operating the way they do without any change in behavior.
   
2. If the clients are using MetadataRequest >= v1, they can set api_keys as null, and they will not get any info on supported api versions as part of Metadata response.

A client developer wants to add support for a new feature

1. Client sends metadata request and gets to know what brokers it needs to talk to for producing, consuming, etc. and also gets information on the broker's supported api versions. The client maintains this info.
2. Client opens a connection to other brokers that it needs to talk to and sends Metadata request with topics set as null, as it does not want topic metadata again that it has received in previous step. 
3. Client collects supported api versions from all brokers it is interested in and builds a state that will look something like below.

   api versions received:

   B1 -> (0, 0, 3), (1, 2, 3)
   B2-> (0, 1, 2), (1, 0, 3), (2, 0, 0)
   Versions state built by client.
   (0, 1, 2) // ApiKey, MinVersion across brokers, Max version across brokers
   (1, 2, 3)
4. Client is expected to have a map of feature to supported ApiVersions. Something like this.
   Feature1 -> {(0, 3, 3), (1, 2, 3)} // ApiKey, MinVersion supported, Max version supported
   Feature2 -> {(0, 0, 1), (1, 2, 3)}
   Note, that this KIP is not proposing to provide this information to clients. It is clients responsibility to maintain this information.
5. Using information from 3 and 4, client can see if it can use a feature or not. For the client in example, the result will be as below.
   Feature1 can not be used as version 3 of api_key 0 is not supported by all interested brokers.
   Feature 2 can be used.

Reason for rejection

The only reason for having api_versions info as part of metadata request would have been that it avoids two network cycles. However, in this approach client anyway needs to create connection to each broker to get their supported api versions, so there was no advantage of having api_versions put in metadata response.

Option3: Make protocol documentation as source of truth, instead of code

...