Page History

...

A core Kafka developer wants to deprecate a version of an existing API

...

1. Dev modifies the api description in code to add deprecation information. Protocol documentation, which is auto generated, will automatically be updated with the deprecation information. Note that it will be the client developers responsibility to check deprecation information before using or supporting an api version.

...

1. Client sends metadata request and gets to know what brokers it needs to talk to for producing, consuming, etc. and also gets information on the broker's supported api versions. The client maintains this info.
2. Client opens a connection to other brokers that it needs to talk to and sends Metadata request with topics set as null, as it does not want topic metadata again that it has received in previous step. 
3. Client collects supported api versions from all brokers it is interested in and builds a state that will look something like below.

   api versions received:

   B1 -> (0, 0, 3), (1, 2, 3)
   B2-> (0, 1, 2), (1, 0, 3), (2, 0, 0)
   Versions state built by client.
   (0, 1, 2) // ApiKey, MinVersion across brokers, Max version across brokers
   (1, 2, 3)
4. Client is expected to have a map of feature to supported ApiVersions. Something like this.
   Feature1 -> {(0, 3, 3), (1, 2, 3)} // ApiKey, MinVersion supported, Max version supported
   Feature2 -> {(0, 0, 1), (1, 2, 3)}
   Note, that this KIP is not proposing to provide this information to clients. It is clients responsibility to maintain this information.
5. Using information from 3 and 4, client can see if it can use a feature or not. For the client in example, the result will be as below.
   Feature1 can not be used as version 3 of api_key 0 is not supported by all interested brokers.
   Feature 2 can be used.

 

Reason for rejection

The only reason for having api_versions info as part of metadata request would have been that it avoids two network cycles. However, in this approach client anyway needs to create connection to each broker to get their supported api versions, so there was no advantage of having api_versions put in metadata response.

...

A core Kafka developer wants to add a new version of an existing API or a new API

A core Kafka developer wants to deprecate a version of an existing API

A client developer does not want to be dependent on API versions

A client developer wants to add support for a new feature

 

...

1. Dev initiates a KIP, gets approval, gets implementation ready to be committed.
2. Dev claims the next global version id, i.e., current global version id +1.
3. Dev updates the commit with the claimed global version id. The commit should have following pieces.
   1. Code for the new API or new version of an existing API.
   2. Clone of latest protocol documentation for the branch on which changes are being made, updated with information on new version.
   3. Changes to main/ global protocol documentation page with link to the new protocol documentation page created in step b.
4. Changes get committed in a single commit.

A core Kafka developer wants to deprecate a version of an existing API

1. Dev claims the next global version id, i.e., current global version id +1.
2. Dev creates following changes in a commit. The commit should have following pieces.
   1. Clone of latest protocol documentation for the branch on which changes are being made, updated with information on deprecation of concerned api versions.
   2. Changes to main/ global protocol documentation page with link to the new protocol documentation page created in step a.
3. Changes get committed in a single commit.
   

A client developer does not want to be dependent on API versions

 

1. They continue to operate the way they do.

A client developer wants to add support for a new feature

1. Client sends metadata request and gets to know what brokers it needs to talk to for producing, consuming, etc. and also gets information on the broker's global api version. The client maintains this info.
2. Client opens a connection to other brokers that it needs to talk to and sends Metadata request with topics set as null, as it does not want topic metadata again that it has received in previous step. 
3. Client collects global api versions from all brokers it is interested in and builds a state that will look something like below.

   api versions received:

   B1 -> 4
   B2-> 5
4. Client is expected to have a map of feature to required minimum global api versions. Something like this.
   Feature1 -> 6 // Min. global api version
   Feature2 -> 4
   Note, that this KIP is not proposing to provide this information to clients. It is clients responsibility to maintain this information.
5. Using information from 3 and 4, client can see if it can use a feature or not. For the client in example, the result will be as below.
   Feature1 can not be used as all brokers, client is interested in, do not have >= min. req. global api version.
   Feature2 can be used. 

Reason for rejection

This approach makes selective backporting of certain protocol versions to previous branch difficult, hard to manage. One way to make it work is to have clients maintain specifically what global api versions it supports, instead of a minimum required global api version.