Page History

...

We do not foresee this happening once the migration is over, and IBP has been deprecated. However, it can happen that we would want to switch back to using IBP-based setup while we are in Phase #3 above (for example, due to an unprecedented issue). This will be supported, because we deprecate IBP only in Phase #4 above. To "downgrade", a user would first need one rolling restart of the cluster to reduce the IBP to the required version. Then another rolling restart to change the binary. Note that each feature will have its own set of requirements to make this possible (in some cases, it may not be possible), but this is outside the scope of this document.

Future work

Support for downgrades: As part of future work, we could consider adding better support for downgrades & deprecation:

• We may consider adding more constraints to the controller to even further restrict the downgrade of specific flags. Certain feature version downgrades may be disallowed if it was previously declared to be an incompatible downgrade. For example, group_coordinator feature is tied to data schemas and EOS processing semantics that may not be forward compatible.
• We may consider providing a mechanism through which the cluster operator could learn whether a feature is being actively used by clients at a certain feature version. This information can be useful to decide whether the feature is safe to be deprecated. Note that this could involve collecting data from clients about which feature versions are actively being used, and the scheme for this needs to be thought through.

Rejected alternatives

1. We considered the idea of using the existing AlterConfigsRequest instead of introducing a new API to update features. Reasons to decide against using it:
   1. The AlterConfigsRequest format is not well equipped to elegantly express the notion of Set<FeatureUpdate> operations. The request format is more favorable towards expressing a list of key-value pairs, where each new value replaces the existing value for the specified key. Furthermore, the response format doesn't let us conveniently return the list of finalized feature versions to the caller.
   2. AlterConfigsRequest can be issued to any broker, which is not our requirement. For the future, anything that modifies ZooKeeper needs to go through the controller. This is part of the work that we are doing to build the bridge release for KIP-500.

2. As opposed to fine-granular features, we considered the idea of exposing the notion of IBP directly to the client for discovery. There are a few issues here:

   1. We are introducing a leaky abstraction here. Users/clients (excluding clients that are brokers) have to now operate with the notion of a "broker protocol version" – this is merely a detail that's internal to a cluster of brokers behind an API.
   2. This still doesn't solve the "double roll" problem related to IBP. We would need to still solve this problem.

...