Status

Current state: Under Discussion Adopted

Discussion thread: here

JIRA:

...

• Add version-mapping command to to look up the corresponding features defaults for a given metadata version. Using the command with no  --release-version  argument will return the mapping for the latest stable metadata version.
• Add feature-dependencies command to look up dependencies for a given feature version supplied by --feature flag. If the feature is not known or the version not yet defined, throw an error. 
   

...

For both features, we will be finally using the flexible fields in KIP-915. This means that the records for the state topics of the transaction coordinator and the group coordinator will be determined by their respective feature versions and not metadata version. 

Test Plan

Tests will be added to the StorageToolTest and FeatureCommandTest.

Implicit in this KIP is also the necessity to test various combinations of feature versions. We should test all “release default” combinations and have a test for every version of a given feature with other default features, but do not commit to testing every combination of features.

...

A Note on Production Ready vs Default

One concern with adding new features is the ability to mark the features as stable/production ready and the ability to mark the feature as enabled by default. With MV, the latest production version is the default. However, there were some concerns with marking feature versions that are production ready as the default. Thus, we will have two separate mechanisms. Each feature will have a field to indicate the latest production (stable) version of the feature. However, this version will not become the default until a metadata version that corresponds to the feature is marked as production ready. A feature version must be production ready before the metadata version corresponding to it is marked as production ready. 

Here is an example:

Say we have a feature version group.version=1. In order to mark as production ready, a developer sets the LATEST_PRODUCTION value of GroupVersion to GV_1. From there, the feature can be enabled via --feature without using unstable.feature.versions.enabled. However, if a cluster is formatted or upgraded without this flag for this version, the cluster won't get that feature by default. Once the metadata version mapping (say 4.0_IV0) is marked as production ready, the feature will be enabled by default when flags are not specified in the storage tool and when --release-version 4.0_IV0 is used.

Test Plan

Tests will be added to the StorageToolTest and FeatureCommandTest.

Implicit in this KIP is also the necessity to test various combinations of feature versions. We should test all “release default” combinations and have a test for every version of a given feature with other default features, but do not commit to testing every combination of features.

Rejected Alternatives

Use metadata version to determine production readiness

One proposed approach was to tie a feature's production readiness to the metadata mapping. Thus, a feature could only be enabled if the corresponding metadata version was production ready. One side effect of this approach is that a feature would be both production ready and set as a default at the same time. For some features that take more time to develop and test across releases (the new group coordinator) it is useful to mark the feature as production ready, but not set it as the default. Thus, feature specific production ready values indicate the feature and protocol are stable and ready to use, but the default cluster will not enable those until the metadata version is marked as production ready. 

Use release version to determine default feature versions

...