Current state: Under discussion
Discussion thread: https://lists.apache.org/thread/w4ml9ffkj1j31j8kjpbywq9jsw5ck5sr
JIRA:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Currently, there exists a kafka.server:type=MetadataLoader,name=CurrentMetadataVersion metric which tells the operator the metadata version feature level for every node in a cluster. However, now that there several features with support beyond the metadata.version feature, adding a metric to display those features' levels would be helpful for monitoring upgrade/downgrade scenarios for clusters using those features. Additionally, adding metrics for the minimum and maximum supported feature levels of each node would be helpful for the operator to determine if a feature upgrade/downgrade is safe to perform.
Add a metric for each production feature (i.e. metadata.version , kraft.version , transaction.version , group.version , eligible.leader.replicas.version , share.version, and streams.version ).
This can be accomplished via one generic "feature level" metric that is tagged by feature. Using a generic metric allows for new production features to automatically expose their feature level going forward.
| Name | Type | Description |
|---|---|---|
| kafka.server:type=MetadataLoader,name=FinalizedLevel,featureName=X | Integer | The finalized value of the feature level for a feature named "X". |
| kafka.controller:type=KafkaController,name=MinimumSupportedLevel,featureName=X | Integer | The minimum supported feature level for a feature named "X" on the controller. |
| kafka.controller:type=KafkaController,name=MaximumSupportedLevel,featureName=X | Integer | The maximum supported feature level for a feature named "X" on the controller. |
| kafka.server:type=broker-metadata-metrics,name=minimum-supported-feature-level,featureName=X | Integer | The minimum supported feature level for a feature named "X" on the broker. |
kafka.server:type=broker-metadata-metrics,name=maximum-supported-feature-level,featureName=X | Integer | The maximum supported feature level for a feature named "X" on the broker. |
The FinalizedLevel metric will report the finalized feature level for each production feature. If the feature level is not set, the metric will return a value of 0, since that means the feature is not enabled.
We will deprecate the kafka.server:type=MetadataLoader,name=CurrentMetadataVersion metric since this per-feature tagged metric also exposes this value, and the previous metric can be removed starting in 5.0.
We will add junit tests to verify the new metrics.
Running kafka-feature describe on every node is one way to track feature level upgrades/downgrades, but it is not straightforward to monitor since it is not a metric.