...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

TodayCurrently, Apache Kafka lacks a mechanism to enforce configuration constraints that are tied to specific metadata.versionlevels. As Kafka’s metadata evolves, certain configuration values that were considered valid in older versions . This creates two problemsmay become semantically incorrect or unsafe in newer ones. Without a validation layer to enforce these version-specific constraints, two primary issues arise:

1. Unsafe metadata version upgrades: Administrators may not be aware that their configuration changes were ignored, leading to unexpected cluster behavior.

2. No runtime enforcement of version-specific constraints: Even after upgrading, administrators can use AlterConfigs or IncrementalAlterConfigs to set config values that violate constraints introduced by the current metadata version. There is no guardrail beyond the static ConfigDef validators, which are version-agnostic.

...

1. Broker static reporting: Brokers report their static (non-sensitive) configurations to the controller during registration. This enables the controller to perform pre-flight validation of static configs during metadata version upgrades.
2. Metadata-Version-Aware Validators::Configuration definitions enhance to support metadata-version-aware validators. The ConfigDef.define() method accepts a mvValidators parameter — a map navigableMap from feature level (short) to ConfigDef.Validator instances.
   1. During AlterConfigs and IncrementalAlterConfigs operations, the Metadata-Version-Aware validators will be invoked to ensure that any proposed changes are compliant with the cluster's current metadata version. Any configuration that fails this runtime validation will be rejected with an INVALID_CONFIG error.
   2. When an administrator performs a metadata.version upgrade or downgrade, the Controller will execute a comprehensive audit of all existing configurations—both static (via BrokerRegistrationRecord) and dynamic (via ConfigRecord). The transition will only proceed if all current settings satisfy the constraints of the target metadata version, ensuring a safe and atomic version migration.

...