Page History

...

1. Unsafe metadata version upgrades: Administrators may not be aware that their configuration changes were ignored, leading to unexpected cluster behavior.

2. No runtime enforcement of version-specific constraints: Even after upgrading, administrators can use AlterConfigs or IncrementalAlterConfigs to set config values that violate constraints introduced by the current metadata version. There is no guardrail beyond the static ConfigDef validators, which are version-agnostic.

...

1. Broker static reporting: Brokers report their static (non-sensitive) configurations to the controller during registration. This enables the controller to perform pre-flight validation of static configs during metadata version upgrades.
2. Metadata-Version-Aware Validators::Configuration definitions enhance to support metadata-version-aware validators. The ConfigDef.define() method accepts a mvValidators parameter — a map from feature level (short) to ConfigDef.Validator instances.When a broker attempts to register with the cluster, the Controller must audit its static configurations stored in the BrokerRegistrationRequest. If any configuration violates the constraints of the current active metadata.version, the Controller will reject the registration, preventing the broker from joining the cluster in an inconsistent state.
   1. During AlterConfigs and IncrementalAlterConfigs operations, the Metadata-Version-Aware validators will be invoked to ensure that any proposed changes are compliant with the cluster's current metadata version. Any configuration that fails this runtime validation will be rejected with an INVALID_CONFIG error.
   2. When an administrator performs a metadata.version upgrade or downgrade, the Controller will execute a comprehensive audit of all existing configurations—both static (via BrokerRegistrationRecord) and dynamic (via ConfigRecord). The transition will only proceed if all current settings satisfy the constraints of the target metadata version, ensuring a safe and atomic version migration.

...

ConfigDef

We are introducing a new mvValidators field to the ConfigDef class, enabling configurations to be validated against specific metadata.version thresholds. This allows the system to enforce version-specific constraints dynamically as the cluster evolves.

...

BrokerRegistrationRequest

We propose to bump BrokerRegistrationRequest and BrokerRegistrationResponse to a new version to include a map of static configurations. This allows brokers to report their local server.properties settings to the Controller during the registration phase.

...

BrokerRegistrationRecord

We propose updating RegisterBrokerRecord to Version 5. This version introduces a new tagged field StaticConfigs, which is a collection of BrokerStaticConfig objects. This allows the Controller to persist the broker's reported static settings directly within the metadata log.

...