DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Broker static reporting: Brokers report their static (non-sensitive) configurations to the controller during registration. This enables the controller to perform pre-flight validation of static configs during metadata version upgrades. On the broker side, BrokerLifecycleManager is responsible for collecting and sending static configs. During registration, it iterates over all config entries need to be validated.
Metadata-Version-Aware Validators: Configuration definitions are enhanced to support metadata-version-aware validators. The ConfigDef.define() method accepts a mvValidators parameter — Uses NavigableMap.floorEntry() to find the validator registered at the highest feature level ≤ the given featureLevel. This allows constraints to be introduced incrementally across metadata versions.
- When an administrator runs kafka-storage format --release-version <version>, the tool now validates the broker's static configs from server.properties against the MV constraints of the specified release version. If any config violates the constraints, the format command fails with a descriptive error before writing any data to disk.
- During During
AlterConfigsandIncrementalAlterConfigsoperations, the Metadata-Version-Aware validators will be invoked to ensure that any proposed changes are compliant with the cluster's current metadata version. Any configuration that fails this runtime validation will be rejected with anINVALID_CONFIGerror. - When an administrator performs a
metadata.versionupgrade or downgrade, the Controller will execute a comprehensive audit of all existing configurations—both static (viaBrokerRegistrationRecord) and dynamic (viaConfigRecord). The transition will only proceed if all current settings satisfy the constraints of the target metadata version, ensuring a safe and atomic version migration.
Example:
Suppose IBP_4_4_IV1 raises the minimum allowed value for log.segment.bytes from 1 byte to 3 MB. This constraint is registered as an MV-aware validator on the log.segment.bytes config key. Because log.segment.bytes now has an MV validator, it will be included in the static config payload that brokers send to the controller during registration.
- Format time: An administrator prepares a new broker with log.segment.bytes=1024 in server.properties and runs kafka-storage format --release-version 4.4. Before writing any data to disk, the tool validates the static configs against the constraints of IBP_4_4_IV1 and finds that log.segment.bytes=1024 violates the minimum of 3 MB. The command fails with a descriptive error, prompting the administrator to fix server.properties before retrying.
- AlterConfigs at runtime: Once the cluster is running at IBP_4_4_IV1, an administrator attempts to set log.segment.bytes=1024 on a topic via AlterConfigs. The controller validates the proposed value against the current MV's constraints and immediately rejects the request with an INVALID_CONFIG error before any change is persisted.
- Metadata version upgrade: A cluster is running at IBP_4_3_IV0. Topic payments has a long-standing dynamic override of log.segment.bytes=1024. Broker node-3 also has log.segment.bytes=2048 in its server.properties; since log.segment.bytes has an MV validator, this value was reported to the controller during registration and persisted in the metadata log. When an administrator runs kafka-storage format --release-version <version>, the tool now validates the broker's static configs from server.properties against the MV constraints of the specified release version. If any config violates the constraints, the format command fails with a descriptive error before writing any data to diskthe administrator requests an upgrade to IBP_4_4_IV1, the controller audits all effective configs — both dynamic overrides and the static configs reported by each broker — and finds two violations. The upgrade is rejected with a descriptive error listing both the topic and the broker. The administrator fixes both values and retries, at which point the upgrade proceeds successfully.
Public Interfaces
ConfigDef
...