This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here [Change the link from KAFKA-1 to your own ticket]

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Apache Kafka’s configuration validation is currently version-agnostic. Validation logic is defined in ConfigDef and executed during broker startup, which tightly couples the broker’s static codebase with the cluster’s operational configuration state.

This makes it difficult to strengthen configuration constraints over time. For example, if the minimum allowed value for log.segment.bytes is increased (e.g., from 1 byte to 3 MB), brokers with legacy values in server.properties will fail to start after a binary upgrade due to a ConfigException. Because the broker cannot join the cluster, administrators cannot use dynamic APIs such as AlterConfigs to fix the configuration and must instead manually update configuration files across the fleet.

To address the limitations of the current static, version-agnostic validation model, Kafka needs a mechanism to enforce configuration constraints tied to metadata.version. As Kafka evolves, configuration values that were previously valid may become unsafe or semantically incorrect in newer versions. Without version-aware validation, two issues arise:

Unsafe metadata version upgrades: administrators may upgrade metadata.version without realizing that existing configurations violate new constraints.
No Metadata-version-aware enforcement: administrators can still use AlterConfigs or IncrementalAlterConfigs to set values that violate constraints introduced in newer metadata versions.

Additionally, the controller currently has no visibility into broker static configurations (those set in server.properties). Since these configs need to be changed dynamically, they must be validated before an upgrade is allowed — otherwise, a broker whose static config violates a new constraint would fail after the upgrade.

Proposed Changes

This KIP introduces three interconnected features:

Broker static reporting: Brokers report their static (non-sensitive) configurations to the controller during registration. This enables the controller to perform pre-flight validation of static configs during metadata version upgrades.
Metadata-Version-Aware Validators::Configuration definitions enhance to support metadata-version-aware validators. The ConfigDef.define() method accepts a mvValidators parameter — a navigableMap from feature level (short) to ConfigDef.Validator instances.
1. During AlterConfigs and IncrementalAlterConfigs operations, the Metadata-Version-Aware validators will be invoked to ensure that any proposed changes are compliant with the cluster's current metadata version. Any configuration that fails this runtime validation will be rejected with an INVALID_CONFIG error.
2. When an administrator performs a metadata.version upgrade or downgrade, the Controller will execute a comprehensive audit of all existing configurations—both static (via BrokerRegistrationRecord) and dynamic (via ConfigRecord). The transition will only proceed if all current settings satisfy the constraints of the target metadata version, ensuring a safe and atomic version migration.

Public Interfaces

ConfigDef

We are introducing a new mvValidators field to the ConfigDef class, enabling configurations to be validated against specific metadata.version thresholds. This allows the system to enforce version-specific constraints dynamically as the cluster evolves.

    public static class ConfigKey {
        public final String name;
        public final Type type;
        public final String documentation;
        public final Object defaultValue;
        public final Validator validator;
        public final Importance importance;
        public final String group;
        public final int orderInGroup;
        public final Width width;
        public final String displayName;
        public final List<String> dependents;
        public final Recommender recommender;
        public final boolean internalConfig;
        public final String alternativeString;
        public final NavigableMap<Short, Validator> mvValidators; // new field
...
}

BrokerRegistrationRequest

We propose to bump BrokerRegistrationRequest and BrokerRegistrationResponse to a new version to include a map of static configurations. This allows brokers to report their local server.properties settings to the Controller during the registration phase.

diff --git a/clients/src/main/resources/common/message/BrokerRegistrationRequest.json b/clients/src/main/resources/common/message/BrokerRegistrationRequest.json
index 53e37f21d5..cdd53c4466 100644
--- a/clients/src/main/resources/common/message/BrokerRegistrationRequest.json
+++ b/clients/src/main/resources/common/message/BrokerRegistrationRequest.json
@@ -18,12 +18,13 @@
 // Version 3 adds the PreviousBrokerEpoch for the KIP-966
 // Version 4 fixes KAFKA-17011, which blocked SupportedFeatures.MinVersion in the response from being 0.
 // Version 5 adds the CordonedLogDirs flexible field
+// Version 6 adds StaticConfigs for broker static config reporting.
 {
   "apiKey":62,
   "type": "request",
   "listeners": ["controller"],
   "name": "BrokerRegistrationRequest",
-  "validVersions": "0-5",
+  "validVersions": "0-6",
   "flexibleVersions": "0+",
   "fields": [
     { "name": "BrokerId", "type": "int32", "versions": "0+", "entityType": "brokerId",
@@ -63,6 +64,15 @@
     { "name": "PreviousBrokerEpoch", "type": "int64", "versions": "3+", "default": "-1", "ignorable": true,
       "about": "The epoch before a clean shutdown." },
     { "name": "CordonedLogDirs", "type":  "[]uuid", "versions":  "5+", "taggedVersions": "5+",
-      "tag": "0", "about": "Log directories that are cordoned." }
+      "tag": "0", "about": "Log directories that are cordoned." },
+    { "name": "StaticConfigs", "type": "[]StaticConfig", "versions": "6+",
+      "about": "static configs from the broker's server.properties and default value.", "fields": [
+      { "name": "Name", "type": "string", "versions": "6+",
+        "about": "The config name." },
+      { "name": "Value", "type": "string", "versions": "6+", "nullableVersions": "6+",
+        "about": "The config value. Null if the config is sensitive." }
+    ]}
   ]
 }

// Version 1 adds Zk broker epoch to the request if the broker is migrating from Zk mode to KRaft mode.
// Version 2 adds the PreviousBrokerEpoch to the request for the KIP-966
// Version 3 is the same as version 2 (new field in request).
// Version 4 is the same as version 2 (new field in request).
// Version 5 is the same as version 2 (new field in request).
// Version 6 is the same as version 2 (new field in request).
{
  "apiKey": 62,
  "type": "response",
  "name": "BrokerRegistrationResponse",
+  "validVersions": "0-6",
-  "validVersions": "0-5",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
      "about": "Duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "ErrorCode", "type": "int16", "versions": "0+",
      "about": "The error code, or 0 if there was no error." },
    { "name": "BrokerEpoch", "type": "int64", "versions": "0+", "default": "-1",
      "about": "The broker's assigned epoch, or -1 if none was assigned." }
  ]
}

BrokerRegistrationRecord

We propose updating RegisterBrokerRecord to Version 5. This version introduces a new tagged field StaticConfigs, which is a collection of BrokerStaticConfig objects. This allows the Controller to persist the broker's reported static settings directly within the metadata log.

diff --git a/metadata/src/main/resources/common/metadata/RegisterBrokerRecord.json b/metadata/src/main/resources/common/metadata/RegisterBrokerRecord.json
index b7db680cbd..5570cbd96f 100644
--- a/metadata/src/main/resources/common/metadata/RegisterBrokerRecord.json
+++ b/metadata/src/main/resources/common/metadata/RegisterBrokerRecord.json
@@ -17,11 +17,12 @@
 // Version 2 adds IsMigratingZkBroker
 // Version 3 adds LogDirs
 // Version 4 adds CordonedLogDirs
+// Version 5 adds StaticConfigs for broker static config reporting.
 {
   "apiKey": 0,
   "type": "metadata",
   "name": "RegisterBrokerRecord",
-  "validVersions": "0-4",
+  "validVersions": "0-5",
   "flexibleVersions": "0+",
   "fields": [
     { "name": "BrokerId", "type": "int32", "versions": "0+", "entityType": "brokerId",
@@ -61,6 +62,16 @@
     { "name": "LogDirs", "type":  "[]uuid", "versions":  "3+", "taggedVersions": "3+", "tag": 0,
       "about": "Log directories configured in this broker which are available." },
     { "name": "CordonedLogDirs", "type":  "[]uuid", "versions":  "4+", "taggedVersions": "4+", "tag": "1",
-      "about": "Log directories that are cordoned." }
+      "about": "Log directories that are cordoned." },
+    { "name": "StaticConfigs", "type": "[]BrokerStaticConfig", "versions": "5+",
+      "taggedVersions": "5+", "tag": 2,
+      "about": "Non-default static configs reported by the broker.", "fields": [
+      { "name": "Name", "type": "string", "versions": "5+",
+        "about": "The config name." },
+      { "name": "Value", "type": "string", "versions": "5+", "nullableVersions": "5+",
+        "about": "The config value. Null if the config is sensitive." },
+    ]}
   ]
 }

Compatibility, Deprecation, and Migration Plan

When admin upgrade or downgrade to specific metadata version, if there is any illegal config, it will return InvalidConfigError.

Test Plan

Add unit test and integration test to cover new feature.

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.