DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Kafka's controller currently has no visibility into each broker's static configuration. When a broker registers with the controller via BrokerRegistrationRequest, it reports its supported feature versions, listeners, rack, and log directories, but not its configuration values (i.e., the settings defined in server.properties). This creates a fundamental gap that blocks two categories of improvements:
This gap is not hypothetical — it has already produced observable technical debt in trunk. [KAFKA-20544] tracks the refactor of cordoned.log.dirs validation in ConfigurationControlManager, whose description explicitly states that the cleanup depends on making static configurations available in the controller. Today, because the controller cannot see each broker's log.dirs, validating cordoned.log.dirs ⊆ log.dirs requires a resource-specific code path inside ConfigurationControlManager and a forwarded flag plumbed through the Controller interface — a workaround that the standard ConfigDef validator path should have made unnecessary. KAFKA-20544 is concrete evidence that the missing static configuration data is already shaping controller code in undesirable ways, and removing it is the prerequisite for that cleanup.
Aligning incrementalAlterConfigs validation (KIP-1256)
...
This KIP addresses the root cause shared by both problems: the controller lacks broker static configuration data. We propose that brokers include their static configurations in BrokerRegistrationRequest, and that the controller persists this information in RegisterBrokerRecord so it survives controller failovers. This KIP intentionally KIP intentionally contains no validation logic — it provides only the infrastructure that KIPthat KIP-1256 and KIP-1294 build upon.
...
- We bump the versions of RegisterBrokerRecord, BrokerRegistrationRequest, BrokerRegistrationResponse, the old record and RPC version still support.
Test Plan
- Integration test: End-to-end broker registration with static configs in a KRaft cluster.
- Unit test
...
- : BrokerLifecycleManager populates StaticConfigs with only configs that have a ConfigDef validator, and excludes PASSWORD-type configs.
- Unit test: ClusterControlManager.replay() populates the in-memory BrokerRegistration with static configs from the record.
Rejected Alternatives
Only report configs with non-default values - During a rolling upgrade, different brokers may be running different Kafka versions with different defaults for the same config. A broker running an older version may omit a config because it matches its own default, but that default may differ from the controller's version. The controller cannot reliably fill in the "correct" default because it only knows its own version's defaults, not the defaults of every broker version in the cluster.
- Report all non-sensitive config -
Reporting all non-sensitive configs (~388 entries, ~18 KB per broker) was considered. While simpler in filtering logic, it includes 163 configs that have no ConfigDef validator. The additional ~8 KB per broker adds up in large clusters (8 MB for 1000 brokers) with no practical benefit — configs without validators cannot be validated regardless of whether they are reported. The chosen approach (only configs with validators, ~225 entries, ~10 KB per broker) keeps the payload smaller and automatically expands when new validators are added in future KIPs.