...
Proposed Changes
This proposal will generalize all remove the code in TopicConfigManager to make it possible to handle any kind of config change TopicConfigManager and introduce a "DynamicConfigManager" to handle config changes for all entities via Zookeeper. Initially, we will have only 2 specific entities:
...
entities Topic and Clients.
ZNode Structure:
Code Block |
---|
There will be 2 paths within config /config/clients/<client_id> /config/topics/<topic_name> Internally, the znodes are comma-separated key-value pairs where key represents the configuration property to change. {"version": x, "config" : {X1 : Y1, X2 : Y2..}} |
...
Code Block |
---|
The notification data can be: {"version" : 1, "entity":"topic/client", "value" : "topic_name/client_id"} |
Config API
This proposal also adds broker APIs to alter and view configs for any entity. These requests can be sent to any broker within the cluster. See KIP-4 for more details on the implementation. Once work for KIP-4 completes, we can add these API's as described below.
AlterConfig
- AddedConfigEntry is an array of configs that will be added for that entity
- DeletedConfig is an array of configs that will be deleted for that entity. For deletion on the config property name needs to be specified and not the value. Also, when a config property is deleted, that entity will reapply the default value for that property. For example: if you delete the quota for a clientId, the quota for that client will be set to the default quota.
Code Block |
---|
// EntityType can be either topic or client. AddedConfigEntry and DeletedConfig will be an array.
AlterConfigRequest => [EntityType EntityName [AddedConfigEntry] [DeletedConfig]]
EntityType => string
EntityName => string
AddedConfigEntry => ConfigKey ConfigValue
ConfigKey => string
ConfigValue => string
DeletedConfig => string
AlterConfigResponse => [EntityType EntityName ErrorCode]
EntityType => string
EntityName => string
ErrorCode => int16 |
DescribeConfig
The DescribeConfig request is used to query configs for entities.
Code Block |
---|
DescribeConfigRequest => [EntityType EntityName]
EntityType => string
EntityName => string
// ConfigEntry is an array. It will be empty if there is an error. ErrorCode will be non-zero in case of error
DescribeConfigResponse => [EntityType EntityName ConfigEntry]
EntityType => string
EntityName => string
ErrorCode => int16
ConfigEntry => ConfigKey ConfigValue
ConfigKey => string
ConfigValue => string |
CLI and Interactive Shell
As described in KIP-4, these commands will have scripts and an interactive shell.
Code Block |
---|
# Topic Commands - options are ported from TopicCommand.scala
bin/kafka.sh --alter-config --entityType [topic/client] --entityName name --added-config key=value,key=value --deleted-config key,key,key --broker-list <host : port>
bin/kafka.sh --describe-config --entityType [topic/client] --entityName name --broker-list <host : port> |
Compatibility, Deprecation, and Migration Plan
...