You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Status

Current state: WIP

Discussion thread:

JIRA: 

Motivation

Currently, all cluster configuration is managed via property files read by the brokers on startup. There is no mechanism to change broker configuration without having to do a rolling restart of the entire cluster. Additionally, there is no way to model configuration per-client i.e. quotas, permissions etc.

This proposal attempts to build a unified mechanism for modeling configuration across various entities like topics, clients and brokers.

Public Interfaces

  • New Tooling will be added in AdminTools to change client and broker configuration. This will be similar to the topic config tooling already present.

There will be new zookeeper paths under "config" but they are not considered to be public interfaces.

Proposed Changes

This proposal will reuse pieces of the TopicQuotaManager. Similar to topic configs, the proposal is to add a BrokerQuotaManager and ClientConfigManager. These classes will listen to Zk notifications from a certain path and apply the notifications within the brokers on a per-topic, broker or clientId basis. These approaches can share significant amounts of code amongst themselves. The only thing that is different is the entity that is being modeled.

ZNode Structure

 

There should be 3 paths within config
/config/clients/<client_id>
/config/topics/<topic_name>
/config/brokers/<broker_id>
 
ZNode internally can look like this.
{"version": x, "config" : {X1=Y1, X2=Y2..}
All config changes will be sequential znodes under
/config/clients/__config_changes/config_change_XX
/config/brokers/__config_changes/config_change_XX
/config/topics/__config_changes/config_change_XX

 

The notification znodes are intentionally separate from the one used currently by TopicConfigManager. This is to avoid processing each others changes and more importantly for safety (can't accidentally change topic configs instead of client configs etc..) Upon startup, all brokers will load all the configs from zookeeper. Here is the workflow for change a config of any type:

  • Create a Config znode or modify an existing config. Since brokers are not watching every single path within config/, a sequential znode needs to be created under the appropriate __config_changes directory. This will trigger

We can add tooling within AdminUtils to update client configuration on the fly.

NOTE: This brings a nice consistency with how we do dynamic topic based configs currently. There is some overlap with KIP-5 but that is more focussed on internal broker configs whereas this is entity based (topic and clients). Additionally, IMO KIP-5 is already a very large feature and it may be easier to tackle these issues separately.

Modeling Default values

In addition to client/broker level overrides, we also need a mechanism to model default configuration. For example: say we have default quotas for all clients and we need to selectively override them on a per-client basis. The above approach doesn't provide a way to specify the default values. We can treat the defaults as a special znode 

The properties in this znode are applicable to all clients, topics and brokers respectively. Changing something here should affect all entities unless overridden explicitly.
 
/config/clients/__default
/config/topics/__default
/config/brokers/__default

Compatibility, Deprecation, and Migration Plan

  • TopicConfigManager has a config_change_XX sequential znode under configs/. The configuration changes need to be moved to a directory under configs/clients to be completely separate.
  • When a server has been restarted it will only listen to change notifications from the new path. Config changes made while some brokers haven't been fully upgraded will be lost.

Rejected Alternatives

 

  • No labels