KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum (Accepted)
Current state: Approved.
Discussion thread: here
JIRA: KAFKA-9082
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
With work underway to migrate configuration metadata away from ZooKeeper (KIP-500), components and tools relying on direct ZK access must be updated to issue administrative calls through AK's broker via KafkaAdminClient. ConfigCommand, a tool used for describing and altering configs for various entities (topics, users, clients, brokers, and broker-loggers), is one such tool requiring update. Partial AK support has been implemented, however additional functionality is necessary to maintain ZK-parity. This KIP proposes to relax the --zookeeper restriction for all afflicted commands+flags.
In addition, the ConfigCommand tool would benefit from quality-of-life change to shorten entity specification on the command line.
There are 3 major functions of the config command:
Which functions are supported, and which are proposed by this KIP (green) is as follows:
ConfigCommand | ZK current | AK current | AK proposed |
---|---|---|---|
Topics: alter specific | + | - | + |
Topics: describe specific | + | - | + |
Topics: describe all | + | - | + |
Users/clients: alter specific | + | - | +* |
Users/clients: describe specific | + | - | +* |
Users/clients: describe all | + | - | +* |
Brokers: alter specific | + | + | + |
Brokers: describe specific | + | + | + |
Brokers: describe all | + | - | + |
Broker loggers: alter specific | - | + | + |
Broker loggers: describe specific | - | + | + |
Broker loggers: describe all | - | - | + |
*AK broker-side functionality requiring future KIP
For the quality-of-life change, proposed is the shortening of specifying entity type and name into a single flag:
--entity-type <TYPES> --entity-name <NAME>
can be shortened to --<TYPE> <NAME>
--entity-type topics --entity-name some-topic
can be shortened to --topic some-topic
--entity-type <TYPES> --entity-default
can be shortened to --<TYPE>-defaults
--entity-type brokers --entity-default
can be shortened to --broker-defaults
Note --entity-type, --entity-name, and --entity-default will NOT be deprecated/removed.
Public Interfaces
Summary of the public interfaces proposed. Note these are all additions, no existing commands/flags are removed or changed.
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type brokers <note --entity-name omitted>
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type broker-loggers <note --entity-name omitted>
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type topics --entity-name some-topic
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type topics <note --entity-name omitted>
bin/kafka-configs.sh --bootstrap-server x --alter --entity-type topics --entity-name topic <--add-config/--delete-config ...>
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-default
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user --entity-type clients --entity-name some-client
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user --entity-type clients --entity-default
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name default --entity-type clients --entity-name some-client
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name default --entity-type clients --entity-default
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients --entity-name some-client
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients --entity-default
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users <note --entity-name omitted>
bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients <note --entity-name omitted>
bin/kafka-configs.sh --bootstrap-server x --alter <same enumerations as user/client descriptions above>
--entity-type topics --entity-name some-topic => --topic some-topic
--entity-type users --entity-name some-user => --user some-user
--entity-type users --entity-default => --user-defaults
--entity-type clients --entity-name some-client => --client some-client
--entity-type clients --entity-default => --client-defaults
--entity-type brokers --entity-name 10 => --broker 10
--entity-type brokers --entity-default => --broker-defaults
--entity-type broker-loggers --entity-name 10 => --broker-logger 10
There is no impact to current usage, the semantics of all existing combinations of flags remain unchanged.
None.