This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state:  [One of "Under Discussion", "Accepted", "Rejected"]Under discussion

Discussion thread: TODOhere [Change the link from the KIP proposal email archive to your own email thread]

JIRA: TODOhere [Change the link from KAFKA-1 to your own ticket]

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

...

With work underway to migrate away from ZooKeeper (KIP-500), components and tools relying on direct ZK access must be updated to issue administrative calls through AK's broker via KafkaAdminClient. ConfigCommand, a tool used for describing and altering configs for various entities (topics, users, clients, brokers, and broker-loggers), is one such tool requiring update. Partial AK support has been implemented, however additional functionality is required to maintain ZK-parity. This KIP proposes to relax the --zookeeper restriction on a subset of commands described below.

In addition, the ConfigCommand tool would benefit from quality-of-life change to shorten the specifying of entities into a single flag on the command line.

Proposed Changes

There are 3 major functions of the config command:

1. Alter an entity's config by adding/updating/removing config entries. The given entity name ma either be a specific entity (via --entity-name) or, if supported, the default config given by the default entity (via --entity-default).
2. Describing an entity's config. Like altering, the given entity name may either be a specific entity (via --entity-name) or, if supported, the default config given by the default entity (via --entity-default).
3. Describing all entity configs. This happens when no identity identifier is provided (i.e. no --entity-name and no --entity-default).

Support for the following commands is proposed to be added:

*AK broker-side functionality requiring future KIP

For the quality-of-life change, proposed is the shortening of specifying entity type and name:

• --entity-type <TYPES> --entity-name <NAME> can be shortened to --<TYPE>-name <NAME>
  • E.g. --entity-type topics --entity-name some-topic can be shortened to --topic-name some-topic
• --entity-type <TYPES> --entity-default can be shortened to --<TYPE>-default
  • E.g. --entity-type brokers --entity-default can be shortened to --broker-default

Public Interfaces

Summary of the public interfaces proposed (all additions):

• Support all broker descriptions for AK:
  • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type brokers <note --entity-name omitted>
• Support all broker-logger descriptions for AK:
  • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type broker-loggers <note --entity-name omitted>
• Support topic config commands for AK:
  • Specific topic description:
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type topics --entity-name some-topic
  • All topic descriptions:
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type topics <note --entity-name omitted>
  • Altering topic config:
    • bin/kafka-configs.sh --bootstrap-server x --alter --entity-type topics --entity-name topic <--add-config/--delete-config ...>
• Support user/client config commands for AK:
  • Specific user/client descriptions:
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-default
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user --entity-type clients --entity-name some-client
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name some-user --entity-type clients --entity-default
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name default --entity-type clients --entity-name some-client
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users --entity-name default --entity-type clients --entity-default
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients --entity-name some-client
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients --entity-default
  • All user/client descriptions:
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type users <note --entity-name omitted>
    • bin/kafka-configs.sh --bootstrap-server x --describe --entity-type clients <note --entity-name omitted>
  • Altering user/client config:
    • bin/kafka-configs.sh --bootstrap-server x --alter <same enumerations as user/client descriptions above>
• Add abbreviation helper flags:
  • --entity-type topics --entity-name some-topic => --topic some-topic
  • --entity-type users --entity-name some-user => --user some-user
  • --entity-type users --entity-default => --user-default
  • --entity-type clients --entity-name some-client => --client some-client
  • --entity-type clients --entity-default => --client-default
  • --entity-type brokers --entity-name 10 => --broker 10
  • --entity-type brokers --entity-default => --broker-default
  • --entity-type broker-loggers --entity-name 10 => --broker-logger 10

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Rejected Alternatives

There is no impact to current usage, the semantics of all existing combinations of flags remain unchanged.

Rejected Alternatives

NoneIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.