Status

Current state: Under DiscussionAccepted

Discussion thread: Discussion, hereVoting

JIRA: 

...

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --add-config-file new.properties

Input from STDIN can be used instead of a file on disk, to do this use a hyphen instead of a file path:

grep 'complex.property' server.properties | bin/kafka-configs --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --add-config-file -

So that the add and delete usages are parallel, we will also add an option to accept a properties file and delete the properties from the file. Any property defined in the file will be deleted, regardless of what value is defined in the file.

bin/kafka-configs --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --delete-config-file old.properties

Input from STDIN can be used instead of a file on disk, to do this use a hyphen instead of a file path:

grep 'complex.property' server.properties | bin/kafka-configs --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --delete-config-file -

Public Interfaces

We will add the --add-config-file option to kafka-configs.sh. It will be mutually exclusive with --add-config.

We will add the --delete-config-file option to kafka-configs.sh. It will not be mutually exclusive with --delete-config because there is no ambiguity about precedence.

Compatibility, Deprecation, and Migration Plan

...

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --add-config 'k1={"key1":"val1","key2":"val2"},k2=[[1,2],[3,4]]'

...

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter \
  --add-config k1=v1 \
  --add-config k2=v1,v2,v3 \
  --add-config k3=v3 \
  --add-config 'k4={"key1": "val1", "key2": "val2"}' \
  --add-config 'k5=[[1,2],[3,4]]'

However this would break existing multi-key uses of kafka-configs.sh.

Allow configuration from STDIN as well as from a file

It might be nice to enable input from standard input, to enable pipe/grep workflows. However, it's not clear that such workflows would be common. A script that would use such a workflow can send the output of the pipe to a temporary file and load that instead. Since the rest of the commands in the Kafka CLI don't support specifying STDIN, we won't support that here either, for consistency. STDIN support across the entire toolset can be added as a separate KIP if the need arises.

grep 'complex.property' server.properties | bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --add-config-file -

Allow --add-config and --add-config-file at the same time

...

There is no similar ambiguity with --delete-config and --delete-config-file, since there's no value to conflict.

Include a --delete-config-file option

Some systems like Kubernetes allow the cleanup of resources by deleting with the same config that was used to create the resources. This is not an exactly parallel situation, because deleting a configuration property doesn't necessarily put the system in the same state that it was in before the addition.

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --delete-config-file old.properties

grep 'complex.property' server.properties | bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers --entity-default \
  --alter --delete-config-file -

Use a format other than a properties file for the configs

Since dynamic configurations are stored in ZooKeeper in JSON format, it might make sense to accept a JSON file in that same format. This seems like it is tying the implementation details too closely to the interface. Users already use properties files to specify configurations. Introducing a different format would add complexity.