Status

Current state: "Under Discussion"

...

I propose we add an option --add-metadata to the kafka-storage tool that will add an ApiMessageAndVersion records directly into the __cluster_metadata topic. (See KIP-801 for details on __cluster_metadata ). The option can be applied multiple times to the format command so that multiple SCRAM records can be added to bootstrap the cluster. Below is the new updates kafka-storage command.

./bin/kafka-storage.sh format -h 
usage: kafka-storage format [-h] --config CONFIG --cluster-id CLUSTER_ID [--add-metadata METADATA] [--release-version RELEASE_VERSION] [--ignore-formatted] 

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG, -c CONFIG 
                        The Kafka configuration file to use. 
  --cluster-id CLUSTER_ID, -t CLUSTER_ID
                        The cluster ID to use. 
  --add-metadata METADATA, -A METADATA 
                        Some METADATA Message to add to the __cluster_metadata log for this node e.g. 
                        'UserScramCredentialsRecord={"Namename":"alice","Mechanismmechanism":1,"Saltsalt":"MWx2NHBkbnc0ZndxN25vdGN4bTB5eTFrN3E=","SaltedPassword":"mT0yyUUxnlJaC99HXgRTSYlbuqa4FSGtJCJfTMvjYCE=","Iterations":8192}' 
  --release-version RELEASE_VERSION, -r RELEASE_VERSION
                        A KRaft release version to use for the initial metadata version. 
  --ignore-formatted, -g

 I I propose the METADATA argument will contain the type of the ApiMessageAndVersion record to be added followed by a JSON set of name value pairs to populate the record. Initially I will only add support for UserScramCredentialsRecord which will require arguments of a Name "name" (which is implicitly type user), a "mechanism" which is 1 for SCRAM-SHA-256 or 2 for SCRAM-SHA-512, the Salt"salt", a Password password or SaltedPassword"saltedPassword", and "iterations" for the Iteration iteration count. The salt and iteration count are not optional unlike in the kafka-config command used to talk with ZK. We want the record data to match exactly on each node and normally when a salt is not specified a random salt is chosen. Since each node is initialized individually this would result in records with different salts which we don’t want.

Compatibility, Deprecation, and Migration Plan

...