...

No public interface changes or new configuration options are required for this KIP.

Four new mechanisms may be provided for the client/broker configuration sasl.mechanism (for client and inter-broker client connections) and the broker configuration sasl.enabled.mechanisms. The new mechanism names are SCRAM-SHA-224, SCRAM-SHA-256, SCRAM-SHA-384 and SCRAM-SHA-512. Any combination of SCRAM mechanisms can be enabled in the broker along with existing mechanisms if required.

Since support for SASL/SCRAM servers and clients is not available in Java, a new login module class will be added that loads and installs the SASL server and client implementations for SCRAM as Java security providers (similar to the existing SASL/PLAIN server support in Kafka). SCRAM is enabled by specifying one of the SCRAM mechanisms as the SASL mechanism (eg. sasl.mechanism=SCRAM-SHA-256) along with the new login module in the JAAS configuration. The login module and the underlying implementations can be overridden if required, for example, to integrate with existing authentication servers.

...

// SCRAM credentials for user alice: Zookeeper persistence path /config/users/alice
{
        "version":1,
        "config": {
          "scram-sha-512" : "s=djR5dXdtZGNqamVpeml6NGhiZmMwY3hrbg==,t=sb5jkqStV9RwPVTGxG1ZJHxF89bqjsD1jT4SFDK4An2goSnWpbNdY0nkq0fNV8xFcZqb7MVMJ1tyEgif5OXKDQ==,k=...,i=4096",
          "scram-sha-256" : "s=10ibs0z7xzlu6w5ns0n188sis5,t=+Acl/wi1vLZ95Uqj8rRHVcSp6qrdfQIwZbaZBwM0yvo=,k=nN+fZauE6vG0hmFAEj/49+2yk0803y67WSXMYkgh77k=,i=4096"
        }
};

...

For each supported mechanism, a new property is added with the mechanism name in lower case. The value of the property is a comma-separated list of key-value pairs similar to SCRAM messages and has the following elements:

• s=<Salt>
• t=<StoredKey>
• k=<ServerKey>
• i=<Iterations>

Tools

kafka-configs.sh will be extended to support management of credentials in Zookeeper as dynamic properties of users. For ease of use, the tool will take a password and an optional iteration count and generate a random salt, ServerKey and StoredKey as specified in in RFC 5802. For example:

...

Default iteration count will be 4096. The actual password "alice-secret" is not stored in Zookeeper and is not known to Zookeeper or Kafka brokers. The hashed properties stored in Zookeeper can be retrieved using the --describe option of kafka-configs.sh. See Credential configuration in Zookeeper for the format of the property persisted in Zookeeper.

For example:

bin/kafka-configs.sh --zookeeper localhost:2181 --describe --entity-type users --entity-name alice

...

One integration test and a system test will be added to test the good path for SASL/SCRAM. A system test will also be added for the upgrade scenario to test rolling upgrade and multiple broker mechanisms that include SCRAM. Unit tests will be added for failure scenarios and to test all supported SCRAM mechanisms.

Rejected Alternatives

Bump up the version of SaslHandshakeRequest to indicate support for new mechanisms

It was suggested during KIP-43/KIP-35 discussions that SaslHandshakeRequest version could be updated when new mechanisms are added to enable client implementors to choose a newer mechanism when connecting to new versions of brokers. After discussions, it was decided that the increase in handshake request version doesn't add value since clients have to use handshake requests to determine if a mechanism has been enabled in the broker and handle the case where a mechanism has been disabled. On failure, clients can indicate which mechanisms are enabled in the broker. At the moment, there are no client implementations or requirements to handle fallback mechanisms and clients fail with an error when the client mechanism is not enabled in the broker.

Specify username, password as Kafka client properties instead of the JAAS configuration 

...