...

1. sasl.mechanism (String) SASL mechanism used for client connections. This may be any mechanism for which a security provider is available in the JVM. Default value is GSSAPI.
2. sasl.enabled.mechanisms (List<String>) The list of SASL mechanisms enabled in the Kafka server. This may include any mechanism for which a security provider is available in the JVM. Default value is GSSAPI.

Protocol changes

A new Kafka SaslHandshakeRequest and response type will be defined to add a handshake request flow to enable clients to communicate their chosen SASL mechanism to the broker. The formats of the request and response are shown below:

SaslHandshakeRequest => Mechanism
  Mechanism => string

SaslHandshakeResponse => ErrorCode EnabledMechanisms
  ErrorCode => int16
  EnabledMechanisms => [string]

Successful Kafka SaslHandshakeRequest/Response flow should be immediately followed by the actual SASL authentication packets using the selected SASL mechanism. SASL authentication exchange consists of opaque client and server tokens as defined by the SASL mechanism and are typically obtained using a standard SASL library. These packets are not prefixed with Kafka request/response headers. No further Kafka requests may be sent until SASL authentication exchange is completed. For interoperability with 0.9.0.x, the Kafka handshake request flow is omitted for inter-broker communication using SASL for connections to older brokers.

Proposed Changes

The changes will be implemented under KAFKA-3149. The implementation will include changes to support additional mechanisms and enable multiple mechanisms in the broker. A simple default implementation for SASL/PLAIN in Kafka that is currently in KAFKA-2658 will also be included. This section gives a summary of the changes and the rationale behind them.

...

To keep the Kafka protocol simple, only a single SASL mechanism is allowed for Kafka clients. For protocols other than GSSAPI, client sends the protocol Clients send the mechanism name to the server in a handshake request before any SASL authentication packets are sent. Kafka server checks the first packet and if it is one of the mechanism names that it recognizes including custom protocol mechanism names, it switches to that mechanism and starts SASL authentication using subsequent packets. If not, the server defaults to GSSAPI mechanism, treating the first packet as the first GSSAPI authentication packet from the client.With GSSAPI, the first context establishment packet starts with byte 0x60 (APPLICATION-0 tag) followed by a variable-length encoded size. The first Kafka handshake request packet containing client mechanism starts with a two-byte API key of 0x0011, making it easy to distinguish from a GSSAPI packet. The Kafka Java client implementation skips handshake request flow for GSSAPI for inter-broker connections if inter.broker.procotolprotocol.version is 0.9.0.x, enabling rolling upgrade of 0.9.0.x clusters which use SASL for replication. Handshake requests are sent by other clients even for GSSAPI from 0.10.0.0 onwards.

With GSSAPI, the first context establishment packet starts with byte 0x60 (APPLICATION-0 tag) followed by a variable-length encoded size. The first Kafka handshake request packet containing client mechanism starts with a two-byte API key of 0x0011, making it easy to distinguish from a GSSAPI packet.

Client flow:

1. If sasl.mechanism is not GSSAPI, send a Kafka handshake request packet with the mechanism name to the server. Otherwise go to Step 3.

...

1. • Request Format: |

...

1. • Kafka RequestHeader | Kafka SaslHandhsakeRequest |
2. Wait for response from the server. If the error code in the response is non-zero, indicating failure, report the error and fail authentication.
3. Perform SASL authentication with the configured client mechanism. SASL authentication packets do not contain a Kafka RequestHeader.
   • Client token Format: | Size (int16) | SASL client authentication token |

Server flow:

1. Wait for first authentication packet from client
2. If this packet is a not valid Kafka handshake request, go to Step 4 and process this packet as the first GSSAPI client token
3. If the client mechanism in the Kafka handshake request received in Step 2 is enabled in the broker, send a response with error code zero and start authentication using the specified mechanism. Otherwise, send an error response including the list of enabled mechanisms and fail authentication.
   
   • Packet Response Format: | KafkaResponseHeader | ErrorCode (Int16) | EnabledMechanisms (ArrayOf(String)) Kafka ResponseHeader | Kafka SaslHandhsakeResponse |
4. Perform SASL authentication with the selected mechanism. If mechanism exchange was skipped, process the initial packet that was received from the client first. SASL authentication packets are expected without a Kafka RequestHeader until SASL authentication exchange completes. SASL server authentication packets are sent back without a Kafka response header.
   
   • Server token Format: | Size (int16) | SASL server authentication token |

Step 3 in the client flow and Step 4 in the server flow correspond to the start of the actual SASL authentication exchange when authentication tokens are exchanged for the selected SASL mechanism. These are treated as opaque tokens and are processed by the SASL provider of the mechanism. No further Kafka requests may be sent until the authentication exchange completes.

SASL/PLAIN implementation

...