...

This KIP addresses the following extensions to the existing implementation:

1. Configurable SASL implementation mechanism to enable integration with existing authentication servers
2. Support for additional standard SASL mechanisms . This enables standard mechanisms that do not have a default implementation in Kafka (eg. DIGEST-MD5) as well as custom mechanismsConfigurable callback handlers to provide mechanism-specific inputwhich do not require custom callbacks or login interfaces to support token refresh
3. SASL/PLAIN implementation to enable simple username/password authentication without complex infrastructure
4. Support for multiple SASL mechanisms within the same server. This will be useful, for example, in organizations where internal and external users require different authentication mechanisms.

This KIP does not address the following extensions which be considered in a follow-on KIP:

1. Support for custom mechanisms
2. Configurable callback handlers to provide mechanism-specific input
3. Configurable login interface that manages the login process and the lifecyle of login context related resources to support custom mechanisms that require periodic ticket refresh

...

1. sasl.mechanism (String) SASL mechanism used for client connections. This may be any mechanism for which a security provider is available in the JVM. Default value is GSSAPI.
2. sasl.enabled.mechanisms (List<String>) The list of SASL mechanisms enabled in the Kafka server. This may include any mechanism for which a security provider is available in the JVM. Default value is GSSAPI.
3. sasl.callback.handler.class  (Class). The fully qualified name of a class that implements the AuthCallbackHandler interface. This class should have a default constructor and implement the callback handlers required for the configured mechanisms. Default implementation supports callback handlers for GSSAPI and PLAIN.
4. sasl.login.class  (Class). The fully qualified name of a class that implements the Login interface. This class should have a default constructor.

Callback handler interface

...

...

...

package org.apache.kafka.common.security.auth;
import org.apache.kafka.common.annotation.InterfaceStability;
import org.apache.kafka.common.network.Mode;
import org.apache.kafka.common.KafkaException;
import javax.security.auth.callback.CallbackHandler;
/*
 * Callback handler for SASL-based authentication
 */
@InterfaceStability.Unstable
public interface AuthCallbackHandler extends CallbackHandler {
    
    /**
     * Configures this callback handler.
     * 
     * @param configs Configuration
     * @param mode The mode that indicates if this is a client or server connection
     * @param subject Subject from login context
     * @param saslMechanism Negotiated SASL mechanism
     */
    void configure(Map<String, ?> configs, Mode mode, Subject subject, String saslMechanism) throws KafkaException;
    /**
     * Closes this instance.
     */
    void close() throws KafkaException;
}

Login interface

package org.apache.kafka.common.security.auth;
import java.util.Map;
import javax.security.auth.Subject;
import javax.security.auth.login.LoginContext;
import javax.security.auth.login.LoginException;
/**
 * Login interface for authentication
 */
public interface Login {    
    /**
     * Configures this login instance.
     */
    void configure(Map<String, ?> configs, String loginContextName);
    /**
     * Performs login for each login module specified for the login context of this instance.
     */
    LoginContext login() throws LoginException;    
    /**
     * Returns the authenticated subject of this login context.
     */
    Subject subject();    
    /**
     * Returns the service name
     */
    String serviceName();
    /**
     * Closes this instance.
     */
    void close();
}

Proposed Changes

The changes will be implemented under KAFKA-3149. The implementation will include changes to support additional mechanisms and enable multiple mechanisms in the broker. A simple default implementation for SASL/PLAIN in Kafka that is currently in KAFKA-2658 will also be included. This section gives a summary of the changes and the rationale behind them.

...

All the four types of configuration above need to be configured consistently for SASL authentication to operate correctly. 1) and 4) are JVM configuration options. 2) and 3) are currently hard-coded in Kafka. The proposed changes enable flexible enable  configuration for 2) and 3)   to enable any more SASL mechanism mechanisms to be supported in Kafka clients and servers. 3) will be addressed in a follow-on KIP to support any mechanism including custom mechanisms.

SASL mechanism

Client connections will use the property sasl.mechanism to specify the mechanism to be used for SASL authentication. Kafka servers may also specify the configuration option sasl.enabled.mechanisms to provide the list of enabled mechanisms when multiple mechanisms are enabled in the server. In order to plug in any SASL mechanism including custom mechanisms, mechanism will be specified as String rather than an enum with a restricted set of values.  GSSAPI will be used as the default mechanism for interoperability of new clients with 0.9.0.0 brokers. Clients may enable only one mechanism and the mechanism name is sent to the server before any SASL authentication packets are sent, if the mechanism is not GSSAPI. Server fails the authentication if the client mechanism is not enabled in the broker.

SASL callback handler

Kafka server and client will have a new configuration option sasl.callback.handler.class to provide a callback handler class. Default callbacks are included in Kafka for the mechanisms which have an implementation in Kafka (GSSAPI and PLAIN). Default client callback handler obtains authentication id and password from the public and private credentials of the Subject respectively as String values. Configurable callbacks will enable other mechanisms to be used with Kafka without any changes to Kafka code. The callback handler interface has been derived from the implementation for GSSAPI and PLAIN mechanisms. Apart from Subject, configuration properties and mechanism are provided to the callback object to enable custom mechanisms to be added without any changes to Kafka.

SASL properties

The current implementation does not specify any properties when the SaslClient or SaslServer is constructed. To make the Kafka implementation flexible with pluggable mechanisms, all properties specified for Kafka client/server will be passed to the SaslClient/SaslServer. These include all properties specified by the user including properties not defined in Kafka, so that additional properties can be added without changes to Kafka.

Login interface

Kafka server and client will have a new configuration option sasl.login.class to manage the login process. This is required to plugin custom mechanisms which require login tokens to be refreshed periodically. Default login implementation will support GSSAPI and PLAIN and will be suitable for most of the standard mechanisms where mechanism-specific code can reside in the login module implementation. This interface is targeted at protocols similar to Kerberos which require a background thread to handle ticket refresh.

Support for multiple mechanisms in a broker

...

Client flow:

1. If sasl.mechanism is not GSSAPI, send a packet with the mechanism name to the server. Otherwise go to Step 3.
   • Packet Format: | Version (Int16) | Mechanism (String) |
2. Wait for response from the server. If the error code in the response is non-zero, indicating failure, report the error and fail authentication.
3. Perform SASL authentication with the configured client mechanism

Server flow:

...

SASL/PLAIN is a simple username/password authentication mechanism that is typically used with TLS for encryption to implement secure authentication. Unlike Kerberos, PLAIN does not require complex authentication infrastructure. Adding a default implementation for PLAIN in Kafka enables a simpler authentication mechanism for organizations which do not already use Kerberos. SASL/PLAIN protocol and its uses are described in https://tools.ietf.org/html/rfc4616..
The PR in KAFKA-2658 will be rebased on the extensible interface from this KIP for this implementation.

Testing

• Due to the complexity of setting up Kerberos, limited unit testing has been implemented for SASL in the clients project. Along with the implementation for SASL/PLAIN, comprehensive unit tests will be added for the existing SASL implementation as well as the new interfaces. 

• End-to-end tests will be added in the core project along with the existing SASL/Kerberos tests for SASL/PLAIN

  ,

  and multi-mechanism configuration

  and to test an additional mechanism using the extension points.

  
  

• System tests will be added for SASL/PLAIN and for multi-mechanism support.
  

...

Since the security requirements and infrastructure used in different organizations vary, default implementation of login modules and security providers and unlikely to be sufficient for all users. Unlike Kerberos, where most users are likely to use the Kerberos module provided in the JDK, other mechanisms are likely to be customized by users to enable integration with existing authentication providers. The proposed implementation removes along with a follow-on KIP to configure callbacks will remove the restriction that a SASL mechanism without a default implementation in Kafka cannot be used at all.

...