...

Making javax.net.ssl.SSLContext setup pluggable provides flexibility for providing Key Material, Secure Random implementation and configure Key/Trust managers in a custom way. Example: Apache HttpComponents and Netty SslContextBuilder. 

Hence we feel we should make SSLContext initialization pluggableHowever, Kafka configures the SSLEngine for Client and Server both modes. Hence according to existing code it would be useful to make SslEngineBuilder pluggable. That will provide us a way to configure SSLContext object in a flexible way and at the same time will allow creation of SSLEngine with Client/Server mode.

Public Interfaces

New configuration

ssl.contextengine.factory.class - This configuration will take class of the below interface's type and will be used to create javax.net.ssl.SSLContext SSLEngine object.

Default value will be as mentioned below.

finalpublic static final String SSL_CONTEXTENGINE_FACTORY_CLASS_CONFIG = "ssl.contextengine.factory.class";
finalpublic static final String DEFAULT_SSL_CONTEXTENGINE_FACTORY_CLASS = "org.apache.kafka.common.security.ssl.DefaultSslContextFactoryDefaultSslEngineFactory";

Interface for

...

SslEngineFactory

Below is the interface suggested for this.

package org.apache.kafka.common.security.ssl;

import org.apache.kafka.common.network.Mode;

import javax.net.ssl.SSLContextSSLEngine;
import java.security.KeyStore;
import java.util.Map;
import java.util.Set;

public interface SslContextFactorySslEngineFactory {

    /**
     * ReturnsCreate SSLContexta loadednew bySSLEngine this factoryobject.
     *
     * @return  @param mode                   Whether to use client or server mode.
     */
 @param peerHost        SSLContext createSSLContext();

    /**
  The peer host *to Returnsuse. theThis currentlyis used configurations by this objectin client mode if endpoint validation is enabled.
     * @return
 @param peerPort               */
    Map<String, Object> configs();

    /**The peer port to use. This is a hint and not used for validation.
     * Returns@param theendpointIdentification reconfigurableEndpoint configsidentification usedalgorithm byfor thisclient objectmode.
     * @return The new SSLEngine.
     */
    Set<String>SSLEngine reconfigurableConfigs(createSslEngine(Mode mode, String peerHost, int peerPort, String endpointIdentification);

    /**
     * Returns true if theSSLEngine SSLContext needs to be rebuilt.
     *
     * @param nextConfigs       The configuration we want to use.
     * @return                  True only if the SSLContextSSLEngine should be rebuilt.
     */
    boolean shouldRebuiltForshouldBeRebuilt(Map<String, Object> nextConfigs);

    /**
     * Returns the names of configs that may be reconfigured.
     */
    Set<String> reconfigurableConfigs();

    /**
     * Returns existing configuration.
     */
    Map<String, Object> configs();

    /**
     * Returns keystore.
     * @return
     */
    KeyStore keystore();

    /**
     * Returns truststore.
     * @return
     */
    KeyStore truststore();
}

Proposed Changes

Currently SslFactory.java uses SslEngineBuilder.java. Instead of that we will modify SslFactory.java to load a class configured via the new configuration 'ssl.engine.factory.class' and delegate the SSLEngine creation call to the implementation. 

...

• SslEngineBuilder.java (functionality of SSLContext loading will be moved to DefaultSslContextFactoryDefaultSslEngineFactory.java and createEngine() method will move to SslFactory.java)

Which classes will be added?

• SslContextFactorySslEngineFactory.java Interface
• DefaultSslContextFactoryDefaultSslEngineFactory.java (mostly having code from existing SslEngineBuilder)

Which classes will be modified primarily?

• SslFactory.java
• Will host the createEngine() method from current SslEngineBuilder and will have mechanism to load the SslContextFactory implementation

How does configs get to the implementation class?

The configuration of Map will be passed to the implementation class via the constructor. See below example,

public DefaultSslContextFactoryDefaultSslEngineFactory(Map<String, ?> configs) {
...
...
}

...

These configuration will be passed from SslFactory to the implementation of the SslContextFactory SslEngineFactory interface via reflection like below

public class SslFactory implement Reconfigurable {
...
...
     sslContextFactoryClasssslEngineFactoryClass.getDeclaredConstructor(Map.class).newInstance(configs);
...
}

Sequence Diagrams for important interactions

SslFactory and SslContextFactory Interaction

Image Removed

Support for reconfiguration of custom configs

By custom configs we mean the configs used by the SslContextFactorySslEngineFactory's implementation. Those configs does not have to be part of definition of Kafka configs since only the implementation class knows what are those. Kafka already supports custom configs so this should not be a new challenge. 

Other challenge

Currently reconfigurations are pushed from Kafka code base to the reconfigurable classes. However, depending upon SslContextFactory's implementation we could have some events/changes detected by the implementation first and we would need to trigger reconfiguration on SslFactory in order to get re-initialized!

Probably this could be achieved by passing listener to those implementation changes but this needs to be further explored

Compatibility, Deprecation, and Migration Plan

...

Not applicable since old code behavior will be kept with default implementation of DefaultSslContextFactory DefaultSslEngineFactory and modification to SslFactory class.

...

There will be good amount of state in the SslContextFactorySslEngineFactory's implementation (as it will be similar to the current SslEngineBuilder class). We believe that making SSLContext creation and SSLEngine object's configuration pluggable is worth to allow SSL experts to write their own implementation having the SSL domain knowledge and keep them free of knowing much about Kafka's reconfigurability - example: Apache HttpComponents. We prefer SslFactory class to do what it is doing right now and keep the responsibility of re-creating underlying SslContextFactory SslEngineFactory object based on the configurations specified by the SslContextFactory's implementation.

Creating builder for SSLContext

Creating We could create a builder for SSLContext object and have a mechanism to configure SSLEngine object for client/server mode non-pluggable. However, creating a builder interface with options to build SSLContext will need to have method(s) to allow keys/trusted-certs. It will also require us to have 'key-password' as input for the keystore. In the current Kafka implementation it requires the password to be configured in the plaintext via 'ssl.key.password', 'ssl.keystore.password' and 'ssl.truststore.password'. If we need to customize how the password is loaded, due to security reasons, this approach will not work since some other mechanism for making password pluggable (See KIP-76 Enable getting password from executable rather than passing as plaintext in config files AND KIP-486: Support custom way to load KeyStore and TrustStore) need to be devised which will add more ssl related configurations to Kafka.

...