Status

Current state: "Under Discussion"

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

It has been a common request to be able to customize SslFactory beyond the existing configuration options. For example, users want encrypted passwords, an alias to choose a certificate, OCSP support, etc. For these, alternative config options make the most sense because these are generic config options that are useful in many environments, so we should support these without having to write Java code.

There are situations where adding configuration options will never be able to satisfy the requirement. For example, the user may need to reuse an existing SSL implementation or share an implementation across multiple applications. For those cases, the the simplest approach would be to support the customization of the SslFactory implementation.

A custom SslFactory implementation might also serve as a stop gap measure when a user needs an SSL feature immediately but the necessary configuration option is not yet available.

Public Interfaces

This KIP will introduce a new interface PluggableSslFactory and a new common configuration option ssl.sslfactory

Proposed Changes

A new public interface will be defined:

public interface PluggableSslFactory extends Reconfigurable {
public SSLEngine createSslEngine(String peerHost, int peerPort);
public SSLContext sslContext();
}

A new configuration option will be added to SslConfigs:
public static final String SSL_SSLFACTORY_CONFIG = "ssl.sslfactory";
public static final String SSL_SSLFACTORY_DOC = "...";
public static final String DEFAULT_SSL_SSLFACTORY_DOC = "org.apache.kafka.common.security.ssl.SslFactory";

config.define(SslConfigs.SSL_SSLFACTORY_CONFIG, ConfigDef.Type.CLASS, SslConfigs.DEFAULT_SSL_SSLFACTORY, ConfigDef.Importance.LOW, SslConfigs.SSL_SSLFACTORY_DOC)

In SaslChannelBuilder and SslChannelBuilder, the type of the member sslFactory will be changed to PluggableSslFactory. Instead of calling the SslFactory constructor directly, the sslFactory will be created by calling Class.newInstance(...) with the same constructor arguments as before.

The class comment of the PluggableSslFactory interface must describe the required constructor taking 3 arguments: (Mode mode, String clientAuthConfigOverride, boolean keystoreVerifiableUsingTruststore)

The PluggableSslFactory implementation receives its configuration options in the call to configure(Map<String, ?> configs) as before.

Compatibility, Deprecation, and Migration Plan

This KIP is backwards compatible because the PluggableSslFactory defaults to the existing SslFactory and the API is the same.

Rejected Alternatives

Kafka could define a new configuration option to hold an instance of SSLSocketFactory. This is similar to many Java libraries that accept an instance of SSLSocketFactory. This was rejected because Kafka tries to be language neutral. It was thought it would make it more difficult to support C and Python.

Ideally, the interface should be called SslFactory and the built-in implementation should be called DefaultSslFactory. This was rejected to improve backwards compatibility for applications that call the SslFactory directly.

Kafka's naming convention does not use a special tag for interfaces. Accordingly, these names were rejected ISslFactory, SslFactoryIFace, SslFactoryInterface.

The EchoServer could be updated to use PluggableSslFactory. It is suggested to keep it the way it is to test backwards compatibility.

The constructor SslFactory(Mode) is not part of the interface contract. It is only used by the EchoServer.

It would be nice if the PluggableSslFactory implementation used a default constructor. For example, we could call config.getConfiguredInstance(SslConfigs.SSL_SSLFACTORY_CONFIG, PluggableSslFactory.class) to create the instance. Unfortunately, we need to set properties through the 3 argument constructor before we can call configure(Map).

We could still have a default constructor if we add a new method to pass the 3 arguments:
preconfigure(Mode mode, String clientAuthConfigOverride, boolean keystoreVerifiableUsingTruststore);
It would not be possible to call config.getConfiguredInstance() but at least the interface members would document the whole contract. This would work but is not backwards compatible with applications calling SslFactory directly.

The arguments clientAuthConfigOverride and keystoreVerifiableUsingTruststore seem somewhat arbitrary for a general pluggable interface. They were kept to minimize the changes and maximize backwards compatibility.

We need the ability to send custom configuration options to the PluggableSslFactory implementation. Should this be another KIP?