Status

Current state:  Under DiscussionDISCARDED

Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg99126.html

Voting thread: https://www.mail-archive.com/dev@kafka.apache.org/msg100715.html

JIRA: KAFKA-8621

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Discard Reason

The discussion on this KIP lead us to create KIP-519: Make SSL context/engine configuration extensible which is in ACCEPTED state now and this particular KIP is not needed anymore.

Motivation

Current Kafka versions supports file based KeyStore and TrustStore via ssl.keystore.location and ssl.truststore.location configurations along with required passwords configurations.

...

1. As documented in public interfaces section, we will introduce two interfaces to allow pluggable implementation to provide key/trust stores loading
2. We will make changes to the SslEngineBuilder#createSSLContext() method to invoke the key/trust store loading from new ssl configurations we introduce.
   1. Pseudocode changes in the SslEngineBuilder#createSSLContext() looks like below

      
      

      Code Block

      String kmfAlgorithm = this.kmfAlgorithm != null ? this.kmfAlgorithm : KeyManagerFactory.getDefaultAlgorithm(); KeyManagerFactory kmf = KeyManagerFactory.getInstance(kmfAlgorithm); KeyStore ks; ... ... if ( keystore != null ) { // load keystore 'ks' in existing way } else if ( 'ssl.keystore.loader' specified ) { // load keystore 'ks' by invoking the pluggable implementation class for the config } kmf.init(ks, ksPassword.toCharArray()); ... ... ... String tmfAlgorithm = this.tmfAlgorithm != null ? this.tmfAlgorithm : TrustManagerFactory.getDefaultAlgorithm(); TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm); KeyStore ts = null; ... if ( truststore != null ) { // load truststore 'ts' in existing way } else if ( 'ssl.truststore.loader' specified ) { // load truststore 'ts' by invoking the pluggable implementation class for the config } ... tmf.init(ts); ...

      
      

3. We will make changes to the SslEngineBuilder#shouldBeRebuilt() method appropriately

...

package providertest;

import java.security.Provider;

public class MyProvider extends Provider {

    private static final String name = "MyProvider";
    private static double version = 1.0d;
    private static String info = "Maulin's SSL Provider v"+version;

    public MyProvider() {
        super(name, version, info);
        this.put("TrustManagerFactory.PKIX", "providertest.MyTrustManagerFactory");
    }
}

Writing a Java security provider and registering it from JRE

Alternative to using ssl.provider configuration is to register the Java security provider in JRE's jre/lib/security/java.security file. This way we won't run into the limitation mentioned in the above rejected approach.

However there are following challenges,

1. We have to modify the java.security file on the system which creates the similar challenge as of hosting jks on local file system - meaning maintaining per box, deployment etc
2. Assuming modifying java.security file is not a challenge (See KIP-492) , we still have to write a Provider with custom algorithm  for TrustManagerFactory and KeyManagerFactory
   1. When we write those factories implementation there is no  easy way to re-use validation logic (example: OpenJDK TrustManagerImpl) done by existing Providers.
   2. The X509ExtendedTrustManager class is having all the methods abstract so we can't re-use any standard implementations easily
   3. We will end up copying  the validation logic (example: OpenJDK TrustManagerImpl) which is "security domain" centric
   4. The validation logic deals with essentially three things as of now,
      1. client side cert checks
      2. server side cert checks
      3. certificate path validations
      4. end point identification verification
   5. The above validation logic we should not have to deal with it in the first place for the purpose of just loading keys/certs from a different source than the file based key/trust stores.

NOTE: You can only realize the above challenges once you try to write the Provider with Trust/Key Manager factories. We would highly encourage you to try writing (using any other open-source library's provider as an example may not give you the idea) a provider to do this before you decide to comment on this approach.

One suggestion could be - Why not use Java's inbuilt rails to "use any provider's implementation" for key/trust manager AND just plugin our own keys/certs? 

That is exactly what we are suggesting to do. Below is the example from our pseudo-code for using TrustManagerFactory.getInstance(). Same applies for KeyManagerFactory.

String tmfAlgorithm = this.tmfAlgorithm != null ? this.tmfAlgorithm : TrustManagerFactory.getDefaultAlgorithm();
TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
KeyStore ks = < load trust store from either the local file or other source >
tmf.init(ks);

Other reason for rejecting this approach

Provider for "standard algorithms" are written in order to be re-used. If we just write a Provider tied to a specific way for loading Trust/Key stores defeats the purpose of the re-use of the Providers.

Provide a way to delegate SSLContext creation

...