Status

Current state: Under Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Current Kafka versions supports file based KeyStore and TrustStore via ssl.keystore.location and ssl.truststore.location configurations along with required passwords configurations.

...

The primary motivation here is to essentially provide an optional way to allow custom way to load KeyStore and TrustStore integration instead of relying on the file system. Of course, that needs to be accompanied by having an ability to load required passwords (for KeyStore, TrustStore and Keys) accordingly.

Public Interfaces

We will introduce an optional way to load KeyStore and TrustStore along with their required passwords as applicable.

...

package org.apache.kafka.common.security.ssl;

import java.security.KeyStore;

public interface TrustStoreLoader {
	
	/**
	 * This loads the truststore. The truststore password will be fetched by whatever mechanism the implementation of this class chooses.
	 * Example: It could use current ssl.truststore.password configuration if it chooses.
	 * @return KeyStore object
	 */
	public KeyStore load();

	/**
	 * This method checks if the given truststore has been modified based on some criteria, typically last-modified timestamp. The definition of 'modified' is left to the 
	 * implementation.
	 * @return true - If the truststore was modified as defined by the implementation; false otherwise
	 */
	public boolean modified();
}

Why we do not specify key/trust store password as input method arguments in the interfaces?

We are not specifying the key/trust store passwords in the KeyStoreLoader/TrustStoreLoader load() method. This is because we want to avoid the dependency in the caller class to load the password. This implementation leaves it open to the Loader implementation to read required configuration or use other mechanism for fetching the password. Typically if you have a Key Manager solution you might be using some sort of 'auth-token' in order to access the Key Manager's API and might not require key/trust store password (you will still need password for unlocking the keys though).

Proposed Changes

Kafka Client library and Kafka Broker both uses SslEngineBuilder class to load KeyStore and TrustStore from the file based configurations.

...

We will keep the existing behavior and add optional new behavior.

Rejected Alternatives

Using existing ssl.provider config

We could think of writing a custom java.security.Provider implementation to load the required KeyStore and TrustStore in a custom way. However, it would not be the correct way to use the java.security.Provider as per its responsibilities documented in java docs.

This was not the correct way.

Provide a way to delegate SSLContext creation

We could create a new configuration like ssl.context.loader/ssl.context.initializer and use the implementation class to obtain the object of javax.net.ssl.SSLContext instead of using SslEngineBuilder#createSSLContext(). 

...

Overall, we didn't find enough justification to follow this path.

Generated the required SSL configuration values from the Key Manager API

If we have a Key Manager solution which provides APIs like Hashicorp's Vault we need to find a way to generate the required ssl configurations for Kafka (key/trust stores files, passwords etc) from the same.

...