Apache kafka allows clients to conenct over SSL . By default SSL is disabled but can be turned on as needed.
1. Generating the key and the certificate for each kafka broker
...
Code Block |
---|
$ keytool -keystore server.keystore.jks -alias localhost -validity {validity} -genkey |
You need to specify two parameters in the above command:
Code Block |
---|
keystore: the keystore file that stores the certificate. The keystore file contains the private key of the certificate; therefore, it needs to be kept safely.
validity: the valid time of the certificate in days. |
Ensure that common name (CN) matches exactly with the fully qualified domain name (FQDN) of the server. The client compares the CN with the DNS domain name to ensure that it is indeed connecting to the desired server, not the malicious one.
2. Creating your own CA
...
Code Block |
---|
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365 |
...
Code Block |
---|
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert |
...
Code Block |
---|
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert |
...
3. Signing the certificate
The next step is to sign all certificates generated by step 1 with the CA generated in step 2. First, you need to export the certificate from the keystore:
Code Block |
---|
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file |
...
Code Block |
---|
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days {validity} -CAcreateserial -passin pass:{ca-password} |
...
Code Block |
---|
$ keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
$ keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed |
...
Code Block | ||
---|---|---|
| ||
keystore: the location of the keystore
ca-cert: the certificate of the CA
ca-key: the private key of the CA
ca-password: the passphrase of the CA
cert-file: the exported, unsigned certificate of the server
cert-signed: the signed certificate of the server |
All of the above steps in a bash script
Code Block | ||
---|---|---|
| ||
#!/bin/bash
#Step 1
keytool -keystore server.keystore.jks -alias localhost -validity 365 -genkey
#Step 2
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
#Step 3
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:test1234
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed |
4. Configuring Kafka Broker
Kafka Broker comes with the feature of listenting on multiple ports thanks to [KAFKA-1809](https://issues.apache.org/jira/browse/KAFKA-1809) .
we need to configure following property in server.properties
Code Block | ||
---|---|---|
| ||
listeners |
This property must have a PLAINTEXT port along with a SSL port. Since we don't have interbroker SSL support yet if we only configure SSL port than with-in broker communication will not work.
Code Block | ||
---|---|---|
| ||
listners=PLAINTEXT://host.name:port,SSL://host.name:port |
...
Code Block | ||
---|---|---|
| ||
ssl.protocol = TLS
ssl.provider (Optional. The name of the security provider used for SSL connections. Default value is the defaultsecurity provider of the JVM.)
ssl.cipher.suites = "A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a networkconnection using TLS or SSL network protocol."
ssl.enabled.protocols = TLSv1.2,TLSv1.1,TLSv1 (list out the SSL protocols that you are goingaccept from clients . Do note SSL is deprecated using that in production is not recommended)
ssl.keystore.type = JKS
ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password = test1234
ssl.key.password = test1234
ssl.truststore.type = JKS
ssl.truststore.location = /var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password = test1234
ssl.client.auth = none ( "required " = > client authentication is required, "requested" => client authentication is requested and client without certs can still connect when this option chosen" ) |
...
Code Block | ||
---|---|---|
| ||
with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL -> EndPoint(192.168.64.1,9093,SSL) |
To check quickly if the server keystore and truststore are setup properly you can run the following command
Code Block | ||
---|---|---|
| ||
openssl s_client -debug -connect localhost:9093 -tls1 (Note: TLSv1 should be listed under ssl.enabled.protocols)
In the output of this command you should see server's certificate
Server certificate
-----BEGIN CERTIFICATE-----
MIID+DCCAuACCQCx2Rz1tXx3NTANBgkqhkiG9w0BAQsFADB6MQswCQYDVQQGEwJV
UzELMAkGA1UECAwCQ0ExFDASBgNVBAcMC1NhbnRhIENsYXJhMQwwCgYDVQQKDANv
cmcxDDAKBgNVBAsMA29yZzEOMAwGA1UEAwwFa2FmYWsxHDAaBgkqhkiG9w0BCQEW
DXRlc3RAdGVzdC5jb20wHhcNMTUwNzMwMDQyOTMwWhcNMTYwNzI5MDQyOTMwWjBt
MQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFDASBgNVBAcTC1NhbnRhIENsYXJh
MQwwCgYDVQQKEwNvcmcxDDAKBgNVBAsTA29yZzEfMB0GA1UEAxMWU3JpaGFyc2hh
IENoaW50YWxhcGFuaTCCAbcwggEsBgcqhkjOOAQBMIIBHwKBgQD9f1OBHXUSKVLf
Spwu7OTn9hG3UjzvRADDHj+AtlEmaUVdQCJR+1k9jVj6v8X1ujD2y5tVbNeBO4Ad
NG/yZmC3a5lQpaSfn+gEexAiwk+7qdf+t8Yb+DtX58aophUPBPuD9tPFHsMCNVQT
WhaRMvZ1864rYdcq7/IiAxmd0UgBxwIVAJdgUI8VIwvMspK5gqLrhAvwWBz1AoGB
APfhoIXWmz3ey7yrXDa4V7l5lK+7+jrqgvlXTAs9B4JnUVlXjrrUWU/mcQcQgYC0
SRZxI+hMKBYTt88JMozIpuE8FnqLVHyNKOCjrh4rs6Z1kW6jfwv6ITVi8ftiegEk
O8yk8b6oUZCJqIPf4VrlnwaSi2ZegHtVJWQBTDv+z0kqA4GEAAKBgB+Pdz0306bq
TpUAdb2FERMPLFsx06H0x+TULivcp7HbS5yrkV9bXZmv/FD98x76QxXrOq1WpQhY
YDeGDjH+XQkJ6ZxBVBZNJDIpCnfQpfzXAvryQ+cm8oXUsKidtHf4pLMYViXX6BWX
Oc2hX4rG+lC8/NXW+1zVvCr9To9fngzjMA0GCSqGSIb3DQEBCwUAA4IBAQBfyVse
RJ+ugiNlWg5trZscqH0tlocbnek4UuV/xis2eAu9l4EFOM5kRt5GmkGZRcM/zHF8
BRJwXbf0fytMQKSPFk8R4/NGDOlzoK+F7uXeJ0S2u/T29xk0u2i4tjvleq6OCphE
i9vdjM0E0Whf9SHRhOXirOYFX3cL775XwKdzKKRkk+AszFR+mRu90rdoaepQtgGh
9Kfwr4+6AU/dPtdGuomtBQqMxCzlrLd8EYhVVQ97wHIZ3sPvlM5PIhOJ/YHSBJIC
75eo/4acDxZ+j3sR5kcFulzYwFLgDYBaKH/w3mYCgTALeB1zUkX53NVizIvhUd69
XJO4lDSDtGOlfort
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Sriharsha Chintalapani
issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafak/emailAddress=test@test.com
If the certificate not showed up or if there are any other error messages than your keystores not setup properly. |
4. Configuring Kafka Producer & Kafka Consumer
SSL supported only for new Kafka Producer & Consumer , older api is not supported.
The configs for SSL will be same for both producer & consumer.
Code Block | ||
---|---|---|
| ||
security.protocol = SSL
ssl.provider (Optional. The name of the security provider used for SSL connections. Default value is the default security provider of the JVM.)
ssl.cipher.suites (Optional) ."A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol."
ssl.enabled.protocols= TLSv1.2,TLSv1.1,TLSv1 **Should list atleast one of the protocols configured on the broker side**
if you are configuring client authentication than you must create keystore like step-1 otherwiser keystore config is optional for client.
ssl.keystore.type = "JKS"
ssl.keystore.location = "/var/private/ssl/kafka.client.keystore.jks"
ssl.keystore.password = "test1234"
ssl.key.password = "test1234"
ssl.truststore.type = "JKS"
ssl.truststore.location = "/var/private/ssl/kafka.client.truststore.jks"
ssl.truststore.password = "test1234" |
...
language | text |
---|
...
The information here has been migrated to the SSL section of the website docs.