THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Apache kafka allows clients to conenct over SSL . By default SSL is disabled but can be turned on as needed.
1. Generating the key and the certificate for each kafka broker
...
| Code Block |
|---|
$ keytool -keystore {tmp.server.keystore.jks} -alias localhost -validity {validity} -genkey |
You need to specify two parameters in the above command:
| Code Block |
|---|
keystore: the keystore file that stores the certificate. The keystore file contains the private key of the certificate; therefore, it needs to be kept safely.
validity: the valid time of the certificate in days. |
Ensure that common name (CN) matches exactly with the fully qualified domain name (FQDN) of the server. The client compares the CN with the DNS domain name to ensure that it is indeed connecting to the desired server, not the malicious one.
2. Creating your own CA
...
| Code Block |
|---|
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365 |
...
| Code Block |
|---|
keytool -keystore {client.truststore.jks} -alias CARoot -import -file {ca-cert} |
...
| Code Block |
|---|
keytool -keystore {server.truststore.jks} -alias CARoot -import -file {ca-cert} |
...
3. Signing the certificate
The next step is to sign all certificates generated by step 1 with the CA generated in step 2. First, you need to export the certificate from the keystore:
| Code Block |
|---|
keytool -keystore {tmp.server.keystore.jks} -alias localhost -certreq -file {cert-file} |
...
| Code Block |
|---|
openssl x509 -req -CA {ca-cert} -CAkey {ca-key} -in {cert-file} -out {cert-signed} -days {validity} -CAcreateserial -passin pass:{ca-password} |
...
| Code Block |
|---|
$ keytool -keystore {server.keystore.jks} -alias CARoot -import -file {ca-cert}
$ keytool -keystore {server.keystore.jks} -alias localhost -import -file {cert-signed} |
...
| Code Block | ||
|---|---|---|
| ||
keystore: the location of the keystore
ca-cert: the certificate of the CA
ca-key: the private key of the CA
ca-password: the passphrase of the CA
cert-file: the exported, unsigned certificate of the server
cert-signed: the signed certificate of the server |
4. Configuring Kafka Broker
Kafka Broker comes with the feature of listenting on multiple ports thanks to [KAFKA-1809](https://issues.apache.org/jira/browse/KAFKA-1809) .
we need to configure following property in server.properties
| Code Block | ||
|---|---|---|
| ||
listeners |
This property must have a PLAINTEXT port along with a SSL port. Since we don't have interbroker SSL support yet if we only configure SSL port than with-in broker communication will not work.
| Code Block | ||
|---|---|---|
| ||
listners=PLAINTEXT://host.name:port,SSL://host.name:port |
...
| Code Block | ||
|---|---|---|
| ||
ssl.protocol = TLS
ssl.provider (Optional. The name of the security provider used for SSL connections. Default value is the defaultsecurity provider of the JVM.)
ssl.cipher.suites = "A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a networkconnection using TLS or SSL network protocol."
ssl.enabled.protocols = TLSv1.2,TLSv1.1,TLSv1 (list out the SSL protocols that you are goingaccept from clients . Do note SSL is deprecated using that in production is not recommended)
ssl.keystore.type = JKS
ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password = test1234
ssl.key.password = test1234
ssl.truststore.type = JKS
ssl.truststore.location = /var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password = test1234
ssl.client.auth = none ( "required " = > client authentication is required, "requested" => client authentication is requested and client without certs can still connect when this option chosen" ) |
...
| Code Block | ||
|---|---|---|
| ||
with addresses: PLAINTEXT -> EndPoint(192.168.64.1,9092,PLAINTEXT),SSL -> EndPoint(192.168.64.1,9093,SSL) |
To check quickly if the server keystore and truststore are setup properly you can run the following command
| Code Block | ||
|---|---|---|
| ||
openssl s_client -debug -connect localhost:9093 -tls1 (Note: TLSv1 should be listed under ssl.enabled.protocols)
In the output of this command you should see server's certificate
Server certificate
-----BEGIN CERTIFICATE-----
MIID+DCCAuACCQCx2Rz1tXx3NTANBgkqhkiG9w0BAQsFADB6MQswCQYDVQQGEwJV
UzELMAkGA1UECAwCQ0ExFDASBgNVBAcMC1NhbnRhIENsYXJhMQwwCgYDVQQKDANv
cmcxDDAKBgNVBAsMA29yZzEOMAwGA1UEAwwFa2FmYWsxHDAaBgkqhkiG9w0BCQEW
DXRlc3RAdGVzdC5jb20wHhcNMTUwNzMwMDQyOTMwWhcNMTYwNzI5MDQyOTMwWjBt
MQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0ExFDASBgNVBAcTC1NhbnRhIENsYXJh
MQwwCgYDVQQKEwNvcmcxDDAKBgNVBAsTA29yZzEfMB0GA1UEAxMWU3JpaGFyc2hh
IENoaW50YWxhcGFuaTCCAbcwggEsBgcqhkjOOAQBMIIBHwKBgQD9f1OBHXUSKVLf
Spwu7OTn9hG3UjzvRADDHj+AtlEmaUVdQCJR+1k9jVj6v8X1ujD2y5tVbNeBO4Ad
NG/yZmC3a5lQpaSfn+gEexAiwk+7qdf+t8Yb+DtX58aophUPBPuD9tPFHsMCNVQT
WhaRMvZ1864rYdcq7/IiAxmd0UgBxwIVAJdgUI8VIwvMspK5gqLrhAvwWBz1AoGB
APfhoIXWmz3ey7yrXDa4V7l5lK+7+jrqgvlXTAs9B4JnUVlXjrrUWU/mcQcQgYC0
SRZxI+hMKBYTt88JMozIpuE8FnqLVHyNKOCjrh4rs6Z1kW6jfwv6ITVi8ftiegEk
O8yk8b6oUZCJqIPf4VrlnwaSi2ZegHtVJWQBTDv+z0kqA4GEAAKBgB+Pdz0306bq
TpUAdb2FERMPLFsx06H0x+TULivcp7HbS5yrkV9bXZmv/FD98x76QxXrOq1WpQhY
YDeGDjH+XQkJ6ZxBVBZNJDIpCnfQpfzXAvryQ+cm8oXUsKidtHf4pLMYViXX6BWX
Oc2hX4rG+lC8/NXW+1zVvCr9To9fngzjMA0GCSqGSIb3DQEBCwUAA4IBAQBfyVse
RJ+ugiNlWg5trZscqH0tlocbnek4UuV/xis2eAu9l4EFOM5kRt5GmkGZRcM/zHF8
BRJwXbf0fytMQKSPFk8R4/NGDOlzoK+F7uXeJ0S2u/T29xk0u2i4tjvleq6OCphE
i9vdjM0E0Whf9SHRhOXirOYFX3cL775XwKdzKKRkk+AszFR+mRu90rdoaepQtgGh
9Kfwr4+6AU/dPtdGuomtBQqMxCzlrLd8EYhVVQ97wHIZ3sPvlM5PIhOJ/YHSBJIC
75eo/4acDxZ+j3sR5kcFulzYwFLgDYBaKH/w3mYCgTALeB1zUkX53NVizIvhUd69
XJO4lDSDtGOlfort
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Sriharsha Chintalapani
issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafak/emailAddress=test@test.com
If the certificate not showed up or if there are any other error messages than your keystores not setup properly. |
4. Configuring Kafka Producer & Kafka Consumer
SSL supported only for new Kafka Producer & Consumer , older api is not supported.
The configs for SSL will be same for both producer & consumer.
| Code Block | ||
|---|---|---|
| ||
security.protocol = SSL
ssl.provider (Optional. The name of the security provider used for SSL connections. Default value is the default security provider of the JVM.)
ssl.cipher.suites (Optional) ."A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol."
ssl.enabled.protocols= TLSv1.2,TLSv1.1,TLSv1 **Should list atleast one of the protocols configured on the broker side**
if you are configuring client authentication than you must create keystore like step-1 otherwiser keystore config is optional for client.
ssl.keystore.type = "JKS"
ssl.keystore.location = "/var/private/ssl/kafka.client.keystore.jks"
ssl.keystore.password = "test1234"
ssl.key.password = "test1234"
ssl.truststore.type = "JKS"
ssl.truststore.location = "/var/private/ssl/kafka.client.truststore.jks"
ssl.truststore.password = "test1234" |
...
| language | text |
|---|
...
The information here has been migrated to the SSL section of the website docs.