This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Under Discussion"

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: KAFKA-3665

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Apache Kafka is increasingly used in environments where security is critical. TLS can be used a security protocol with Kafka to enable server authentication, client authentication and encryption. Even though Kafka supports server hostname verification and the documentation talks about setting hostnames in server certificates, hostname verification is disabled by default. This in an insecure default value since hostname verification is required to prevent man-in-the-middle attacks.

JSSE docs says:

When using raw SSLSocket and SSLEngine classes, you should always check the peer's credentials before sending any data. The SSLSocket and SSLEngine classes do not automatically verify that the host name in a URL matches the host name in the peer's credentials. An application could be exploited with URL spoofing if the host name is not verified.

RFC-2818 talks about endpoint identification required to establish server identity for HTTP over TLS:

In general, HTTP/TLS requests are generated by dereferencing a URI. As a consequence, the hostname for the server is known to the client. If the hostname is available, the client MUST check it against the server's identity as presented in the server's Certificate message, in order to prevent man-in-the-middle attacks.

If the client has external information as to the expected identity of the server, the hostname check MAY be omitted. (For instance, a client may be connecting to a machine whose address and hostname are dynamic but the client knows the certificate that the server will present.) In such cases, it is important to narrow the scope of acceptable certificates as much as possible in order to prevent man in the middle attacks.  In special cases, it may be appropriate for the client to simply ignore the server's identity, but it must be understood that this leaves the connection open to active attack.

http://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf gives a good explanation of the risk of using TLS without hostname verification.

This KIP proposes to enable hostname verification by default for Kafka client connections to prevent man-in-the-middle attacks.

Public Interfaces

The default value of ssl.endpoint.identification.algorithm will be changed to HTTPS. Hostname verification can be disabled if required by setting the config to an empty string. The default will apply to both client configs (producer/consumer/admin client etc.) and inter-broker connections. The config can be set at listener-level.

Proposed Changes

This will be a simple change of the default value of ssl.endpoint.identification.algorithm to HTTPS. Any tests that rely on this config to be disabled will be updated to explicitly set the config to an empty string. Any test which doesn't care about the value of this config, but generates certificates without hostnames will be updated to set hostnames in certificates. All system tests are currently run wth ssl.endpoint.identification.algorithm=HTTPS anyway, so no changes are required.

Compatibility, Deprecation, and Migration Plan

Even though hostname verification should be enabled to protect against man-in-the-middle attacks, there may be environments were certificates are generated without hostnames and hostname verification is performed in other ways. It is possible that these installations are run with the default config. They will need to explicitly set ssl.endpoint.identification.algorithm to an empty string when upgrading. This will be documented in upgrade notes.

Rejected Alternatives

Disable hostname verification for SASL_SSL with Kerberos by default

Kerberos performs hostname verification, so we could leave default value as-is, i.e. disabled for client connections with SASL_SSL using GSSAPI as the mechanism. But it is safer to use the same default value for TLS across all security protocols and SASL mechanisms. Hence this KIP proposes to simply use a secure default, leaving it up to the user to disable if required.