Table of Contents |
---|
Status
Current state: Under DiscussionApproved
Discussion thread: here
JIRA: KAFKA-4029
Pull Request: https://github.com/apache/kafka/pull/4429
Released: 1.1.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
When the listeners
parameter is configured, the existing options rest.host.name
and rest.port
will be ignored. The fields rest.host.name
and rest.port
will be marked as deprecated.
The HTTPS listener (when configured in listeners
) will by default use the SSL configuration from the ssl.*
options. In case the user needs to use different SSL configuration for connecting to Kafka brokers and for the REST interface, the default settings can be overridden by using the prefix listeners.https.
- for example:listeners.https.ssl.keystore.location=/my/path/keystore.jks
The rest.advertised.host.name
and rest.advertised.port
options will continue to be used as today to specify the connection address which should be used by other workers. In addition a new option rest.advertised.security.protocol
will define whether the listener
will define whether other workers should connect using HTTP or HTTPS protocols. In case HTTPS is selected, the connecting worker will use the SSL configuration from the existing ssl.*
options. Even in case when rest.advertised.host.name
and rest.advertised.port
options are not specified this field will be used to define which protocol should be advertised to other workers in combination with the appropriate hostname and port from the listener field.
This proposal doesn't include any authorization / ACL features. Only encryption and authentication. Authorization / ACLs should be subject of separate KIP in order to keep the scope of this KIP under control. It also doesn't add any other authentication options than SSL/TLS client authentication.
Public Interfaces
Configuration of SSL / TLS for the Kafka Connect REST interface will tries to follow the configuration for other SSL / TLS enabled server interfaces. It will be done through the properties configuration file for the distributed Kafka Connect workers. It will add following
Following new options will be added:
Parameter | Default value | Note |
---|---|---|
listeners | null | List of REST listeners in the format protocol://host:port,protocol2://host2:port2 where the protocol is one of HTTP and HTTPS. |
| null | Configures the listener used for communication between workers. Valid values are either When the listeners configuration is not defined or when it contains |
ssl.client.auth | none | Valid values are none , requested and required . It will controls whether:
This is the only authentication option suggested as part of this KIP. |
listeners.https.ssl.* | The listeners.https. prefix can be used with any SSL configuration option mentioned below to override the default SSL configuration which is shared with the connections to Kafka broker. In case at least one option with this prefix exists, the implementation will use only SSL options with this prefix and will ignore all options without prefix. In case no option with prefix listeners.https. exists, the options without prefix will be used. |
Following existing options will be affected by this KIP:
Parameter | Default value | Note |
---|---|---|
rest.host.name | null | When listeners option is defined, this field will be ignored. |
rest.port | 8083 | When listeners option is defined, this field will be ignored. |
The rest.host.name
and rest.port
will be marked as deprecated. The listeners
field would be the one preferred for the long term future.
Following existing options will be reused by this KIP without any changes:
Parameter | Default value | Note | ||||
---|---|---|---|---|---|---|
rest.advertised.host.name | null | |||||
rest. | securityadvertised. | protocolport | null | This field will be | SSLreused without any changes. | |
ssl.keystore.location | null | |||||
ssl.keystore.password | null | |||||
ssl.keystore.type | JKS | |||||
ssl.key.password | null | |||||
ssl.truststore.location | null | |||||
ssl.truststore.password | null | |||||
ssl.truststore.type | JKS | |||||
ssl.enabled.protocols | TLSv1.2,TLSv1.1,TLSv1 | |||||
ssl.provider | null | |||||
ssl.protocol | TLS | |||||
ssl.cipher.suites | null | |||||
ssl.keymanager.algorithm | SunX509 | |||||
ssl.secure.random.implementation | null | |||||
ssl.trustmanager.algorithm | PKIX | |||||
ssl. | client.authnone | Other valid values are "required" and "requested". |
The rest.security.protocol
option will support only PLAINTEXT
and SSL
values. PLAINTEXT
will be the default value and will keep all SSL/TLS functionality disabled to keep the backwards compatibility.
The rest.ssl.client.auth
option would support values of required
, requested
and none
(none
being the default). It will control whether:
- the connecting client is required to do SSL/TLS client authentication (
required
) - it can decide to skip the SSL/TLS client authentication (
requested
) - the SSL/TLS authentication will be completely disabled (
none
)
This is the only authentication option suggested as part of this KIP.
The other options would follow the semantics of the same options in the Kafka broker.
endpoint.identification.algorithm | null |
Migration Plan and Compatibility
This KIP is a new implementation and doesn't have any backwards compatibility issues or special requirements on migration from older versions. Existing Kafka Connect installation would work in the same way as before this change. Without the SSL configuration, the REST interface will continue to be configured as today - i.e. without SSL / TLS.
Rejected Alternatives
- The first version of this KIP suggested only single REST listener - either HTTP or HTTPS. This was changed based on the feedback from the discussion
Technically it would be possible to define multiple "listeners". For example one with HTTP and one with HTTPS. However, given the functionality provided over the REST interface I see only limited value in it. And therefore I rejected it. If you see some value / use case in this, please raise it in the discussion.