Page History

...

• What impact (if any) will there be on existing users? Users who use Kafka clusters with Zookeeper clients older than 3.5.x won't be able to communicate with a Zookeeper cluster using 3.8.1. As mentioned in the accompanying JIRA ticket Apache Kafka has been using Zookeeper since version 2.4, everything above and including this version should be stable. It is acceptable to break compatibility with Apache Kafka versions prior to 2.4 as they are considered beyond their end of life and are not maintained (source: Time Based Release Plan#WhatIsOurEOLPolicy).

These are the configurations that Kafka passes onto Zookeeper clients:

{code: java}

def zkClientConfigFromKafkaConfig(config: KafkaConfig, forceZkSslClientEnable: Boolean = false): ZKClientConfig = {
  val clientConfig = new ZKClientConfig
  if (config.zkSslClientEnable || forceZkSslClientEnable) {
    KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslClientEnableProp, "true")
    config.zkClientCnxnSocketClassName.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkClientCnxnSocketProp, _))
    config.zkSslKeyStoreLocation.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslKeyStoreLocationProp, _))
    config.zkSslKeyStorePassword.foreach(x => KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslKeyStorePasswordProp, x.value))
    config.zkSslKeyStoreType.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslKeyStoreTypeProp, _))
    config.zkSslTrustStoreLocation.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslTrustStoreLocationProp, _))
    config.zkSslTrustStorePassword.foreach(x => KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslTrustStorePasswordProp, x.value))
    config.zkSslTrustStoreType.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslTrustStoreTypeProp, _))
    KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslProtocolProp, config.ZkSslProtocol)
    config.ZkSslEnabledProtocols.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslEnabledProtocolsProp, _))
    config.ZkSslCipherSuites.foreach(KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslCipherSuitesProp, _))
    KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslEndpointIdentificationAlgorithmProp, config.ZkSslEndpointIdentificationAlgorithm)
    KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslCrlEnableProp, config.ZkSslCrlEnable.toString)
    KafkaConfig.setZooKeeperClientProperty(clientConfig, KafkaConfig.ZkSslOcspEnableProp, config.ZkSslOcspEnable.toString)
  }
  // The zk sasl is enabled by default so it can produce false error when broker does not intend to use SASL.
  if (!JaasUtils.isZkSaslEnabled) clientConfig.setProperty(JaasUtils.ZK_SASL_CLIENT, "false")
  clientConfig
}

{code}

Below is a list of changes to behaviours which Kafka uses to communicate with Zookeeper:

Kafka-related Notable changes in Zookeeper 3.7.0 related to security

• Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-3959

  - previously there could be only one SASL SuperUser. With this change multiple SuperUsers could be specified via a configurationZookeeper now allows multiple super users. Kafka does not pass on the value of zookeeper.superUser to its Zookeeper client so this change should not affect it.

• Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-3301

  - quotas Quotas which were previously logged but not previously enforced are now enforced. These quotas have to do with create/update/delete etc. operations. This will affect Kafka users who put quotas in their Zookeeper clusters.

• Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-3482

  - Kerberos authentication did not work over SSL , but has now been fixedis now supported. Kafka does not support Kerberos authentication with Zookeeper so this change should not affect it.

• Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-3561

  - user User enforced authentication was only available for SASL before this change. User enforced authentication extends to all other types of authentication supported by Zookeeper. The point of this change is that no additional ACLs are needed to prevent unauthenticated access if one authentication method is enabled.

Notable Kafka-related changes in Zookeeper 3.8.0 related to security

• Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key ZOOKEEPER-4396

  - instead of storing password as plaintext use password protected filesZookeeper used to use plaintext password for its trust and key stores. This change makes files which store those passwords to take precedence, but they don't remove the already working logic.

• If we are changing behavior how will we phase out the older behavior? It should gradually be phased out as users update their Kafka versions
• If we need special migration tools, describe them here. N/A
• When will we remove the existing behavior? N/A

...