Status

Current state: "Under Discussion"

Discussion thread: 

JIRA: 

Motivation

The KIP which established dynamic broker configuration, KIP-226, specified that this configuration data would be stored in ZooKeeper. It also established an encryption mechanism for secrets such as passwords. We would like to have the same level of protection for secret data in the post-ZooKeeper world of KRaft.

The threat model for the KIP-226 encryption was to protect against an attacker who had compromised the ZooKeeper service, but not the Kafka service. Since there is no more ZooKeeper service when using KRaft, our model is different. Our threat model is to protect against an attacker who has gained access to the metadata log, but not other aspects of the system (such as the system used to store encryption keys).

Different users may wish to store metadata encryption keys in different ways. For example, in a system with both local storage and remote network-accessed storage, it may make sense to store the metadata encryption keys on the local storage, but the metadata log on remote-accessed storage. Alternately, some users may want to store the metadata encryption key remotely, so that it doesn't appear on disk storage accessible to the broker at any time. A third user may not want to encrypt metadata at all. Therefore, this KIP makes the metadata encryption key mechanism both pluggable and optional.

Finally, metadata key rotation is very important to security. We want it to be possible to do this rotation online, without shutting down the cluster and taking downtime. It should also be possible to do this without doing a rolling restart.

Overview

Scope of Encryption

As mentioned previously, only secrets in the metadata log will be encrypted by this mechanism. That includes things like configurations which are passwords, and new metadata encryption keys themselves.

Metadata Encryption Keys

Metadata encryption keys are sequences of bytes used to encrypt the metadata via a symmetrical encryption algorithm. In order to facilitate key rotation, many metadata encryption keys may be "registered" with the system. However, only one key can be "active." The active key is the one which is actually used for encryption.

Each metadata encryption key is identified by a 16-byte UUID.

This KIP introduces the concept of a metadata encryptor. Each encryptor is identified by a unique 16-byte UUID and is capable of encrypting arbitrary binary data.

While a node can have any number of encryptors configured, only one encryptor is active at once. KRaft configuration records that contain secret data (such as passwords) are encrypted using the currently active encryptor.

The threat model for this 

Mechanism

In general, 

Public Interfaces

MetadataEncryptionKeyAccessor

RegisterMetadataEncryptionKeyRecord

This record registers a metadata encryption key, which is identified by a 16-byte UUID. The "mechanism" field is the full name of a Java class which implements the given metadata encryption key. Note that this record does not contain the actual encryption key. It simply registers it.

This record appears once in the metadata log and once in the metadata snapshot for each registered encryption key.

{
  "apiKey": ...,
  "type": "metadata",
  "name": "RegisterMetadataEncryptionKeyRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "KeyId", "type": "uuid", "versions": "0+",
      "about": "The metadata encryption key ID." },
    { "name": "AccessorClass", "type": "string", "versions": "0+",
      "about": "The MetadataEncryptionKeyAccessor class to use for this key." }
  ]
}

UnregisterMetadataEncryptionKeyRecord

This record unregisters a metadata encryption key.

This record appears once in the metadata log for each registered encryption key that needs to be deregistered. It does not appear in the metadata snapshot.

{
  "apiKey": ...,
  "type": "metadata",
  "name": "UnregisterMetadataEncryptionKeyRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "KeyId", "type": "uuid", "versions": "0+",
      "about": "The metadata encryption key ID." }
  ]
}

TransmitMetadataEncryptionKeyRecord

This record transmits a metadata encryption key in the metadata log. Note that the key is encrypted with the currently active encryption key.

This record appears once in the metadata log for each registered encryption key that needs to be transmitted. It does not appear in the metadata snapshot.

{
  "apiKey": ...,
  "type": "metadata",
  "name": "TransmitMetadataEncryptionKeyRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "KeyId", "type": "uuid", "versions": "0+",
      "about": "The metadata encryption key ID." },
    { "name": "KeyData", "type": "bytes", "versions": "0+",
      "about": "The key data. Encrypted with the current encryption key." }
  ]
}

ActivateMetadataEncryptionKeyRecord

This record activates a metadata encryption key in the metadata log.

This record appears once in the metadata log each time we activate a new metadata encryption key. It appears once at the beginning of each metadata snapshot.

{
  "apiKey": ...,
  "type": "metadata",
  "name": "ActivateMetadataEncryptionKeyRecord",
  "validVersions": "0",
  "flexibleVersions": "0+",
  "fields": [
    { "name": "EncryptorId", "type": "uuid", "versions": "0+",
      "about": "The ID of the encryptor to install, or all zeroes to install no encryptor." }
  ]
}

create encryptor record

delete encryptor record

encrypted config record

default encryptor

(how to add new encryptor)

Compatibility, Deprecation, and Migration Plan

Rejected Alternatives