Versions Compared

Old Version 1

changes.mady.by.user Unknown User (showuon)

Saved on

compared with

New Version 2

changes.mady.by.user Unknown User (showuon)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: "Under Discussion"

Discussion thread: here

JIRA: KAFKA-18005


Motivation

Currently, when describing config for a resource, we'll get `null` if the config is a sensitive config, ex: "ssl.keystore.certificate.chain", "ssl.keystore.password". And when describing configs with them it'll always return something like this:

...

Note: The value of `listener.name.myssl.ssl.keystore.key` should not be a non-null value , but returning null for security consideration.

...

Consider the dynamic configurations ssl.keystore.location, ssl.keystore.password and ssl.keystore.type are stored in "vault" and their values are resolved using a VaultConfigProvider. 

...
ssl.keystore.location=${vault:/path/to/variables.properties:ssl.keystore.location}

ssl.keystore.password=${vault:/path/to/variables.properties:ssl.keystore.password}

ssl.keystore.type=${vault:/path/to/variables.properties:ssl.keystore.type}
 
config.providers=vault
config.providers.file.class=org.apache.kafka.connect.configs.VaultConfigProvider

  1. The user will update the ssl.keystore.location, ssl.keystore.password, ssl.keystore.type in the vault.
  2. After the updates are complete he will send a adminClient request to the broker to notify that configs are updated.
  3. Once the Broker receives a alteredConfig request it will invoke the get function in VaultConfigProvider.
  4. The VaultConfigProvider will fetch the actual values for ssl.keystore.location, ssl.keystore.password, ssl.keystore.type from the vault.
  5. The broker will validate these configs and apply the changes.

...

In the step (2), the operator needs a way to know if these configs are up-to-date. With current design, the operator will never know it, and blindly run the alter config multiple times, or worse, the operator thought it is already updated and skipped this update, and cause the broker connection failure.


In the Kubernetes world, one of the purpose for the operator is:

Kubernetes Operators manage application logic and are part of the Kubernetes control plane. As such, they are controllers that execute loops to check the actual state of the cluster and the desired state, acting to reconcile them when the two states are drifting apart. 

from the CNCF blog post.


It would be great if the broker/controller can return some metadata of these sensitive configs, like "last modified timestamp", to allow the operators have a way to know if this config is up-to-date.

...