Versions Compared

Old Version 14

changes.mady.by.user TEJAL ADSUL

Saved on

compared with

New Version Current

changes.mady.by.user Colin McCabe

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Status

Current state:  "Under Discussion"accepted

Discussion thread: here 

JIRA: here 

...

This proposal intends to enhance the AbstractConfig base class to automatically resolve variables of the form specified in KIP-297. This simple change will make it very easy for all the components client applications, Kafka Connect, and Kafka Streams to use shared code to easily incorporate externalized secrets and other variable replacements within their configurations. All the components broker, connect, producer, consumer, admin client, and so forth.  will be able to automatically resolve the external configurations. This proposal does not add other ConfigProvider implementations or change the behavior of existing methods. By default we will enable the automatic resolution of externalized configurations, each component can disable the feature if needed.

Public Interfaces

This proposal will add 2 a new interfaces for constructor to the AbstractConfig base class :
AbstractConfig

/**
 * Construct a configuration with a ConfigDef, the configuration properties, optional {@link ConfigProviders}.
 * @param definition the definition of the configurations
 * @param originals the name-value pairs of the configuration
 * @param configProviders the map of properties of config providers which will be instantiated by the constructor to resolve any variables in {@code originals}
 * @param doLog whether the configurations should be logged
 */
public AbstractConfig(ConfigDef definition, Map<?, ?> originals, Map<?, ?> configProviders, boolean doLog) {
...
}

...

  1. Identify the configs which need to be stored in external systems.
  2. Implement a or use an existing ConfigProvider which will fetch the config value from external system.
  3. Replace the configs values with variables as defined by KIP-297

Once these variables are added, the components will be automatically get the resolved configurations. As Connect component already supports variables in connector configs via KIP 297 it will use this feature to resolve variables in the worker configurations via the existing 'StandaloneConfig' and 'DistributedConfig' subclasses of AbstractConfig.

...