Table of Contents

So

Current state: Under Discussion

...

This KIP proposes to include 2 additional metadata information - dataType, and documentation for the individual config entries.

The motivation behind extending the schema to include dataType of the configuration entry including dataType is to enable the client to provide better validation to the user. In the absence of dataType the only way to know if the user specified value is correct is to make an `alter` call and check if there is no error, which is a suboptimal sub-optimal experience. Including the dataType could open up potentially other use cases as well.

...

Public Interfaces

Network protocol

Request

There is no change in the request schema except The changes to the Request schema include

Bump up the valid versions range
New fields - IncludeSynoyms and IncludeDocumentation - with default value of false

Code Block

{
  "apiKey": 32,
  "type": "request",
  "name": "DescribeConfigsRequest",
  // Version 1 adds IncludeSynoyms.
  // Version 2 is the same as version 1.
  // Version 3 is the same as version 2.
  "validVersions": "0-3", <--- Updated Field
  "flexibleVersions": "none",
  "fields": [
    { "name": "Resources", "type": "[]DescribeConfigsResource", "versions": "0+",
      "about": "The resources whose configurations we want to describe.", "fields": [
      { "name": "ResourceType", "type": "int8", "versions": "0+",
        "about": "The resource type." },
      { "name": "ResourceName", "type": "string", "versions": "0+",
        "about": "The resource name." },
      { "name": "ConfigurationKeys", "type": "[]string", "versions": "0+", "nullableVersions": "0+",
        "about": "The configuration keys to list, or null to list all configuration keys." }
    ]},
    { "name": "IncludeSynoyms", "type": "bool", "versions": "1+", "default": "false", "ignorable": false,
      "about": "True if we should include all synonyms." }
	{  ]
}

Response

The changes to the Response schema include

Bump valid versions range
Add new Fields - ConfigValueType, and Documentation

Code Block

{
  "apiKey": 32,
  "type": "response",
  "name": "DescribeConfigsResponse",
  "validVersions": "0-3", "name": "IncludeType", "type": "bool", "versions": "1+", "default": "false", "ignorable": false,   <--- UpdatedNew Field
      "flexibleVersionsabout": "none",
True if "fields": [
    we should include configuration type." }
	{ "name": "ThrottleTimeMsIncludeDocumentation", "type": "int32bool", "versions": "01+",
      "aboutdefault": "false", "ignorable": false,  <--- New Field
      "about": "True if we should include configuration documentation." }
  ]
}

Response

The changes to the Response schema include

Bump valid versions range
Add new Fields - ConfigValueType, and Documentation

Code Block

{
  "apiKey": 32,
 The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota." },
    { "name": "Results", "type": "[]DescribeConfigsResultresponse",
  "versionsname": "0+DescribeConfigsResponse",
      "aboutvalidVersions": "The results for each resource.",0-3", <--- Updated Field
  "flexibleVersions": "none",
  "fields": [
      { "name": "ErrorCodeThrottleTimeMs", "type": "int16int32", "versions": "0+",
        "about": "The errorduration code,in ormilliseconds 0for ifwhich wethe wererequest ablewas tothrottled successfullydue describeto thea configurations."quota }violation,
    or zero if the request did not violate any quota." },
    { "name": "ErrorMessageResults", "type": "string[]DescribeConfigsResult", "versions": "0+", "nullableVersions": "0+",
        "about": "The errorresults message,for or null if we were able to successfully describe the configurations." },each resource.", "fields": [
      { "name": "ResourceTypeErrorCode", "type": "int8int16", "versions": "0+",
        "about": "The resource type error code, or 0 if we were able to successfully describe the configurations." },
      { "name": "ResourceNameErrorMessage", "type": "string", "versions": "0+",
 "nullableVersions": "0+",
        "about": "The resource name error message, or null if we were able to successfully describe the configurations." },
      { "name": "ConfigsResourceType", "type": "[]DescribeConfigsResourceResultint8", "versions": "0+",
        "about": "EachThe listedresource configurationtype.", "fields": [
  },
       { "name": "NameResourceName", "type": "string", "versions": "0+",
          ""about": "The configurationresource name." },
        { "name": "ValueConfigs", "type": "string[]DescribeConfigsResourceResult", "versions": "0+", "nullableVersions": "0+",
          "about": "TheEach listed configuration value." },.", "fields": [
        { "name": "ReadOnlyName", "type": "boolstring", "versions": "0+",
          "about": "True if theThe configuration is read-onlyname." },
        { "name": "IsDefaultValue", "type": "boolstring", "versions": "0+", "nullableVersions": "0+",
          "about": "True if theThe configuration is not setvalue." },
        // Note{ "name": the v0 default for this field that should be exposed to callers is
        // context-dependent. For example, if the resource is a broker, this should default to 4.
        // -1 is just a placeholder value.
        { "name": "ConfigSource", "type": "int8", "versions": "1+", "default": "-1", "ignorable": true,
          "about": "The configuration source." },
        { "name": "IsSensitive", "type": "bool", "versions": "0+",
          "about": "True if this configuration is sensitive." },"ReadOnly", "type": "bool", "versions": "0+",
          "about": "True if the configuration is read-only." },
        { "name": "IsDefault", "type": "bool", "versions": "0",
          "about": "True if the configuration is not set." },
        // Note: the v0 default for this field that should be exposed to callers is
        // context-dependent. For example, if the resource is a broker, this should default to 4.
        // -1 is just a placeholder value.
        { "name": "SynonymsConfigSource", "type": "[]DescribeConfigsSynonymint8", "versions": "1+", "default": "-1", "ignorable": true,
          "about": "The synonyms for this configuration keysource.", "fields": [
   },
        { "name": "NameIsSensitive", "type": "stringbool", "versions": "10+",
            "about": "TheTrue if synonymthis name." }configuration is sensitive." },
          { "name": "ValueSynonyms", "type": "string[]DescribeConfigsSynonym", "versions": "1+", "nullableVersionsignorable": "0+"true,
            "about": "The synonym value synonyms for this configuration key." },, "fields": [
          { "name": "SourceName", "type": "int8string", "versions": "1+",
            "about": "The synonym sourcename." },
        ]},
<--- Start of new Fields
        { { "name": "ConfigValueTypeValue", "type": "int8string", "versions": "31+", default"nullableVersions": "0+",
 "ignorable": true,
          "about": "The configurationsynonym data typevalue." Type},
 can be one of the following values - BOOLEAN, STRING, INT, SHORT, LONG, DOUBLE, LIST, CLASS, PASSWORD"},
        { "name{ "name": "DocumentationSource", "type": "stringint8", "versions": "31+",
 "nullableVersions": "0+",
          "about": "The configurationsynonym documentationsource." },
<-
        ]},
<--- EndStart of new Fields
        { "name":  ]}
    ]}
  ]
"ConfigValueType", "type": "int8", "versions": "3+", default": "0",
          "about": "The configuration data type."},
        { "name": "Documentation", "type": "string", "versions": "3+", "nullableVersions": "0+",
          "about": "The configuration documentation." },
<---- End of new Fields
      ]}
    ]}
  ]
}

AdminClient Class

No new method will be added or updated to the AdminCient class.

...

Under the proposed change, response for AdminClient.describeConfigs response (DescribeConfigsResponse.ConfigEntry) would include following new properties. These new properties would be part of DescribeConfigsResponse.ConfigEntry class.

Type: ConfigType - This is an Enum of all the supported data type.
Code Block
/** * Data type of configuration entry. */ public enum ConfigType { UNKNOWN, BOOLEAN, STRING, INT, SHORT, LONG, DOUBLE, LIST, CLASS, PASSWORD }
ConfigType.Unknown is for ensuring backward compatibility (explained is Compatibility section below)/

The above enum directly maps to the values in above Enum are derived from existing Type Enum as defined in ConfigDef.java below
Code Block
/** * The config types */ public enum Type { BOOLEAN, STRING, INT, SHORT, LONG, DOUBLE, LIST, CLASS, PASSWORD }

2. Documentation: String - field's documentation

DescribeConfigsResponse.ConfigEntrywith the above new fields

Code Block

public class ConfigEntry {
    private final String name;
    private final String value;
    private final ConfigSource source;
    private final boolean isSensitive;
    private final boolean isReadOnly;
    private final List<ConfigSynonym> synonyms;

.....
    private final StringConfigType type; // This is the new property
	private final String documentation; // This is the new property
.....

...

Following example show how the response would look like in JSON format.
Note: Kafka uses binary protocol over TCP. ConfigType Enum is serialized as byte instead of string.

Code Block

language	js

{
  "name": "offsets.topic.num.partitions",
  "value": "50",
  "source": "DEFAULT_CONFIG",
  "type": "INT",   // <---- New Field
  "synonyms": [
    {
      "name": "offsets.topic.num.partitions",
      "value": "50",
      "source": "DEFAULT_CONFIG"
    }
  ],
  "documentation": "The number of partitions for the offset commit topic (should not change after deployment)", //  <---- New Field
  "isReadOnly": true,
  "isSensitive": false,
 }: false,
 }

Default Behavior

By default, the new fields would not be included in the response to describeConfigs. This is to avoid the response from boating up as the value for documentation field can be quite large.

Compatibility, Deprecation, and Migration Plan

The proposed change is backward compatible in a sense that for same version of Kafka server and client response would include additional fields

For example, if one runs kafka-config.sh
bin/kafka-configs.sh --describe --entity-type brokers --bootstrap-server localhost:9092 --entity-name 0

the output would be as

Code Block
replication.quota.window.num=11 type=INT sensitive=false synonyms={} documentation=...

...

Code Block
request.timeout.ms=30000 type=null sensitive=false synonyms={} documentation=null

...

For backward compatibility we rely on existing schema versioning technique. Under this, client sends its version information in the request header. This in turn is used on the server to select the schema for that version.

	Server version	Client version	Expected Behavior
1	X	X	Both `Type` and `Documentation` is visible to the client.
2	X	X-1	`Type` and `Documentation` are not visible to the client. Not serialized as part of the response.
3	X-1	X	IllegalArgumentException: Invalid version for API key...
4	X+1	X	Both `Type` and `Documentation` is visible to the client. If version X+1 has new value in enum `ConfigType`, client would get `ConfigType.Unknown` instead for that new value.

Rejected Alternatives

NA

Space shortcuts

Child pages

Versions Compared

Old Version 14

New Version 15

Key

So

Public Interfaces

Network protocol

Request

Response

Response

AdminClient Class

Default Behavior

Compatibility, Deprecation, and Migration Plan

Rejected Alternatives

Space shortcuts

Child pages

Page History

Versions Compared

Old Version 14

New Version 15

Key

So

Public Interfaces

Network protocol

Request

Response

Response

AdminClient Class

Default Behavior

Compatibility, Deprecation, and Migration Plan

Rejected Alternatives