Note that this KIP proposal is derived from a proposal by Grant Henke that was part of KIP-4 - Command line and centralized administrative operations.

Status

Current state: Under Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

KIP-4 - Command line and centralized administrative operations outlines the motivation for exposing admin operations via the protocol:

...

1. The Metadata request exposes topic metadata, but it does not expose topic configs. ListConfigs DescribeConfigs will make that information available to any client of the Kafka protocol and the AdminClient will expose it to normal users.
2. AlterConfigs would make it possible to update topic configs, but also client and replication quotas via the protocol and the AdminClient.
3. One should be able to tell how a topic or broker is configured including defaults and overrides.

Public Interfaces

ACLs

We will introduce 2 new ACL operations: ReadConfigs and WriteConfigs. The goal is for these operations to apply to any resource that has configs. In the initial version, they will only apply to the Cluster resource type.

Protocol APIs

Wire Format types

KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs introduced a wire format representation for ResourceType and AclOperation. We weill add new values to both types as follows:

...

• 0: Unknown

• 1: Any

• 2: Topic

• 3: Group

• 4: Cluster

• 5: Broker (new)

• 6: Client (new)

• 7: User (new)

...

Describe Configs

ListConfigsDescribeConfigs Request (Version: 0) => [resource]   
  resource => resource_type resource_name
    resource_type => INT8
    resource_name => STRING

...

ListConfigsDescribeConfigs Response (Version: 0) => error_code [entities]
  entities => error_code resource_type resource_name [configs]
	error_code => INT16
	resource_type => INT8
    resource_name => STRING
    configs =>
      config_name => STRING
      config_value => STRING
      read_only => BOOLEAN
      is_default => BOOLEAN
      is_sensitive => BOOLEAN

Alter Configs

AlterConfigs Request (Version: 0) => [resources] validate_only
  validate_only => BOOLEAN
  resources => resource_type resource_name [configs]
    resource_type => INT8
    resource_name => STRING
    configs =>
      config_name => STRING
      config_value => STRING

...

Users will have to ensure that the policy implementation code is in the broker's classpath. Implementations should throw the existing PolicyViolationException with an appropriate error message if the request does not fulfill the policy requirements. We chose a generic name for the only parameter of the validate method in case we decide to add parameters that are not strictly related to the topic (e.g. session information) in the future. The constructor of RequestMetadata is public to make testing convenient for users. Under normal circumstances, it should only be instantiated by Kafka. We chose to create separate API classes instead of reusing request classes to make it easier to evolve the latter.

AdminClient APIs

They follow a similar pattern as existing AdminClient APIs:

public class AdminClient {
    public ListConfigsResultDescribeConfigsResult listConfigsdescribeConfigs(Collection<ConfigResource> resources, ListConfigsOptionsDescribeConfigsOptions options);
    public AlterConfigsResult alterConfigs(Map<ConfigResource, Config> configs, AlterConfigsOptions options);
}

public class ListConfigsOptionsDescribeConfigsOptions { 
    public ListConfigsOptionsDescribeConfigsOptions timeoutMs(Integer timeout);
}

public class ListConfigsResultDescribeConfigsResult {
    public Map<ConfigResource, KafkaFuture<Config>> results()
    public KafkaFuture<Map<ConfigResource, Config>> all();
}
 
public class AlterConfigsOptions { 
    public AlterConfigsOptions timeoutMs(Integer timeout);
	public AlterConfigsOptions validateOnly(boolean validateOnly);
}

public class AlterConfigsResult {
    public KafkaFuture<Void> all();
    public Map<ConfigResource, KafkaFuture<Void>> results();
}
 
public class ConfigResource {
    enum Type {
		CLIENT, BROKER, TOPIC, USER, UNKNOWN;
    }
 
    public ConfigResource(Type type, String name) { ... }
    
    public Type type() { ... }
    public String name() { ... }
}

 
public class Config {
    public Config(Collection<ConfigEntry> configs) { ... }
    public Collection<ConfigEntry> entries() { ... }
}
public class ConfigEntry {
    public ConfigEntry(String name, String value) {
        this(name, value, false, false, false);
    }
    public ConfigEntry(String name, String value, boolean isDefault, boolean isSensitive, boolean isReadOnly) { ... }
    public String name() { ... }
    public String value() { ... }
    public boolean isDefault { ... }
    public boolean isSensitive { ... }
    public boolean isReadOnly { ... }
}

...

The "Public Interfaces" section covers all the proposed changes.

Compatibility, Deprecation, and Migration Plan

We are adding new ACL operations, so third-party Authorizer implementations will potentially have to be updated in order to support them. Since this only affects newly introduced protocol and AdminClient APIs, it's not a compatibility issue. It simply means that the new functionality won't be available for users of such Authorizer implementations until they are updated.

With regards to forwards compatibility, ConfigResource.Type has an UNKNOWN element in case it receives data from a newer broker that cannot be mapped to one of the existing enum types.

Rejected Alternatives

1. Allowing sensitive data to be returned: it's good security practice to never expose sensitive data. If necessary, the user can reset the relevant sensitive data (e.g. a password).
2. Introducing a new Configs resource instead of ReadConfigs and WriteConfigs operations: there is always a one to one mapping between a resource and its configs, so there isn't much value in creating a separate resource for Configs. By adding new operations to existing resources, it's easier to see all the ACLs that affect a given resource.

Future work

1. Forward requests to the relevant brokers in order to return `read-only` broker configs.
2. More fine-grained ACLs so that a user authorized to ReadConfigs/WriteConfigs on a given resource type (topic, broker, etc.) can read/write the relevant configs.