You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 18 Next »

Status

Current state: "Under Discussion"

Discussion thread: here

JIRA: KAFKA-7740

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Currently, the AdminClient supports a long list of APIs that can be leveraged by users in many different ways. For instance, the AdminClient allows users to manage entities such as topics, partitions, consumer offset, ACLs, replication log directories and so on. In these scenarios, users only need talk to brokers through AdminClient and there is no direct dependency between client applications and Zookeeper any more.

One missing piece in the AdminClient is the configuration of users and clientId entities, which causes some obstacles for users. For instance, there is no way for users to manage their produce/consume client quota through AdminClient. In addition, there is no support for users to manage any other user or clientId related properties through AdminClient in the future. In these scenarios, users have to direct communicate with Zookeeper, which is exactly the anti-AdminClient pattern.

Public Interfaces

Given that the AdminClient has already supported the configuration management for brokers and topics, we will leverage the existing APIs to manage the configurations of users and clients as well. Blew are the two APIs will be reused:

public abstract class AdminClient implements AutoCloseable {
 public abstract DescribeConfigsResult describeConfigs(Collection<ConfigResource> resources, DescribeConfigsOptions options);
 public abstract AlterConfigsResult alterConfigs(Map<ConfigResource, Config> configs, AlterConfigsOptions options);

}

Proposed Changes

There are two changes needed for this KIP: client side changes and server side changes:

client side:
1. Package: org.apache.kafka.common.config

USER and CLIENT resources need to be added in the ConfigResource.java interface. They represent the targeting entities for the configuration management. 

public final class ConfigResource {
 public enum Type {
        BROKER((byte) 4), USER((byte) 3), TOPIC((byte) 2), CLIENT((byte) 1), UNKNOWN((byte) 0); // we add USER and CLIENT type.
    ...
 }

}

2. Package: org.apache.kafka.clients.admin

We would like to dynamically config user and client entities.

/**
 * Source of configuration entries.
 */
public enum ConfigSource {
 ...
 DYNAMIC_USER_CONFIG, // dynamic user config that is configured for a specific user. 
 DYNAMIC_CLIENT_CONFIG, // dynamic client config that is configured for a specific client.
 ...
}

3. Package: org.apache.kafka.common.requests

Same as above section, add the user and client entry in the ConfigSource definition.
public enum ConfigSource {
 ...
 DYNAMIC_USER_CONFIG((byte) 6), 
 DYNAMIC_CLIENT_CONFIG((byte) 7);
 ...
}
server side:

Package: kafka.server

In the AdminManager class, add the user and client case processing logics in the describeConfigs and alterConfigs handler. 

Internally, they will call the adminZkClient, in order to read/write configurations from/to Zookeeper, and then return the results to users. 

Compatibility, Deprecation, and Migration Plan

There is no known compatibility issue introduced in this KIP.

There is also no feature to deprecate from this KIP.

No need to migration, as data are always in the Zookeeper. 

Rejected Alternatives

Set the user and client configurations directly through Zookeeper. This will introduce additional dependency for user applications and bring more overhead for client applications.

  • No labels