You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Status

Current state: Draft

Discussion thread: TODO

JIRA: TODO

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Quota management via Admin Client has gone through a couple drafts of proposals (KIP-248, KIP-422). While improvements have been made to the Admin interface for configuration handling, fitting quotas into the API would be restrictive because they don't fit the natural key-value pairing. Therefore, it'd be beneficial to have a quota-native API for administrating quotas, which would offer a more direct and less error-prone interface, convey additional useful information beyond what a configuration could provide, and provide for future extensibility as quotas types are added or evolve.

Background

Quotas are defined in terms of a user and client ID, where the user acts as a specific opaque principal, and the client ID as a more generic group identifier.

When setting quotas, an administrator has flexibility in how it specifies the user and client ID for which the quota applies to, where the user and client ID may be named, indicated as the default, or omitted entirely. This creates a hierarchy structure for which quotas apply, where the most specific, defined quota will be matched to a request's user and client ID.

As represented by the current ZK node structure, the order in which quotas are matched are as follows. Note <user> is a specified user name, <client-id> is a specified client ID, and <default> is a special default user/client ID represented as the literal string "<default>".

        /config/users/<user>/clients/<client-id>
        /config/users/<user>/clients/<default>
        /config/users/<user>
        /config/users/<default>/clients/<client-id>
        /config/users/<default>/clients/<default>
        /config/users/<default>
        /config/clients/<client-id>
        /config/clients/<default>

As such, reasoning around quotas can be complex, as it's not immediately obvious which quotas may apply to a given user and/or client ID. Providing descriptive information is a goal of this KIP.

Public Interfaces


Unknown macro: {dsadas}
Unknown macro: {dsadas}

/**

  • Describes how to filter quota entities.
    */
    public class QuotaFilter
    Unknown macro: { // Matches the default entity name. public static final String DEFAULT; // Matches only entities with the name omitted. public static final String NULL; // Matches every entity name. public static final String ANY; /** * @param user the user to match, or a sentinel filter value * @param clientId the client ID to match, or a sentinel filter value */ public QuotaFilter(String user, String clientId); public String user(); public String clientId(); }

public class ListQuotasOptions

Unknown macro: { // Default. }

public class ListQuotasResult

Unknown macro: { /** * @return the collection of entities that matched the filter */ Collection<QuotaEntity> entities(); }

/**

  • Lists entities matching the provided filter that have quotas defined.
    *
  • For example, to determine the quota for users that belongs to a particular client, set
  • filter.user() to QuotaFilter.ANY and filter.clientId() to the client ID.
    *
  • Alternatively, to determine which users have specific quota values, set filter.user()
  • to the user name, and filter.clientId() to QuotaFilter.NULL.
    */
    ListQuotasResult listQuotas(QuotaFilter filter);

/**

  • The quota types.
    */
    public enum QuotaType
    Unknown macro: { CONSUMER_BYTE_RATE, PRODUCER_BYTE_RATE, REQUEST_PERCENTAGE, }

/**

  • The quota configuration for a specific user and client ID.
    */
    public class QuotaConfig {

/**

  • Where in the hierarchy the quota value was specified.
    */
    public enum Source
    Unknown macro: { // In order of descending precedence. USER_CLIENT, // /config/users/<user>/clients/<client-id> USER_DEFAULT_CLIENT, // /config/users/<user>/clients/<default> USER, // /config/users/<user> DEFAULT_USER_CLIENT, // /config/users/<default>/clients/<client-id> DEFAULT_USER_DEFAULT_CLIENT, // /config/users/<default>/clients/<default> DEFAULT_USER, // /config/users/<default> CLIENT, // /config/clients/<client-id> DEFAULT_CLIENT, // /config/clients/<default> DEFAULT, // static default }

/**

  • Information about a quota value.
    */
    public class Value
    Unknown macro: { /** * @param source the source for the value * @param value the non-null value */ public Value(Source source, Double value); public Source source(); public Double value(); }

/**

  • Information about the value for a quota type.
    */
    public class Entry
    Unknown macro: { /** * @param value the active quota value * @param overriddenValues list of all values lower that are ignored due to being lower in the hierarchy */ public Entry(Value value, List<Value> overriddenValues); public Value value(); public List<Value> overriddenValues(); }

/**

  • Maps quota type to its configuration entry.
    *
  • Note that, depending upon the applied quota filter, this config may not map every
  • quota type to an entry. If a key is not contained in the map, then that quota type's value
  • is not specified.
    */
    public Map<QuotaType, Entry> config();
    }

/**

  • Identifies a quota entity for a given user and client ID. If the provided user or client ID is
  • `Optional.empty()`, then the default name is used, otherwise if null, then the name is omitted
  • entirely. At least one of user or client ID must be specified.
    *
  • For example:
    *
  • Unknown macro: {user="kafka-user", clientId="kafka-client-id"}
    corresponds to quota source `USER_CLIENT` with
  • specified user "kafka-user" and client ID "kafka-client-id".
    *
  • Unknown macro: {user="kafka-user", clientId=Optional.empty()}
    corresponds to quota source `USER_DEFAULT_CLIENT`
  • with specified user "kafka-user" and default client ID.
    *
  • Unknown macro: {user=null, clientId="kafka-client-id"}
    corresponds to quota source `CLIENT` for client with
  • ID "kafka-client-id".
    */
    public class QuotaEntity
    Unknown macro: { public QuotaEntity(Optional<String> user, Optional<String> clientId); public Optional<String> user(); public Optional<String> clientId(); }

public class DescribeQuotasOptions

Unknown macro: { // Whether to include the inherited values in the result. If false, then undefined values for // the config entries will be ommitted in the results. // // Default}

public class DescribeQuotasResult

Unknown macro: { Map<QuotaEntity, QuotaConfig> configs; }

/**

  • Describes the quotas for the provided entities.
    */
    DescribeQuotasResult describeQuotas(Collection<QuotaEntity> quotaEntities);

public class QuotaAlteration

Unknown macro: { /** * @param type the quota type to alter * @param the new value for the quota, otherwise if null, then the existing value is cleared */ public QuotaAlteration(QuotaType type, Double value); public QuotaType type(); public Double value(); }

public class AlterQuotasOptions

public class AlterQuotasResult

Unknown macro: { /** * @return map of a quota entity update to its future */ public Map<QuotaEntity, KafkaFuture<Void>> futures(); }

/**

  • Alters the quotas as specified for the provided quota entities.
    *
  • @param alterations the alterations to perform
  • @return the result of the alterations
    */
    AlterQuotasResult alterQuotas(Map<QuotaEntity, QuotaAlteration> alterations);



Proposed Changes

Compatibility, Deprecation, and Migration Plan

All changes would be forward-compatible, and no migration plan is necessary. It's outside the scope of this KIP to deprecate any functionality.

Rejected Alternatives




  • No labels