You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 14 Next »

Status

Current state: Draft

Discussion thread: TODO

JIRA: TODO

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Quota management via Admin Client has gone through a couple drafts of proposals (KIP-248, KIP-422). While improvements have been made to the Admin interface for configuration handling, fitting quotas into the API would be restrictive because they don't fit the natural key-value pairing. Therefore, it'd be beneficial to have a quota-native API for administrating quotas, which would offer a more direct and less error-prone interface, convey additional useful information beyond what a configuration could provide, and provide for future extensibility as quotas types are added or evolve.

Background

Quotas are defined in terms of a user and client ID, where the user acts as a specific opaque principal, and the client ID as a more generic group identifier.

When setting quotas, an administrator has flexibility in how it specifies the user and client ID for which the quota applies to, where the user and client ID may be named, indicated as the default, or omitted entirely. This creates a hierarchy structure for which quotas apply, where the most specific, defined quota will be matched to a request's user and client ID.

As represented by the current ZK node structure, the order in which quotas are matched are as follows. Note <user> is a specified user name, <client-id> is a specified client ID, and <default> is a special default user/client ID represented as the literal string "<default>".

        /config/users/<user>/clients/<client-id>
        /config/users/<user>/clients/<default>
        /config/users/<user>
        /config/users/<default>/clients/<client-id>
        /config/users/<default>/clients/<default>
        /config/users/<default>
        /config/clients/<client-id>
        /config/clients/<default>

As such, reasoning around quotas can be complex, as it's not immediately obvious which quotas may apply to a given user and/or client ID. Providing descriptive information is a goal of this KIP.

Public Interfaces

    /**
     * Describes how to filter quota entities.
     */
    public class QuotaFilter {
        // Matches the default entity's name.
        public static final String DEFAULT;

        // Matches only entities with the name omitted.
        public static final String NULL;

        // Matches every entity name.
        public static final String ANY;

        /**
         * @param non-null user the user to match, or a sentinel filter value
         * @param non-null clientId the client ID to match, or a sentinel filter value
         */
        public QuotaFilter(String user, String clientId);
        public String user();
        public String clientId();
    }

    public class ListQuotasOptions {
        // Default.
    }

    public class ListQuotasResult {
        /**
         * @return the collection of entities that matched the filter
         */
        Collection<QuotaEntity> entities();
    }

    public interface Admin extends AutoCloseable {
        /**
         * Lists entities matching the provided filter that have quotas defined.
         *
         * For example, to determine the quota for users that belongs to a particular client, set
         * filter.user() to QuotaFilter.ANY and filter.clientId() to the client ID.
         *
         * Alternatively, to determine which users have specific quota values, set filter.user()
         * to the user name, and filter.clientId() to QuotaFilter.NULL.
         */
        ListQuotasResult listQuotas(QuotaFilter filter);
    }
    /**
     * The quota types.
     */
    public enum QuotaType {
        CONSUMER_BYTE_RATE,
        PRODUCER_BYTE_RATE,
        REQUEST_PERCENTAGE,
    }

    /**
     * The quota configuration for a specific user and client ID.
     */
    public class QuotaConfig {

        /**
         * How the quota is applied to the quota entity.
         */
        public enum Entity {
            USER,        // The user principal.
            CLIENT_ID,   // The client ID.
            // Others may be added in the future.
        }

        /**
         * How the quota is applied to the quota entity.
         */
        public enum Behavior {
            SET,        // Sets the value, overwriting and lower precedence SET values.
            RESTRICT,   // Adds a limit (upper bound) to higher precedence values.
            // Others may be added in the future.
        }

        /**
         * The units for the quota.
         */
        public enum Units {
            BPS,             // Bytes per second, on a global scale.
            BPS_PER_BROKER,  // Bytes per second, on a per-broker level.
            SHARE,           // The shares, used for determining proportional quota.
        }

        /**
         * Information about a specific quota configuration value.
         */
        public class Value {
            /**
             * @param source the entity source for the value
             * @param behavior the behavior for applying the quota
             * @param units the units to apply to the value
             * @param value the non-null value
             */
            public Value(Map<Entity, String> source, Behavior behavior, Units units, Double value);
            public Map<Entity, String> source();
            public Behavior behavior();
            public Units units();
            public Double value();
        }

        /**
         * Information about the value for a quota type.
         */
        public class Entry {
            /**
             * @param value the active quota value
             * @param overriddenValues list of all values lower that are ignored due to being lower in the hierarchy
             */
            public Entry(Value value, List<Value> overriddenValues);
            public Value value();
            public List<Value> overriddenValues();
        }

        /**
         * Maps quota type to its configuration entry.
         *
         * Note that, depending upon the settings in the request's options, this config may not map every
         * quota type to an entry. If a key is not contained in the map, then that quota type's value
         * is not specified.
         */
        public Map<QuotaType, Entry> config();
    }

    /**
     * Identifies a quota entity. If a entity key type has a value, then it's a specified name,
     * if the value is the empty string, then the default name is used, otherwise if the key is
     * omitted or the value is null, then the name is omitted entirely.
     *
     * For example:
     *
     *   {user="kafka-user", clientId="kafka-client-id"} corresponds to quota source `USER_CLIENT` with
     *   specified user "kafka-user" and client ID "kafka-client-id".
     *
     *   {user="kafka-user", clientId=Optional.empty()} corresponds to quota source `USER_DEFAULT_CLIENT`
     *   with specified user "kafka-user" and default client ID.
     *
     *   {user=null, clientId="kafka-client-id"} corresponds to quota source `CLIENT` for client with
     *   ID "kafka-client-id".
     */
    public class QuotaEntity {
        public QuotaEntity(Map<Entity, String> entity);
        public Optional<String> user();
        public Optional<String> clientId();
    }

    public class DescribeQuotasOptions {
        // Whether to include the inherited values in the result. If false, then undefined values for
        // the config entries will be ommitted in the results.
        //
        // Default: true.
        DescribeQuotasOptions setWantInheritedValues(boolean wantInheritedValues);

        // Whether to include the overridden values for every quota type in the result.
        //
        // Default: true.
        DescribeQuotasOptions setWantOverriddenValues(boolean wantOverriddenValues);
    }

    public class DescribeQuotasResult {
        Map<QuotaEntity, QuotaConfig> configs;
    }

    public interface Admin extends AutoCloseable {
        /**
         * Describes the quotas for the provided entities.
         */
        DescribeQuotasResult describeQuotas(Collection<QuotaEntity> quotaEntities);
    }
    public class AlterQuotasOp {
        /**
         * @param type the quota type to alter
         * @param the new value for the quota, otherwise if null, then the existing value is cleared
         */
        public QuotaAlteration(QuotaType type, Double value);
        public QuotaType type();
        public Double value();
    }

    public class AlterQuotasOptions {
        // Default.
    }

    public class AlterQuotasResult {
        /**
         * @return map of a quota entity update to its future
         */
        public Map<QuotaEntity, KafkaFuture<Void>> futures();
    }

    public interface Admin extends AutoCloseable {
        /**
         * Alters the quotas as specified for the provided quota entities.
         *
         * @param alterations the alterations to perform
         * @return the result of the alterations
         */
        AlterQuotasResult alterQuotas(Map<QuotaEntity, AlterQuotasOp> alterations);
    }



Proposed Changes

Compatibility, Deprecation, and Migration Plan

All changes would be forward-compatible, and no migration plan is necessary. It's outside the scope of this KIP to deprecate any functionality.

Rejected Alternatives




  • No labels