...

1. Quota configuration for user principal. This prevents users generating heavy traffic from monopolizing resources and impacting the performance of other users in a multi-tenant cluster.
2. Sub-quotas for clients of an authenticated user.  Like the current client-id implementation, this enables a user to rate-limit some producers or consumers to ensure that they don’t impact other more critical clients.  For instance, users may be able to rate-limit an auditing client running in the background, leaving resources always available for a critical event processing client.
3. Client-id quotas for users with unlimited quota. Existing quota configuration for client-ids will continue to be applied to users with unlimited quota, but these will be applied as per-user quota for clients that share both user principal and client-id.
4. Shared quotas for groups of clients that span multiple users will no longer be supported.

The goal of this KIP is to provide a single unified quota implementation with a simple configuration for client-id based quotas, a similar simple configuration for user quotas and a more complex but flexible configuration for hierarchical quotas.

Public Interfaces

Quota Entity

...

• producer_byte_rate : The total rate limit for the user’s producers
• consumer_byte_rate : The total rate limit for the user’s consumers
• client_producer_byte_rates: Comma separated list of reserved sub-quotas for client-ids of the user (eg. clientA:=10,clientB:=20). Clients not listed share the remaining quota of the user.
• client_consumer_byte_rates: Comma separated list of reserved sub-quotas for client-ids of the use. (eg. clientA:=30,clientB:=40). Clients not listed share the remaining quota of the user.

...

// Quotas for user1 (without sub-quotas).
// Zookeeper persistence path /users/<encoded-user1>
{
    "version":1,
    "config": {
        "total" : {"producer_byte_rate":"1024",
        "consumer_byte_rate":"2048"}
    }
}
// Quotas for user2 with sub-quotas for clients.
// Zookeeper persistence path /users/<encoded-user2>
{
    "version":1,
    "config": {
    "total"   : {"producer_byte_rate":"4096",
        "consumer_byte_rate":"8192"},
    "clients": {
       "clientA" : {"client_producer_byte_raterates" : "clientA=10","consumer_byte_rate":"20"},clientB=30",
       "clientB" : {"producer_byte_rate":"30"," "client_consumer_byte_raterates" : "clientA=20,clientB=40"}
    }
} 
// Quotas for client-id clientA of users without config override if default user quota is unlimited. 
// Zookeeper persistence path /clients/clientA
{
    "version":1,
    "config": {
        "producer_byte_rate":"100",
        "consumer_byte_rate":"200"
    }
}

...