...

The default quota configs will apply to authenticated users user principals if quota.secure is =true.

• quota.producer.default: If quota.secure=false, this is the default producer quota for each unique client- id. Otherwise, this is the default quota for each authenticated principal.
• quota.consumer.default: If quota.secure=false, this is the default consumer quota for each client-id. Otherwise, this is the default quota for each authenticated principal.

...

When quota.secure=true, quota related metrics will be generated for the authenticated principal principals rather than client-ids.

Tools

kafka-configs.sh will be extended to support authenticated user quotas and sub-quotas for users clients of a clientuser.  A new entity type “users” will be added.  The key-value pairs supported for users will be:

• producer_byte_rate=p : The total rate limit for the user’s producers set to p MB/sec
• consumer_byte_rate=c : The total rate limit for the user’s consumers set to c  MB/sec
• client_producer_byte_rates=clientA:pA,clientB:pB : The rate limit for the user’s producers with client-id clientA is set to pA MB/sec. Similarly for clientB. Other clients share p-pA-pB MB/sec.
• client_consumer_byte_rates=clientA:cA,clientB:cB : The rate limit for the user’s consumers with client-id clientA is set to cA MB/sec. Other clients share c-cA-cB MB/sec.

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Rejected Alternatives

quota.secureis set to  false as default to be consistent with Kafka 0.9.0.x. Hence the existing quota configurations will apply if new secure quotas are not defined. If quota.secure  is set to true and new quotas are configured for users, clients may be throttled based on the new quota configurations. But no client API changes are necessary to work with the new implementation.

Rejected Alternatives

Unified configuration for client-id and authenticated-principal based quotas

This KIP proposes to use a broker configuration option to switch between client-id based quotas and authenticated-principal based quotas for simplicity. An alternative would be to define a unified configuration where client-id based quotas are a special case of a unified quota config with the same username applied to all clients. The internal quota implementation will use common code for both options with only the quota-id being different. But the externally visible configuration and defaults are much simpler to define with separate options that are consistent with 0.9.0.x since it is unlikely that a cluster would support both.

 If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.