Versions Compared

Old Version 25

changes.mady.by.user Aditya Auradkar

Saved on

compared with

New Version 26

changes.mady.by.user Aditya Auradkar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Configuration Management

We need a mechanism to configure the default quotas and the per-user overrides. There is general agreement that we eventually need dynamic configs per-user to fully operationalize quotas but for the purposes of this proposal, we will proceed with static configs.

Code Block
// Default bytes-out per consumer.
quota.consumer.default=2M
quota.producer.default=2M
 
// Overrides
quota.producer.override="clientA:4M;clientB:10M"
quota.consumer.override="clientC:3M;clientD:5M"

 There is a separate discussion for dynamic configs per-user that isn't fully fleshed out. If it makes sense and people agree on a final design, we can model quotas using it.

 

Tooling/Monitoring Changes

Along with this feature, we are proposing to add the following tools/admin operations:

  • Dynamically disable/enable quota enforcement for the entire cluster. Such a feature is very useful while rolling out this feature to production environments.
  • Ability to disable quotas on a per-client basis. For example: we may not want to throttle mirror makers.
  • Dynamically change quotas for any client id.

We also plan to expose the fraction of quota used on a per-client basis via JMX (0-100%, where 100 means throttled)Upon implementation, we will start exposing per-client byte rate metrics metrics to JMX. Since these are new metrics there are no backward compatibility issues. These metrics will be documented prior to the release.

Compatibility, Deprecation, and Migration Plan

...