Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. /config/users/<user>/clients/<client-id>
  2. /config/users/<user>
  3. /config/users/config/users/<default>/clients/<client-id>
  4. /config/users/<default>/clients
  5. /config/users
  6. /config/clients/<client-id>
  7. /config/clients

...

Dynamic configuration for default quotas

  • Default user quota for <user, client-id> will be stored in Zookeeper at the top level config for /config/users. If not specified, user quota will be unlimited and <user, client-id> defaults will apply. Default user quota can be updated dynamically.Default quota for <user, client-id> /<default>/clients.
  • Default user quota will be stored in /config/users/<default>/clients. If not specified, client-id defaults will applyZookeeper at the top level config for /config/users.
  • Default client-id quota will be stored in Zookeeper at the top level config for /config/clients. If not specified, the broker properties quota.producer.default, quota.consumer.default will be used as the default client-id quota for clients of users with unlimited quota.

...

  1. If quota override is defined for <userN, clientX> in /config/users/userN/clients/clientX, this quota is allocated for the sole use of <userN, clientX>.
  2. If user quota override is defined for userN, clientX in /config/users/userN shares this quota with other clients of userNIf default user quota is defined in /config/users, clientX shares this default quota with other clients of userN
  3. If <user, client-id> quota is defined in /config/users/<default>clients/clientX, this quota is allocated for the sole use of <userN, clientX>
  4. If default <user, client-id> quota is defined in /config/users/<default>clients, this quota is allocated for the sole use of <userN, clientX>
  5. If default user quota is defined in /config/users, clientX shares this default quota with other clients of userN
  6. If client-id quota override is defined for clientX in /config/clients/clientX, this quota is shared across client-id <clientX> of all users
  7. If dynamic client-id default is configured in /config/clients, this default quota is shared across client-id <clientX> of all users
  8. If quota.producer.default is configured for the broker in server.properties, this is shared across client-id <clientX> of all users
  9. Client is not throttled

...

  • Simple client-id based quotas are configured using client-id quota override, dynamic client-id default and static quota.producer.default : (steps  6, 7, 8, 9)
  • Simple user-principal based quotas are configured using user quota override and user quota default : (steps 2, 35, 9)
  • More specific <user, client-id> quotas and defaults for users and client-ids can be configured if required: (steps 1 - 9)

...

kafka-configs.sh will be extended to support a new entity type "users". Quota configuration for users will be provided as key-value pairs to be consistent with other configuration options. Hence no new command line arguments will be added to the tool. The tool will parse the key-value pairs specifying rate limits, validate these and convert them to the equivalent JSON for persistence in Zookeeper. The existing entity “clients” will continue to be supported to set client-id quotas for users with unlimited quota. The tool will be extended to accept multiple entity types to configure <user, client-id> quotas. The tool will also be updated to configure default quotas at the top-level (/config/users or , /config/clients, /config/users/<default>/clients).

Compatibility, Deprecation, and Migration Plan

...