...

The user should be able to see what dynamic configs are supported for each application. However, clients may not necessarily support the same dynamic configs even if they are using the same <user, client-id> entity. A partial solution to this is to store a map of client software names and versions to lists of supported configs in each <user, client-id> entity config. The Java producer and consumer clients can register the configs they they support and the broker can tie ties the registered keys to the serialized ClientInformation from the RequestContext so that this compatibility information can be displayed to the user. As long as the user knows what version of client each application is using, they will know which applications will be affected by each dynamic config.

A dynamic client config supported.configs can be added to hold the value serialized version of this config which will be a JSON of what is described above. map.

This makes the compatibility information refreshable since `supported.configs` can be deleted from an entity. Clients register configs every time they request configs, so the compatibility information will eventually be added again and continuously updated as new clients associate themselves clients of different versions register configs with the entity.

Public Interfaces

...

The same authorization that is necessary for {Describe,Alter}ClientQuotas, CLUSTER authorization, will be used when handling {Describe,Alter}ClientConfigsRequest.

Producer Changes

The Java producer will have a DynamicProducerConfig that will periodically fetch dynamic configs from the producer’s IO thread asynchronously. The interval on which dynamic configs are fetched will be the same amount of time as the interval for MetadataRequest, metadata.max.age.ms. It will use DescribeClientConfigsRequest as the RPC, validate the dynamic configs returned in DescribeConfigsResponse against the user provided configs, and log any configurations that are accepted. The client will reconfigure its acks value by using a method in DynamicProducerConfig that gets the current value of acks. The dynamic acks config will take precedence over user provided acks config unless the user provided configs require acks to be a certain value, such as enable.idempotence=true. In this case the dynamic update will be ignored.

...

The GroupCoordinator in the broker receives a group member’s session timeout upon the JoinGroupRequest and stores this with the rest of the group member's metadata. This means that to dynamically configure a consumer’s session timeout, the consumer must send a JoinGroupRequest. Currently, this could trigger an expensive rebalance operation when members are stable. To get around this, JoinGroup behavior will be changed in the broker so that the session timeout can be updated using JoinGroup without triggering a rebalance in stable group members. The Java consumer's initial DesccribeClientConfigsRequest DescribeClientConfigsRequest will still be done synchronously before the first JoinGroupRequest to avoid sending an unnecessary JoinGroupRequest.

There The Java consumer will be have a DynamicConsumerConfig that will periodically fetch dynamic configs. The interval on which dynamic configs are fetched will be the same amount of time as the interval for MetadataRequest, metadata.max.age.ms. It will use DescribeConfigsRequest as the RPC, validate the dynamic configs that are returned in DescribeConfigsResponse against the user provided configs and log any configurations that are accepted. The client will either reconfigure itself by changing the session timeout and heartbeat interval in the GroupRebalanceConfig, or discard the configs if the heartbeat interval is greater than or equal to the session timeout. The dynamic configs will take precedence over user provided client configs as long as the heartbeat interval is strictly less than the session timeout.

...

The user can also add configs specific to a client-id that will override the user's default dynamic configs:

bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --alter \
  --entity-type users \
  --entity-name alice \
  --entity-type clients \
  --entity-name clientid-override \
  --add-config acks=0,heartbeat.interval.ms=2000,producer_byte_rate=60000
  Completed updating config for user alice.

...