Versions Compared

Old Version 41

changes.mady.by.user Ryan Dielhenn

Saved on

compared with

New Version 42

changes.mady.by.user Ryan Dielhenn

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Introducing new entity types for kafka-configs.sh that producers and consumers can associate themselves with. This would make the tool more cumbersome to use and it is most intuitive that client configurations be dynamically altered with the clients and users entity types.

  • Use the {Describe, IncrementalAlter}Configs APIs. Client config entities are more dynamic than entities with a singular resource name and type which makes it hard to fit them into generic APIs that expect a distinct entity name and type.
  • Use the <user/client-id> hierarchy implemented for client quotas in KIP-55 and extended for the admin client in KIP-546. Quota APIs are different and quotas Quotas are inherently hierarchical but client configs are not, so it seems reasonable to use a hierarchy of shallow depth for dynamic client configs.
  • Interesting config hierarchies could be constructed if the Java producer and consumer resolved the dynamic configs instead of the broker. For example,  from most precedent to least precedent:
    • /config/users/<user>/clients/<client-id>
    • .properties file configs
    • /config/users/<user>
    • Static default configs defined in ProducerConfig and ConsumerConfig.
  • Adding a config enable.dynamic.config to producers and consumers to enable the feature. Adding the dynamic config supported.configs is a better alternative since the user doesn't have to remember what types of clients support the capability but instead can just see which entities have clients that are registered along with the configs that are supported for each software name and version.

  • Making certain client configurations topic level configurations on the broker.

    • The semantic for the ProduceRequest API would be undefined since the producer would not receive a response with an offset for the ProduceRequests with acks=0.

    • If this were implemented for acks there would also be quite a bit of overhead associated with extra round trips since the RecordAccumulator sends batches that may contain records from multiple topics. If these topics have different acks configurations the records would need to be sent in different batches based on the acks value.

    • For example, if a producer is consistently producing to 2 different topics and one is configured as acks=0 while the other is acks=-1. This would require twice the amount of round trips to produce the same number of messages.