...

KafkaProducer is designed to be thread-safe and we encourage users to share a single producer instance across multiple threads[link]. These threads often need to send records to different topics. However, another important configuration—acks—is currently set at the producer level only.

As a result, users needing different acks settings for each topic must create additional producer instances instead of reusing the existing one and this approach requires 3x producer instances. This goes against our original intent and prevents users from reusing KafkaProducer instance.

This is harmful for edge device(resource-constrained device) since users need to Currently, we use a producer pool to handle different acks values, which requires 3x producer instancesvalue. Additionally, this approach creates many idle producers if a sensor with a specific asks setting has no data for a while. 

In this KIP, we propose adding topic-level acks, enabling users reuse a single KafkaProducer even when different topics require different acks settings. This change will reduce the need for multiple producers and minimize resource usage on edge devices (e.g resource-constrained environments).

Public Interfaces

We add a new config TOPIC_ACKS_CONFIG  and TOPIC_ACKS_DOC into ProducerConfig and define the format.

...