This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.
Current state: "Under Discussion"
Discussion thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
KafkaProducer is designed to be thread-safe and we encourage users to share a single producer instance across multiple threads[link]. These threads often need to send records to different topics. However, another important configuration—acks—is currently set at the producer level only. As a result, users needing different acks settings for each topic must create additional producer instances instead of reusing the existing one. This goes against our original intent and prevents users from reusing KafkaProducer instance.
Currently, we use a producer pool to handle different acks values, which requires 3x producer instances. Additionally, this approach creates many idle producers if a sensor with a specific asks setting has no data for a while.
In this KIP, we propose adding topic-level acks, enabling users reuse a single KafkaProducer even when different topics require different acks settings. This change will reduce the need for multiple producers and minimize resource usage on edge devices (e.g resource-constrained environments).
We add a new config TOPIC_ACKS_CONFIG and TOPIC_ACKS_DOC into ProducerConfig and define the format.
Format: acks.topic=acks.<TopicA>=<acks1>:acks.<TopicB>=<acks2>
Name | Type | Importance | Default | Description |
|---|---|---|---|---|
| acks.topic | String | LOW | null | This configuration item will set acks for specific topics. |