THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
// These settings are designed to ensure there is no data loss. They *may* be overridden via configs passed to the
// worker, but this may compromise the delivery guarantees of Kafka Connect.
...
producerProps.put(ProducerConfig.ACKS_CONFIG, "all");
producerProps.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "1");
Idempotent Idempotent producers have been introduced in KIP-98 - Exactly Once Delivery and Transactional Messaging and allow produces to be idempotent and therefore:
- Prevent duplicates on network failure (idempotence)
- Increase performance of safe producers by using MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION=5. See:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key KAFKA-5494
Currently Kafka Connect does not benefit from these two improvements which could be very valuable to existing connectors in order to improve both the performance and safety
Public Interfaces
We will add the following code to Worker.java:
...