Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Kafka Connect is the standard framework for building data pipelines into and out of Kafka. Integrating Share Groups into Connect's sink connector

runtime gives every existing sink connector access to queue semantics **with a configuration change only** -- no connector code modifications required.

1.4 Delivery Semantics


ModeGuaranteeMechanism
At-least-once(default)Every record is delivered at least once; duplicates possible on failureShareConsumer.acknowledge(ACCEPT) after successful task.put(); records released on failure for re-delivery 
Exactly-onceEvery record is delivered exactly onceKIP-1289 transactional acknowledgments: producer.sendShareAcksToTransaction() binds acknowledgments to the output transaction

At-least-once is the initial target. Exactly-once requires KIP-1289 (Transactional Acknowledgments for Share Groups) to be implemented and is described as a future phase.

...