Page History

...

Task producers will be given a transactional ID of ${groupId}-${connector}-${taskId}, where ${groupId} is the group ID of the Connect cluster, ${connector} is the name of the connector, and ${taskId} is the ID of the task (starting from zero). Users will not be able to override this with the worker-level producer.transactional.id  or connector-level producer.override.transactional.id  property. If they attempt to do so, the user-provided value will be ignored and a warning message will be logged notifying them of this fact.

The worker-level offset.flush.timeout.ms  property will be ignored for exactly-once source tasks. They will be allowed to take as long as necessary to complete an offset commit, since the cost of failure at that point is to fail the source task. Currently, all source task offset commits take place on a single shared worker-global thread. In order to support source task commits without a timeout, but also prevent  laggy tasks from disrupting the availability of other tasks on the cluster, the worker will be modified to permit simultaneous source task offset commits.

It may take longer than the transaction timeout for a task to flush all of its records to Kafka. In this case, there are two actions that users can take to nurse their connector back to health: reconfigure it to produce records with a lower throughput, or increase the transaction timeout for the producers used by the connector. We will include include these steps in the error message for a task that fails due to producer transaction timeout.

SourceTask record commit API

...