Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updates based on discussion thread

...

Proposed Changes

  1. We'll add the isClosing method to the SinkTaskContext as mentioned above.
  2. We'll add the implementation for that to WorkerSinkTaskContext
  3. We'll add a setter named setClosingInProgress to WorkerSinkTaskContext that'll change the internal state
  4. In WorkerSinkTask.commitOffsets:
    1. If the closing parameter is set to true, we'll invoke setClosingInProgress on the context to true. This will occur before the invocation of preCommit.
    2. Before commitOffsets returns, it will setClosingInProgress will setClosingInProgress back to false.
  5. Add a note to the preCommit javadoc to indicate the importance of checking SinkTaskContext.isClosed when running,

This change will give preCommit the opportunity to provide some additional commits that it would like Kafka Connect to make before the task is fully shut down.

...

  • We considered adding a closing parameter to preCommit, but that would be a breaking change for all connectors and would require them to change their code when upgrading to the next version of Connect. It would also break binary compabilityunless it was done very carefully. The additional complexity would buy us marginal extra value, if any at all, so we decided to go with the addition to the context object.