Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

As SinkTask can only be created by SinkConnector, MirrorSinkConnector will be implemented and follow the most same logics as current MirrorSourceConnector. To minimize the duplicate code, a new class, e.g. "MirrorCommonConnector", may be proposed to host the common code as a separate code change merged before this KIP.

WorkerSinkTask

Due to the fact that group offsets from target cluster is single-source-of-truth, each time when MirrorSInkTask restarts, it loads group offsets from target cluster and supply them to the consumer in WorkerSinkTask so that the consumer will rewind to the last committed offsets without duplicating or missing data. However in our test, we ran into 

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-10370
 so resolving it is a prerequisite of this KIP.

Migration from MirrorSourceConnector to MirrorSinkConnector /w EOS

...