...

• Independence from the server node process (JVM) - issues and failures of the consumer shouldn't lead to server node instability.
• Notification guarantees and failover - i.e. track and save a pointer to the last consumed record. Continue notification from this pointer in case of restart.
• Resilience for the consumer - it's not an issue when a consumer temporarily consumes slower than data appear.

Description:

...

On the segment archiving, utility iterates it using the existing WALIterator and notificates notifications CDCConsumer of each record from the segment.

Design choices:

• CDC application works as a separate process.
• CDC relies on the existing Ignite mechanism - WAL.
• IEP Scope - deliver local WAL events to a local consumer.
• CDC keeps consumer offset in a special file.
  WAL process will start from this offset on restart.
• To prevent interference between the WAL archive process and CDC Ignite will create a hard link to the newly created segment in a special folder.
  After success processing, CDC will delete this link.
  Note, data will be removed from the disk only after CDC and Ignite will remove the link to a segment from both corresponding folders.
• To manage minimal event gap new configuration timeout introduced - WalForceArchiveTimeout.
• Flag to distinguish DataEntry on primary and backup nodes introduced.

CDCConsumer public API interface:

...