Page History

...

However, one example is that the implementation of the database's changed data capture (CDC) is not allowed to take checkpoint in the snapshot phase[1]. The current implementation is holding a checkpoint lock in the snapshot phase, which is deprecated in the current versionSimilar to the problem of CDC, the external data source could have some pieces of data that are transaction data. It is unreasonable to take a checkpoint in the middle of transaction data.

Therefore, what we want to address in this FLIP are:

• Add an interface for operators so that the operator implementors could utilize it to check the availability of taking snapshots when a checkpoint is triggered.
• Implement it to the SourceOperator and SourceReader to address the problem of CDCof not taking checkpoints in the middle of an atomic piece of data.

With this implementation, users can determine when to reject the checkpoint.

...