...
Current state: Under Discussion
Discussion thread: https://listss.apache.org/thread.html/r0b90330094ef356997399e4b657e42e6b09fdd731d3069d8bb9841d9%40%3Cdev.flink.apache.org%3Eflip-151-discussion
JIRA:
Released:
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
For (1), potentially multiple unconfirmed versions have to be maintained (corresponding to multiple checkpoints, to avoid full snapshot on failure, see unconfirmed checkpoints)
Removed keys
Currently, entries are just “unlinked” on removal. Therefore, they wouldn’t be included in the snapshot, but will be present after recovery.
...
- An entry can be removed many times - keep (and write) it only once
- An entry can be re-added (and then re-removed)
- A snapshot with a removal can still be unconfirmed by JM by the time of the next checkpoint (see unconfirmed checkpoints)
- An entry key can be mutable, such as BinaryRowData (this doesn’t cause issues with non-incremental snapshots because removal takes place straight ahead, without tracking)
...