Status

Current state: UNDER DISCUSSIONACCEPTED

Discussion thread: <link to mailing list DISCUSS thread> https://mail-archives.apache.org/mod_mbox/samza-dev/202106.mbox/%3cCAMja7KdMNU_Zk-vDnwcm4GSJs==126-mU6dJgTsoukzkxzf+qw@mail.gmail.com%3e

JIRA: 

...

Incremental restore is just the reverse of the incremental backup outlined above. We consider the Checkpoint topic as the source of truth, and restore the checkpoint directory target by retrieving the Index blob ID from Checkpoint obtained from Checkpoint topic. We then retrieve the Index blob from the Blob Store. Index blob contains lists called filesAdded and filesRemoved. We retrieve the filesAdded blob list from the Blob Store and add them to the checkpoint directory. Similarly, we delete the filesRemoved list from the local checkpoint directory. We also compare the same files in remote and local checkpoint directory and match their checksum, size and permission attributes to see if they are the same file. If a file’s content has changed since the remote checkpoint was created, we restore the file from the remote checkpoint and delete the local file.

Image RemovedImage Added

Post-Commit cleanup and Garbage Collection

...