Page History

...

To support multiple keys for reading encrypted data it is required to store key identifier on each encrypted page and on each encrypted WAL record. The key identifier is a sequential counter, and should be the same on all nodes.

1. Check that all baseline nodes online.
2. Start distributed process GROUP_KEY_CHANGE_PREPARE, each node
   1. verifies that reencryption not in progress
   2. ensures that new key identifier does not exists
   3. adds new key
3. After successful completion of PREPARE, start distributed process GROUP_KEY_CHANGE_FINISH, each node
   1. sets new key for writing
   2. adds the mapping "WAL segment -> *old* key identifier" (to safely cleanup this key in the future)
   3. stores current pages count as total pages for background re-encryption (?on applicable partitions).
   4. starts background re-encryption

Background re-encryption

Process applies only for OWNING/MOVING partitions that are not currently cleared.

Scan all pages from specified range (metapageid + [offset -> total])

...

Re-encryption progress is stored into metapage (int offset, int total), updates during checkpoint.

The process aborts for partition that is scheduled for evicting/clearing during re-encryption.

Cleanup old key

Old group key will be removed when

...