Page History

...

To support multiple keys for reading encrypted data it is required to store key identifier on each encrypted page and on each encrypted WAL record. The key identifier is a sequential counter, and should be the same on all nodes.

1. Check that all baseline nodes online.
2. Start distributed process CACHE_GROUP_KEY_CHANGE_PREPARE, each node
   1. verifies that re-encryption not in progress
      
   2. ensures that new key identifier does not exists
   3. adds new key
3. After successful completion of PREPARE, start distributed process CACHE_GROUP_KEY_CHANGE_FINISH, each node
   1. saves logical WAL record (ENCRYPTION_STATUS_RECORD) with current page count in partitions.
   2. stores current page count as total pages for background re-encryption on partitions.
   3. adds the mapping "WAL segment -> *old* key identifier" (to safely cleanup this key in the future)
   4. sets new key for writing
   5. starts background re-encryption

...