Page History

...

To support multiple keys for reading encrypted data it is required to store a key identifier on each encrypted page and on each encrypted WAL record (see for more details). The key identifier is a sequential counter and should be the same on all nodes.

...

1. re-encryption completed for cache group (and after that at least one checkpoint was completed)
2. last WAL segment in which the key was used is removed

Changes in memory page format

PageMetaIO and PagePartitionMetaIO format

Reencryption status requires an additional 8 bytes on the meta page of each partition.
Index partition uses PageMetaIO to read/write meta information.
Each other partition uses PagePartitionMetaIO to read/write meta information.

Partition meta starts just after the end of the page meta.
To store an additional 8 bytes partition meta shifted by 8 bytes.

WAL delta records have also been modified to store re-encryption status.

Encrypted (persisted) page format

Each encrypted page has reserved free space to store CRC of encrypted data.
The size of this free space depends on the size of the encryption block, but cannot be less than 8 bytes (Ignite default encryption implementation (KeystoreEncryptionSpi) uses AES with 16 bytes block size).

Added 1 byte for encryption key ID on each encrypted page (after CRC).
(WAL records ENCRYPTED_RECORD and ENCRYPTED_DATA_RECORD have been changed accordingly)

Fault tolerance

Distributed key rotation

...