Cache encryption key rotation required in case of it compromising or at the end of crypto period (key validity period). in addition, such feature is required to provide support for encrypt and descrypt existing caches in the future.
Payment card industry data security standard (PCI DSS) requires that key-management procedures include a defined cryptoperiod for each key type in use and define a process for key changes at the end of the defined cryptoperiod(s). An expired key should not be used to encrypt new data, but it can be used for archived data, such keys should be strongly protected (section 3.5 - 3.6) [1].
The maximum recommended key lifetime is 2 years [2], and on average it is supposed to be changed every few months [3].
MS SQL Server provide rotation of database encryption key with background re-encryption of existing data [4]. Oracle and MySQL, out of the box, do not provide an automatic procedure for rotating tablespace keys, master key rotation is supported [5][6], Currently, TDE is being developed for PostgreSQL, but support for tablespace key rotation is not planned [7].
At the moment, encryption occurs at the pagememory level, when a page is written to the pagestore or WAL.
This strategy is similar to partition partition snapshotting - create partition snapshot encrypted with the new key and then replace the original partition file with the new one.
Sequentially read all the pages from the datastore and mark as dirty, log them into WAL. Checkpointer writes the pages encrypted with the new key.
This strategy requires changing the format of the encryption page to store the identifier (number) of the encryption key (for recovery). Each encrypted page has a space reserved for a page crc (4 bytes) that has an encryption block size (minimum 8 bytes).
copy | in place | |
---|---|---|
The amount of disk input-output | 2x | 2x+ (depends on WAL settings) |
The amount of required disk space | 2x | limited by WAL settings |
Performance(rough estimate) | faster | slower |
Implementation complexity (rough estimate) | ||
Stable topology, read-only | simple | simple |
Online updates | complex | simple |
Unstable topology | complex | simple |
The overall process consists of the following steps
To support multiple keys for reading encrypted data it is required to store key identifier on each encrypted page and on each encrypted WAL record. The key identifier is a sequential counter, and should be the same on all nodes.
Process applies only for OWNING/MOVING partitions that are not currently cleared.
Scan all pages from specified range (metapageid + [offset -> total])
Re-encryption progress is stored into metapage (int offset, int total), updates during checkpoint.
The process aborts for partition that is scheduled for evicting/clearing during re-encryption.
Old group key will be removed when
If GROUP_KEY_CHANGE_PREPARE has not been successfully completed on all nodes, the process is interrupted and must be restarted.
When the process restarts, a new key identifier is generated (an unused key will be removed on finish-phase, when we set the new key for writing).
It is possible that the node will fail after adding a new key, but before setting it for writing (as an active key).
This node doesn't know whether the PREPARE phase was successful or not, therefore, it does not know which key is currently being used for writing.
By default, it will try to rejoin with the old key, if the join is rejected, then it should be possible to manually set the correct key identifier using the system property or command line tools.
When a non-baseline node joins a cluster (with baseline change), it cleans up all existing data, so this shouldn't be a problem case.
If the node fails during re-encryption, after restarting it must continue re-encryption from the stored offset (if checkpoint failed it should restore physical records from WAL, as usual).
// TBD
Cluster-wide process consists of the following steps:
After completion of the key change preparation process, a new distributed process is initiated to complete the key change.
The distributed process configured action initiates partition swapping on each node (this action may require suspension of local or global operations if WAL records can be reordered during key change).
After changing the encryption key, new WAL records will be encrypted with the new key. However, it must be possible to read older WAL records (at least to support historical rebalance).
reference documentsFor each cache, instead of a key, it is necessary to keep a history of keys in the form WALPointer -> key
(stored the maximum pointer for which the associated key is applicable).
When removing a WAL segment to which WALPointer(s) refers - key(s) should be also removed.
When the WAL is cleared, respectively, the key history must also be cleared (except the last one).
The re-encryption procedure does not start if there are LOST partitions in the cache group or any baseline node is missing (this is a limitation of the initial design and should be improved in the future).
The cache stop operation is rejected, for cache groups in which re-encryption is performed.
By canceling the re-encryption procedure is meant clearing all temporary data.
If a node crashes during the replacement of the partitions, the original backup copies of the partitions are restored when the node starts.
If major topology changes during key rotation - cancelling whole procedure.
Minor topology changes should not affect re-encryption procedure.
If the partition is scheduled for eviction during re-encryption, cancel the re-encryption of this partition.
Re-encryption is a very long process that may never end due to a restriction on the stability of the major topology (if the topology changes the process starts from the beginning).
// TBD
New method will be introduced
public IgniteFuture<Void> changeGroupKey(Collection<Integer> groups) throws IgniteCheckedException;
// TBD
Re-encryption process state.
// TBD