Motivation

Master keyrotation required in case of it compromising  is required if it has beencompromisedor at the end of  the crypto period (key validity period). 

Design assumes that , an administrator will provide an ability to get a new master key by EncryptionSPI from underlying storage.

...

Definitions

• MK – MasterKey. Encrypts cache group keys. Master key is stored in some key storage. Master keys are identified by String masterKeyIdname.

Prerequisites

New master key should must be available to EncryptionSPI for each server node. The cluster must be active.

Process management

Users can control the master key rotation process key via  some kind of user interface(CLI, JMX, Java API,  etc). 

• JAVA API:
  ignite.encryption().changeMasterKey(String masterKeyIdmasterKeyName)  - starts master key rotation process.
  String ignite.encryption().getMasterKeyIdgetMasterKeyName()  - gets current master key idname.
  
  
• JMX:
  changeMasterKey(String masterKeyIdmasterKeyName)  - starts master key rotation process.
  String getMasterKeyIdgetMasterKeyName()     - gets current master key idname.
  
  
• CLI:
  # Starts master key rotation.
  ./control.sh --encryption change_master_key newMasterKeyIdnewMasterKeyName
  
  # Displays cluster's current master key idname.
  ./control.sh --encryption get_master_key_name
  

Process description

...

Master key change process consist of two phases:

1. Prepare master key change.
2. Perform master key change.

Each phase is a distributed process.

Prepare master key change

The goal is to verify that all server nodes have the same master key.

...

The server node begins prepare phase with the MasterKeyChangeRequest that contains:

1. New master
 
2. key
id
3. name.
 
5. New master key
hash
6. digest.
When 

Each server node executes the following actions:

1. Obtains a digest of a

    node processed message following actions are executed: It obtain hash of

   new master key. If the digest is unavailable, the process completes with an error.

2. Compares it with the one in messagethe request.
3. If it differs then error added to the message.the process completes with an error checking the consistency of the master key digest.
4. Stores locally master key name and digest.

The coordinator starts the perform phase when the prepare phase is completed without errors.

Perform master key change

The coordinator node starts the prepare phase with the MasterKeyChangeRequest that contains:

1. New master key name.
2. New master key digest.

Each server node executes the following actions:

1. It checks that the cluster is active (WAL must be writable to correctly log changes and survive cluster restarts). Otherwise, the process completes with the error.
2. Checks that master key name and digest is the same as it was taken from the prepare phase. Otherwice, log it and cancel the process.
3. If on step1 there are some errors we log it and cancel process. Otherwise got to step3.
The ChangeMasterKeyFinishMessage action message is sent by discovery as a custom event.
4. Action message sould contain:
   
   1. New master key id.
When server node processed message following actions are executed: 
6. Blocks creation of encrypted
 cache key. 
7. groupkeys.
8. Re-encrypts
Reencrypt
9. all cache group keys with new master key in a temporary datastructure. No changes in MetaStore.
Create
10. Creates WAL logical record (ChangeMasterKeyRecord ) that consist of:
    1. New master key
id
11. 1. name.
       
    2. Reenctyped cache group keys.
Write
12. Writes cache group keys to MetaStore.
13. Unblock Unblockscreation of encrypted cache key. groupkeys.

Distributed process

Distributed process is a cluster-wide process that accumulates single nodes results to finish itself.

The process consists of the following phases:

1. The initial request starts the process. The InitMessage sent via discovery.
2. Each server node processes the initial request and sends the single node result to the coordinator. The SingleNodeMessage sent via communication.
3. The coordinator accumulate all single nodes results and completes process. The FullMessage sent via discovery.

Several processes of the same type can be started at the same time.

Guarantees:

• Survives on topology and coordinator change (the SingleNodeMessage with a result will be redirected to the new one).
• The exec and the finish actions will be called only ones.

Process completion

...

Process The process completes when all nodes in cluster will process action messagethe perform phase completed (all nodes have been re-encrypted their keys).

Corner cases

Node

...

was down during key rotation

...

. MasterKeyChangeRecord not found.

If some node was unavailable during master key rotation process it will unable to join to the cluster because it has old master key has.

To update this node design introduce master key recovery process:user should run Ignite with system property (IGNITE_MASTER_KEY_NAME_TO_CHANGE_BEFORE_STARTUP=newMasterKeyName)

The node will Option 1 (auto). Cluster send to joining node his master key id. The joining node re-encrypt cache keys with new MK and tries try to join to the cluster.

Option 2 (manual). Administrator change master key id in the configuration. When the node starts it reads master key id from meta storage. If keys differ node will re-encrypt cache keys and tries to join to cluster.

Node left during key rotation process(was starting re-encrypt cache keys):

Node was down during key rotation. MasterKeyChangeRecord found.

A node should not try to join to the cluster before the process of ChangeMasterKeyRecord. Regardless of whether the key rotation was finished successfully or not, the recovery will  be from the record.

1. If we found ChangeMasterKeyRecord  in the process node recovery it was passed to EncryptionManager .
2. When MetaStore becomes writable, EncryptionManager 
3. If during node recovery with logical records we found ChangeMasterKeyRecord it passed to EncryptionManager.
4. When MetaStore becomes available for write, EncryptionManager writes new cache group keys to it.

Node join during key rotation process

...

Reject node join. It may lead to inconsistent master keys in cluster. (Or if it possible to delay until key rotation process ends)

...

Starting cache during key rotation process

...

Cache keys must not be created during the master key rotation process. So, a node will throw an exception if a user users will start cache during the key rotation process. Moreover, if group keys were generated before the master key was change, starting the cache will be rejected (case of client node starts the cache).

Node couldn’t complete the perform phase

Node will process the critical failure error. Failure handler must stop the node to prevent inconsistent keys in the cluster.

Public java API changes

EncryptionSpi

The concept of the masterKeyId will be added to the cache keys encryption process in EncryptionSpi :

...

New methods

...

byte[] masterKeyDigest();

...

byte[] masterKeyDigest(String masterKeyId);

...

byte[] encryptKey(Serializable key);

...

byte[] encryptKey(Serializable key, String masterKeyId);

will be introduced:

• setMasterKeyName(String masterKeyName)  // Sets "current" master key name
• String getMasterKeyName()  // Gets "current" master key name

The following methods will work with master key that was set by previous method:

• byte[] masterKeyDigest() 
• byte[] encryptKey(Serializable key) 
• Serializable

...

• decryptKey(byte[]

...

• key)

...

Serializable decryptKey(byte[] key, String masterKeyId);

This is necessary so that Ignite can decrypt cache keys with the old master key and encrypt with the new one.

Code changes

Meta Storage

Meta storage will store name of master key. Key name from meta storage has a higher priority to key name from EncryptionSpi 

where masterKeyId - master key id. If null the default key will be used for compatibility reason.

IgniteConfiguration

New methods will be added to the IgniteConfiguration:

public IgniteConfiguration setEncryptionMasterKeyId(String keyId) - sets master key id.

...

Code changes

MetaStorage

MetaStorage will store master key id.

Node attribute

Currently, the joining node send sends hash MK for validation in attributes. Attributes can't be modified at runtime. So the joining node will send hash MK in JoiningNodeDiscoveryData .

Tickets