...

...

The Apache Ignite can support cache rebalancing as transferring partition files using zero copy algorithm [1] based on an extension of communication SPI and Java NIO API. When the partition file has been transferred to the demander node there are to possible approaches can be implemented to preload entries from particular partition file.

Hot swap cache data storage storage

Disadvantages:

• A long and complex index reduild procedure that requires the development of additional crash recovery guarantees.
  The index rebuild procedure starts immediately when the partition file is fully received from the supplier node. If the node crashes in the middle of the rebuilding index process it will have an inconsistent index state at the further node startup. To avoid this a new index-undo WAL record must be logged within rebuilding and used on node start to remove previously added index records.

Preload entries from loaded partition file

Disadvantages:

• The approach will require a new temporary FilePageStore to be initialized. It must be created as a part of the temporary cache group or in the separate temporary data region to provide reusing machinery of iteration over the full partition file.

Proposed Changes (

...

Hot swap)

Process Overview

In the process of balancing data:

...

• We must stop updating indexes on demander when the data is ready to be transferred from the supplier node. All async cache updates on demander must not cause the index update;
• The previous partition metadata page and all stored meta information must be destroyed in memory PageMemory and restored from the new partition file;

...

The node is ready to become partition owner when partition data is rebalanced and cache indexes are ready. For the message-based cluster rebalancing approach indexes are rebuilding simultaneously with cache data loading. For the file-based rebalancing approach, the index rebuild procedure must be run finished before the partition state is set to the OWNING state. 

...