...

The Apache Ignite cluster balance procedure currently with enabled persitence currently doesn't utilize network and storage device throughout to its full extent. The balance procedure processes cache data entries one by one which is not efficient enough for the cluster with enabled persistence.

Description

The Apache Ignite needs cache rebalacing with transfering partition files using zero-copy algorithm based on extension of communication SPI. 

Process overview

There are two participants in the process of balancing data: 

...

1. demaner (receiver of partition files)
2. supplier (sender of partition files)

The process of ordering cache groups for rebalancing remains the same. The whole process is described in terms of rebalance single cache group:

1. The GridDhtPreloaderAssignments created for cache group (type of Map<ClusterNode, GridDhtPartitionDemandMessage>)

CommunicationSpi extend

To achieve the zero-copy file tramsmission approach the CommunicationSpi needs to support direct (pipe) connections between two nodes:

1. The WritableByteChannel needs to be accesses on the supplier side
2. The ReadableByteChannel needs to be read on the demander side

Partition file transmission

The new abstraction layer over partition file needs to be created to support FileChannel#transferTo the zero copy feature of Java NIO and to support transferring partition file by chunks.

Checkpoint on demand

Temp-WAL recovery

Checkpoint on demand

Risks and Assumptions

// Describe project risks, such as API or binary compatibility issues, major protocol changes, etcIf operating system (JDK must be greater 1.6.0_18) does not support zero-copy file transfer, sending a file with FileChannel#transferTo might fail or yield worse performance. For example, sending a large file doesn't work well in Windows.

Discussion Links

// Links to discussions on the devlist, if applicable.

...