Page History

...

Wraps several ByteBuffers, extends ByteBuffer. It is created in ByteBufferPool#acquire and destroyed in ByteBufferPool#release when all internal buffers returned to the pool.

Disk Writer Thread

We can often see that disk IO operation is the bottleneck in Ignite. So we should try to make writing to disk efficient. 

...

• put(ByteBuffer) - blocking method
• ByteBuffer[] takeAll() - blocking method, takes all elements
• long bytes() - fast non-blocking method 

Partition Queue Dispatcher Thread

The thread will periodically check disk writer queue size making sure that disk writer always has enough data to process. So, minimum number of elements will be 2 for single disk writer. Once the size goes down, the thread will scan sizes of partition queues and choose the winner with the maximum number of bytes. After that it will take all buffers from the queue and will put them together with file info (channel) to the disk writer queue.

In case of multiple disk writers additionally a flag should be passed, which will be considered when choosing largest partition queue and will be reset on buffer release.

Disk Writer Queue and Free Queue 

They could be simple ArrayBlockingQueues.

Open question: Does it make sense to use non-blocking queue here for faster put/take queue operations?

Disk Writer Thread

Does 3 operations

• Take next ByteBuffer[] from queue
• Save to disk
• Send ByteBuffer[] to Free Queue

ByteBuffer Releaser Thread

Takes buffers from queue and returns to the pool. This operation will probably be fast enough so that extra thread could be redundant.

Some functions

Open file limit

The solution assumes that all partition dump files are open simultaneously. This could break when number of partitions (default 1024) is bigger than OS limit for open files (ulimit -n).

...

• keep track of number of open files and last write time for each partition
• identify number of allowed open files for creating dump, for example, like half of `ulimit -n`
• when number of files reaches threshold, close the least recently written partition file without draining the partition queue
• reopen closed file when written
• log warning message about non-optimal configuration of partitions count and limit on open files

Encryption

In case of encryption 2 buffers will be required. First one as usual will be used for keeping serialized entry, and the second one will be for storing encrypted data from the first buffer. After encryption the first buffer will be returned back to the pool. The second buffer will go to the partition queue.

Note: Avoid deadlock when processing large objects not fitting into pool size. 

Compression

Compression is done after encryption (if enabled).

...

The size of required buffer isn't known while requesting buffer from pool. It is preferable to use medium size buffers. And it is ok to get a bit smaller buffer for this, fill it, send it to queue and request another medium size buffer from pool.

Shutdown / Execution Completion

Once dump creating is over, all resources will be cleaned.

• Newly created threads finish.
• PolledByteBuffer will clean returning buffers to heap (HeapBB) or os (DirectBB).

Error handling

In case of error the whole execution will stop. Resuming creating dump isn't considered, the only option is to rerun creating dump.  

...