Page History

...

When dump created each partition's data is stored into in a separate file. This is performed in several dump executor threads concurrently. At the same time user updates might also trigger storing dump entries in transaction threads to any partition, not necessarily to partitions currently processed by dump executor. From disk writing point of view often random switches to several files is not good for overall throughput. And throughput is one of the important characteristics of creating dump process.

Proposed solution

Main idea

The main idea is to write large number of bytes per each disk write operation, minimizing switching between files and doing overhead work.

Buffer usage for writing

The file write operation should send large number of bytes. There are 3 options for this

...

Need to investigate: Assuming that we want to write the same total number of bytes at what range of byte buffer array size invocation of write(ByteBuffer) and write(ByteBuffer[]) is the same?

High level view

ByteBuffer trip cycle will look like this

...

Open question: Does it make sense to use non-blocking queue here for faster readput/write take queue operations on the queue?

Disk Writer Thread

Does 3 operations

...

Takes buffers from queue and returns to the pool. This operation will probably be fast enough so that extra thread will could be redundant.

Open file limit

...

• keep track of number of open files and last write time for each partition
• identify number of allowed open files for creating dump, for example, like half of ulimit `ulimit -nn`
• when number of files reaches threshold, close the least recently written partition file without draining the partition queue
• reopen closed file when written
• log warning message about non-optimal configuration of partitions count and limit on open files

...

Compression is done after encryption (if enabled).

This can remain in the same thread (part/trans thread). But because of high CPU usage and decreased output size the bottleneck might move from disk writing to compression. If this is observed, we should extract compression to another thread.

Compression output is a sequence of buffers which can't be reordered. Compression per partition can't be done in 2 threads simultaneously.

...

Is there a protection against simultaneous multiple concurrent dump creation? Do we need one?

...