...

Motivation

Currently batch updates on page memory level are not implemented in Ignite. Internal structures, such as BPlusTree and FreeList, do not support them. The performance of batch operations (putAll, datastreamer, preloader) can be improved by implementing batch updates on page memory level.

Competitive analysis

Profiling the current rebalancing process shows that most of the time is spent on working with FreeList and BPlusTree (details).

Process overview

Currently, when updating the batch of cache entries in off heap, each of them is processed separately.

...

1. Remove a page from FreeList, which has enough storage space (if there is no such page - allocate a new one).
2. Lock page.
3. Write cache entry data.
4. Update page counters.
5. Unlock page.
6. Based on the remaining free space insert page into FreeList.

Proposed changes

...

1. Batch writing to data pages.
2. Batch updates in BPlusTree.

Batch writing to data pages.

Find the most free page with enough space for data row in FreeList (if there is no such page - allocate new one) and fill it up to the end.

Improves average insertion time by reducing count of FreeList actions on each data row (insert/remove pages, pages locking and page counters updates).

Batch updates in BPlusTree.

Improves the time of searching and inserting a range of keys.

Risks and Assumptions

// Describe project risks, such as API or binary compatibility issues, major protocol changes, etc.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

// Links to various reference documents, if applicable.

Tickets