Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the latency of disks, several tuning options can be applied. Setting the page size of a memory region to match the page size of the underlying storage, using a separate disk for the WAL, and using production-level SSDs are just a few of them [https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning].

A persistent memory store with low latency and high capacity offers a viable alternative to disks. In light of this, we are proposing an experimental support for Intel Optane DC Persistent Memory (aka 3DXPoint, AEP) for Ignite based on a Low Level Persistent Library, https://github.com/pmem/LLPL/, that we implemented. LLPL offers access to persistent memory using MemoryBlocks allocated from a persistent Heap and uses the libpmemobj library from the Persistent Memory Development Kit (PMDK).

This experimental implementation gets rid of Ignite's current checkpoint process and the WAL file and relies a transactional memory block defined by LLPL to store both data and indexes into a persistent heap. 

Thus far, we have completed the basic persistent memory support for DataGrid and we can successfully run Cache API, Data Streamer, and SQL Grid examples. We still don't know the behavior under transactional mode and have not done any optimizations as well. 

Repo: https://github.com/mulugetam/ignite/tree/persistent-memory

PR: https://github.com/apache/ignite/pull/4381

Examples: examples/src/main/java/org/apache/ignite/examples/aep


  • No labels

Apache Ignite