Motivation

Data storage in 2.x versions of Ignite is tightly coupled to offheap-based implementations called Page Memory used in both in-memory and persistent modes. These implementations work well for some workloads but aren't optimal for others (e.g. in persistent mode Page Memory suffers from write amplification problem and both implementations use B+Tree as a primary key index which has its limitations compared to hash index).

In Ignite 3 we want to provide more flexibility for end users to choose the best storage implementation for their use cases. To achieve this we need to define a minimal API and hide all implementation details about particular storage implementations behind it.

Description

API

As a starting point I suggest to define the following interfaces (of course they'll evolve when other components start integrating them):

/** Interface providing methods to read, remove and update keys in storage. */
public interface Storage {
	/** Reads a DataRow for a given Key. */
	public DataRow read(Key key);

	/** Removes DataRow associated with a given Key. */
	public void remove(Key key);

	/** Executes an update with custom logic implemented by UpdateClosure interface. */
	public update(Key key, UpdateClosure clo);

	/** Obtains Iterator over some DataRows in storage. */
	public Iterator<DataRow> iterator(/* parameters */).
}

DataRow and Key abstractions are not defined yet but should provide additional meta information about keys e.g. table ID the key belongs to.

The first time this interface looks enough for needs of existing components.

Implementation of transactions needs a LockManager enabling to lock and unlock particular keys in Storage:

/** Interface enabling obtaining locks on Keys in Storage. */
public interface LockManager {
	/** Locks given Key. */
	public void lock(Key key);

	/** Unlocks given Key. */
	public void unlock(Key key);
}

Modules structure

As we aim to have more than one option of storage we need to keep storage api separated from implementation, so we'll have one module containing api and a separate module for each specific implementation.

Specific implementations

Persistent storages

1. Page Memory with persistence support used in Ignite 2.x is a good candidate as a persistent storage in 3.0. Some of issues with current implementation (e.g. write amplification, excessive disk consumption for recovery purposes) can be addressed during porting it to 3.0 code base.
   This approach with more details is described here: https://github.com/apache/ignite-3/blob/ignite-14647/modules/vault/README.md
   
   
2. RocksDB is another candidate that provides better performance for write-intensive workloads than Page Memory: https://rocksdb.org/

In-memory storages

For in-memory storages we can consider both heap and off-heap solutions, including porting Page Memory without persistence support.

Risks and Assumptions

//NA

Discussion Links

//NA

Reference Links

//NA