ID | IEP-74 |
Author | |
Sponsor | |
Created |
|
Status | DRAFT |
Data storage in 2.x versions of Ignite is tightly coupled to offheap-based implementations called Page Memory used in both in-memory and persistent modes. These implementations work well for some workloads but aren't optimal for others (e.g. in persistent mode Page Memory suffers from write amplification problem and both implementations use B+Tree as a primary key index which has its limitations compared to hash index).
In Ignite 3 we want to provide more flexibility for end users to choose the best storage implementation for their use cases. To achieve this we need to define a minimal API and hide all implementation details about particular storage implementations behind it.
As a starting point I suggest to define the following interfaces (of course they'll evolve when other components start integrating them):
/** Interface providing methods to read, remove and update keys in storage. */ public interface Storage { /** Reads a DataRow for a given Key. */ public DataRow read(Key key); /** Removes DataRow associated with a given Key. */ public void remove(Key key); /** Executes an update with custom logic implemented by UpdateClosure interface. */ public update(Key key, UpdateClosure clo); /** Obtains Iterator over some DataRows in storage. */ public Iterator<DataRow> iterator(/* parameters */). }
Proposed API is based on IgniteCacheOffheapManager interface from Ignite 2.x which is significantly simplified and cleaned up from Ignite 2.x concepts like partitions, caches and data stores.
However it keeps useful ideas like KeyCacheObject (in the form of Key class, its shape and responsibilities are subject of discussion and clarification), CacheDataRow (in the form of DataRow) and InvokeClosure needed to effectively implement read-modify-write pattern (in the form of UpdateClosure).
The first time this interface looks enough for needs of existing components.
Implementation of transactions needs a LockManager enabling to lock and unlock particular keys in Storage:
/** Interface enabling obtaining locks on Keys in Storage. */ public interface LockManager { /** Locks given Key. */ public void lock(Key key); /** Unlocks given Key. */ public void unlock(Key key); }
As we aim to have more than one option of storage we need to keep storage api separated from implementation, so we'll have one module containing api and a separate module for each specific implementation.
For in-memory storages we can consider both heap and off-heap solutions, including porting Page Memory without persistence support.
//NA
//NA
//NA