Pluggable Cache Persistence and Overflow Stores

Overview

When Geode was first developed as a technology storage was fairly straightforward- Disk was the persistent store. Today, however, we see an increasingly diverse storage environment. Distributed stores, such as HDFS, HBase, Cassandra, and others are increasingly used to persist data for analysis and recovery. New persistent memory technologies are adding a new type of non-volatile storage to the mix. Having the core Geode team address every possible use case or store type that may be desired does not scale, nor should it be a gating function on the user community for enabling a new persistence layer. Instead we propose that by developing a properly defined pluggable interface and specification for pluggable persistent stores the Geode persistence layer becomes easily extensible and allow user and the community to add and contribute pluggable use case specific stores. Whether the same applies to over-flow stores is an open question for the community, as the requirements here are different, so we leave that as an open question.

Short list of initial requirements

• Disk store remains as is; all other persistent stores become pluggable
• Overflow is not changed- uses disk store always (open question on this requirement; should overflow be included?)
• A cache store manager layer is created to manage persistence stores
• Cache stores implement a well defined interface for read/write and store information/statistics exposure
• Cache store interface includes methods for getting details on cache store and for statistics
• Categories for store types will be defined, for example node-local or distributed stores to allow Geode logic to correctly handle persistence (e.g., avoid copy multiplication in the case of multi-coopy distributed stores)
• Cache store interface supports standard configuration method interface for a given store type type
• Cache stores are added in configuration via classpath and loaded and validated by the cache storage manager
• Storage manager expose JMX interfaces such that gfsh and dashboards can: list storage types available, list storage configuration parameters for a give store, list details and statistics for a given store.

Overflow Inclusion

Overflow has different potential requirements hence it is an open question as to whether is should be included in the pluggable interface. While one can see storage class memory (SCM) used for overflow this type of memory is likely to be exposed as a file system interface similar to any other disk. It is difficult to imagine other types of storage being used for overflow due to performance and latency requirements. The main use case seems to be for those using proprietary SCM interfaces to provide very fast low latency overflow. The open questions are (1) should overflow be included? (2) what is the incremental increase to the difficulty of implementation of pluggable stores by including overflow? (3) are the requirements similar enough to combine persistence and overflow, or should overflow, if done, be a separate proposal?

Technical Proposal

Technical proposal is added at 

https://cwiki.apache.org/confluence/display/GEODE/Proposal+for+pluggable+cache+stores