Motivation

IEP-43 introduces persistent caches snapshots feature. This feature highly adopted by Ignite users.

In-memory caches snapshots will simplify the following use-cases:

• In-memory cluster restarts.
• Version upgrade.
• Disaster recovery.
• DC/Hardware replacement.
• Data motion.

Description

in-memory snapshots will resuse existing snapshot code when possible. So key design decisions stay the same. It assumed that reader knows and understand IEP-43. So design description will focus on difference on persistence and in-memory snapshot.

Creation

1. PME + Checkpoint guarantees to make sure of visibility of all dirty pages:

Only PME required for in-memory snapshots. We can set write listener during PME because no concurrent transactions allowed.

See:

• PartitionsExchangeAware#onDoneBeforeTopologyUnlock
• IgniteSnapshotManager#onDoneBeforeTopologyUnlock
• SnapshotFutureTask#start

2. Persistent pages contains CRC to make sure data integrity while storing on disk:

CRC for each page must be calculated and written to snapshot metadata during snapshotting.

CRC must be checked during restore. 

3. Metadata:

• StoredCacheData.
• binary_meta
• marshaller

must be properly prepared and saved during snapshot.

Risks and Assumptions

DataRegionConfiguration#persistenceEnabled=false  for in-memory caches by the definition.

The same value must be for DataRegionConfiguration when cache group restored from in-memory snapshot.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

• IEP-43: Cluster persistent caches snapshots

Tickets