Page History

...

1. Initial state:
   1. Ignite nodes started from snapshot, or other consistent state (after graceful cluster stop / deactivation).
2. Ignite node inites a global snapshot, by starting DistributedProcess (by discovery IO).
3. Every (incl. client and non-baseline) node starts a local snapshot process after receiving a message from DistributedProcess. 
4. Phase 1: 
   
   1. Write WAL record - commit LocalState.
   2. fix snpVersion  = GridCacheVersion#next(topVer)
   3. Collect all active transactions originated by near node, which nearXidVersion is less than snpVersion 
   4. Note, continue collecting transactions that are less than snpVersion , and for which local node is near (to exclude them later)
   5. after finishing all collected transactions: notify Ignite nodes with snpVersion (with DistributedProcess protocol).
5. After all nodes finished first phase, they received Map<UUID, GridCacheVersion> from other nodes. 
6. Phase 2: Only server baseline nodes continue work there:
   1. Collect all active transactions, find their near node (by GridCacheVersion#nodeOrderId), filter them with known GridCacheVersion
   2. Await all such transaction completed.
   3. Write WAL record with the received map.
7. Phase 3: 
   1. Stop collecting near transactions that are less than local snpVersion and send them to other nodes.
   2. On receiving such map, write a new WAL record again, that contains additional skip collection.
8. After finishing Phase 3, process of snapshot is finished.

...

1. Increments of GridCacheVersions is CAS operations from different threads. But the version is assigned to a transaction in non-atomic way. No guarantee that snpVersion  is greater than version of transaction created after fixing snpVersion. Ignite should track such transactions:
   1. With fair locking while creating and assigning version to transaction - possible performance degradation.
   2. With additional filter after preparing a snapshot (4.d) - now there are 3 steps for preparing snapshot.
2. For case OPTIMISTIC + PRIMARY_SYNC we can miss backup transaction - looks like need additional sync process? TBD  
3. Client and non-baseline nodes has a job to do: collecting of transactions, awaiting them finished, sending a response. It could be non-reliable, as client nodes can be short-lived:
   
   1. Also should handle special cases when transaction is committed after client node gone and there is no info about it actual version.
4. No safe previous record to restore, if some incremental snapshots created. Need to filter all history.

...