Page History

...

1. Initial state:
   
   1. No concurrent ConsistentCut process is running.
   2. lastFinishedCutId holds previous ConsistentCutId, or null.
2. User starts a command for creating new incremental snapshot:
   1. Ignite node inits a DistributedProcess with special message holds new ConsistentCutId (goal is to notify every node in a cluster about running incremental snapshot). 
   2. DistributedProcess fix the topology version topVer on which ConsistentCut started.
3. Process of creation of incremental snapshot can be started by two events (what will happen earlier):
   1. Receive the ConsistentCutId by discovery.
   2. Receive the ConsistentCutId by communication (transaction message (messages - Prepare, Finish).
4. On receiving the ConsistentCutId, every node: 
   1. Checks whether ConsistentCut has already started or finished for this ID, skip if it has.
   2. If received form CommunicationSPI then compare the topVer with mapped topology version of received transaction message. Skip ConsistentCut initialization if the received version is higher.
   3. In the message thread atomically inits ConsistentCut:
      1. creates new ConsistentCut future.
      2. creates committingTx (goal is to track COMMITTING transactions, that aren't part of IgniteTxManager#activeTx)
      3. starts signing outgoing messages with the ConsistentCutId.
   4. In the background thread:
      
      1. Creates a copy of IgniteTxManager#activeTx. Set listeners on those tx#finishFuture.
      2. Writes a ConsistentCutStartRecord  to WAL with the received ConsistentCutId.
      3. Creates a copy of committingTxs. Set listeners on those tx#finishFuture.
      4. Set committingTxs to null.
5. While the DistributedProcess  is running every node is signing output transaction messages:
   1. Prepare and Finish messages are signed with the ConsistentCutId (to trigger ConsistentCut  on remote node, if not yet).
   2. Finish messages are signed additionally with txCutId on the node that commits first (if it's not null then transaction starts committing After Consistent Cut):
      1. For 2PC it is an originated node.
      2. For 1PC it is a backup node.
6. For every receiving FinishMessage it puts  puts the transaction into committingTxs, and marks the transaction with extracted from the message txCutId.
7. For every listening transaction, the callback is called when transaction finished:
   

   1. check If transaction state is UNKNOWN or status is RECOVERY_FINISH, then complete ConsistentCut with exception.

   2. If transaction mapped to a higher topology version than ConsistentCut topVer, then put it into after.
   3. if tx#txCutId equals to local, then put transaction put into after, otherwise it's put into before.
8. After every listening transaction finished:
   1. Writes a ConsistentCutFinishRecord  into WAL with the collections ( before, after ). 
   2. Completes ConsistentCut  future.
   3. Note, that it continues to sign messages even after local ConsistentCut finish.
9. After ConsistentCut finish, DistributeProcess automatically notifies a node-initiator about local procedure has finished.
10. After all nodes finished ConsistentCut, on every node:
    
    1. Updates lastFinishedCutId with the current id.
    2. ConsistentCut  future becomes null.
    3. Stops signing outgoing transaction messages.
11. Node initiator checks that every node completes correctly.
    1. If any node complete exceptionally - complete Incremental Snapshot with exception.

...

class ConsistentCutAwareMessage {
	/** Original transaction message. */
	Message msg;

	/** Consistent Cut ID. */
	UUID cutId;

	/** Cluster topology version on which Consistent Cut started. */
	long topVer;
}

Also some messages require to be signed with additional ConsistentCutId to check it them on primary/backup node.

...

class ConsistentCutAwareTransactionFinishMessage extends ConsistentCutAwareMessage {
	/** Consistent Cut ID after which transaction committed. */
    @Nullable UUID txCutId;
}

...