Page History

...

/** */
public class ConsistentCutStartRecord extends WALRecord {
	/** Marker that inits Consistent Cut ID. */
	private final ConsistentCutMarkerUUID markerid;
}


/** */
public class ConsistentCutFinishRecord extends WALRecord {
	/** Marker that inits Consistent Cut ID. */
	private final ConsistentCutMarkerUUID markerid;

    /**
     * Collections of transactions committed BEFORE.
     */
    private final Set<GridCacheVersion> before;

     /**
     * Collections of transactions committed AFTER.
     */
    private final Set<GridCacheVersion> after;
 }

...

1. Initial state:
   
   1. No concurrent ConsistentCut process is running.
   2. lastFinishedCutId holds previous ConsistentCutId, or null.
2. User starts a command for creating new incremental snapshot:
   1. Ignite node inits a DistributedProcess with special message holds new ConsistentCutMarker ConsistentCutId (goal is to notify every node in a cluster about running incremental snapshot).  
3. Process of creation of incremental snapshot can be started by two events (what will happen earlier):
   1. Receive the ConsistentCutMarker by the ConsistentCutId by discovery.
   2. Receive the ConsistentCutMarker ConsistentCutId by transaction message (Prepare, Finish)
4. On receiving the marker ConsistentCutId, every node: 
   1. Checks whether ConsistentCut has already started for this markerID, skip if it has.
   2. Checks local topVersion  with received in marker. Skip if it is different.
   3. In message thread atomically:
      1. creates new ConsistentCut future ConsistentCut<Boolean> future. Complete with true if consistent, otherwise false. 
      2. creates committingTx , goal (goal is to track COMMITTING transactions, that aren't part of IgniteTxManager#activeTx)
      3. starts signing outgoing messages with the ConsistentCutMarker ConsistentCutId.
   4. In background thread:
      
      1. Creates a copy of IgniteTxManager#activeTx. Set listeners on tx#finishFuture.
      2. Writes a ConsistentCutStartRecord  to WAL with the received ConsistentCutMarker . ConsistentCutId.
      3. Creates a copy of committingTxs. Set listeners on tx#finishFuture.
      4. Stops filling committingTxsCollects active transactions - concat of IgniteTxManager#activeTx and committingTxs .
5. While the DistributedProcess  is alive every node signs output transaction messages:
   1. Prepare messages signed with the ConsistentCutMarker ConsistentCutId (to trigger ConsistentCut  on remote node, if not yet).
   2. Finish messages signed with the ConsistentCutMarker ConsistentCutId  (to trigger...) and transaction ConsistentCutMarker ConsistentCutId  (to notify nodes which side of cut this transaction belongs to).
   3. Finish messages is signed with transaction ConsistentCutMarker ConsistentCutId on node that commits first:
      1. For 2PC it is an originated node.
      2. For 1PC it is a backup node.
6. For every receiving FinishMessage it put the transaction into committingTxs.
7. For every collected active listening transaction, it starts listening tx#finishFuture with callback. Callback the callback is called when transaction finished:
   

   1. check

      that transaction completes in consistent way (tx#state != UNKNOWN, tx.finalizationStatus != RECOVERY_FINISH). If it isn't then this cut is inconsistent. Complete ConsistentCut exceptionally.

      If transaction state is UNKNOWN or status is RECOVERY_FINISH, then complete ConsistentCut as inconsistent.

   2. if tx#txCutId equals to if tx#txMarker is null or differs from local, then transaction put into beforeif tx#txMarker equals to local, then transaction put into afterafter, otherwise it's put into before.
8. After every collected listening transaction finished:
   1. Writes a ConsistentCutFinishRecord  ConsistentCutFinishRecord  into WAL with the collections ( before, after ). 
   2. Stops filling committingTxs and clean this collection.
   3. Completes ConsistentCut  future , and as consistent.
   4. Note, that it continues to sign messages even after local ConsistentCut finish.
9. After ConsistentCut finish, DistributeProcess automatically notifies a node-initiator about finishing local procedure (with DistributedProcess  protocol).
10. After all nodes finished ConsistentCut :
    
    1. every node stops signing outgoing transaction messagesUpdates lastFinishedCutId with the current.
    2. ConsistentCut  future becomes null.Ignite
    3. node now in the initial state againStops signing outgoing transaction messages.
11. Node initiator checks that every node completes correctly and that topVer wasn't changed since start.
    1. If any node complete exceptionally , or topology changed - complete IS with exception.

...

Every ignite nodes tracks current ConsistentCutMarker :

class ConsistentCutMarker {
	UUID id;

	AffinityTopologyVersion topVer;
}

id is just a unique ConsistentCut  ID (is assigned on the node initiator).topVer is topology version on node initiator before Incremental Snapshot starts.

Signing messages

On the picture below on left side is a diagram of sending transaction messages. Before sending message it checks whether cut is running with cutMarker(). If it is then wrap message, otherwise send ordinary message (PrepareRequest in example).

...