Page History

...

1. Message - transaction message (...FinishRequest for 2PC, ...PrepareResponse for 1PC)
   1. it is guaranteed that it's sent after all transaction DataRecords are written into WAL on sending node.
2. Channel - TCP communication connection from one node to another, by that the Messages is sent.
3. ChannelState - for single channel it's a set of messages Messages that was sent (changed LocalState of sending node), but not received yet (hasn't changed LocalState of receiving node)committed yet on receiver.
   1. In Ignite we can think that incoming ChannelState is represented by active transactions in PREPARING+ state (the Message might be sent by other nodes, but isn't received yet), and greater, states.
4. IncrementalSnapshot - on Ignite node it is represented with 2 WAL records (ConsistentCutStartRecord commits the WAL state, ConsistentCutFinishRecord describes the ChannelState). It guarantees that every node in cluster includes in the snapshot:
   1. transactions committed before ConsistentCutStartRecord and weren't included into ConsistentCutFinishRecord#after();
   2. transactions committed between ConsistentCutStartRecord and ConsistentCutFinishRecord and were included into ConsistentCutFinishRecord#before().
5. Marker - mark that piggy backs on the Message, and notifies a node about running snapshot.

   1. After IS start and before finish all PrepareRequest, FinishRequest are wrapper by ConsistentCutMarkerMessage instead of regular Message. This is done to notify target node ⁣⁣via communication channel about running IS.

...

1. Initial state:
   1. Ignite WAL are in consistent state relatively to previous full or incremental snapshot.
   2. Every Ignite node has local ConsistentCut future equals to null.
   3. Empty collection committingTxs (Set<GridCacheVersion>) that goal is to track COMMITTING+ transactions, that aren't part of IgniteTxManager#activeTx.
2. Ignite node inits a IncrementalSnapshot, by starting DistributedProcess with special message holds new ConsistentCutMarker.
3. Every nodes starts a local snapshot process after receiving the marker message (whether by discovery, or by communication with transaction message) 
   1. Atomically: creates new ConsistentCut future, creates committingTxs, starts signing outgoing messages with the ConsistentCutMarker.
   2. Write a snapshot record ConsistentCutStartRecord to WAL with the received ConsistentCutMarker (commits LocalState).
   3. Collect of active transactions - concat of IgniteTxManager#activeTx and committingTxs 
   4. Prepares 2 empty collections - before andafter cut (describes ChannelState).
4. While global Consistent Cut is running every node signs output transaction messages:
   1. Prepare messages signed with the ConsistentCutMarker (to trigger ConsistentCut on remote node, if not yet).
   2. Finish messages signed with the ConsistentCutMarker (to trigger...) and transaction ConsistentCutMarker (to notify nodes which side of cut this transaction belongs to).
   3. Finish messages is signed on node that commits first (near node for 2PC, backup or primary for 1PC).
5. For every collected active transaction, node waits for Finish message, to extract the ConsistentCutMarker and fills before, after collections:
   1. if received marker is null or differs from local, then transaction on before side
   2. if received color equals to local, then transaction on after side
6. After all transactions finished:
   1. Writes a finish WAL record with ChannelState (before, after). 
   2. Stops filling committingTxs.
   3. Completes ConsistentCut future, and notifies a node-initiator about finishing local procedure (with DistributedProcess protocol).
7. After all nodes finished ConsistentCut, every node stops signing outgoing transaction messages - ConsistentCut future becomes null.

...