...
- Initial state:
- No concurrent ConsistentCut process is running.
lastFinishedCutId
holds previous ConsistentCutId
, or null.
- User starts a command for creating new incremental snapshot:
- Ignite node inits a DistributedProcess with discovery message
SnapshotOperationRequest
that holds new ConsistentCutId
(goal is to notify every node in a cluster about running incremental snapshot). - DistributedProcess store the topology version topVer on which ConsistentCut started.
- Process of creation of consistent cut can be started by two events (what will happen earlier):
- Receive the
SnapshotOperationRequest#ConsistentCutId
by DiscoverySPI (by the DistributedProcess). - Receive the
ConsistentCutAwareMessage#ConsistentCutId
by CommunicationSPI (by transaction messages - Prepare, Finish).
- On receiving the
ConsistentCutId
, every node: - Checks whether ConsistentCut has already started (ConsistentCut is != nullis running) or finished (lastFinishedCutId == id) for this id, skip if it has.
- On baseline nodes:
- In the message thread atomically inits ConsistentCut:
- creates new ConsistentCut future.
- creates empty collection removedActiveTxs (This collection doesn't remove transactions unlike IgniteTxManager#activeTx does).
- In the background thread:
- Writes a ConsistentCutStartRecord to WAL with the received ConsistentCutId.
- Creates a copy (weakly-consistent) of IgniteTxManager#activeTx. Set listeners on those tx#finishFuture.
- For optimization it's safely exclude transactions that tx#status == ACTIVE. It's guaranteed that such transactions belongs After side.
- Creates a copy of removedActiveTxs (contains transactions that are might be cleaned from IgniteTxManager#activeTx). Set listeners on those tx#finishFuture.
- Set removedActiveTxs to null. We don't care of txs concurrently added to removedActiveTxs, they just don't land into "before" or "after" set and will be excluded from recovery.
- Fills removedActiveTxs if ConsistentCut is running and removedActiveTxs is not null:
- Every transaction is added into removedActiveTxsright before it is removed from IgniteTxManager#activeTx.
- For every listening transaction, the callback is called when transaction finished:
- check If transaction state is UNKNOWN or status is RECOVERY_FINISH, then complete ConsistentCut with exception.
- If transaction mapped to a higher topology version than ConsistentCut topVer, then put it into after.
- if tx#txCutId equals to local, then put transaction into after, otherwise put into before.
- After every listening transaction finished:
- Writes a ConsistentCutFinishRecord into WAL with the collections ( before, after ).
- Completes ConsistentCut future.
- On non-baseline nodes (clients, non-baseline servers):
- In case ConsistentCut is inited by CommunicationSPI then compare the ConsistentCutAwareMessage#topVer with local node order:
- Local node order equals to new topVer on the moment when node joined to a cluster.
- If the order is higher than ConsistentCut topVer it means the node joined after ConsistentCut started. Skip start ConsistentCut on this node.
- In the message thread creates completed ConsistentCut future.
- On all nodes (clients, non-baseline, baseline):
- While ConsistentCut != null wraps outgoing messages to ConsistentCutAwareMessage. It contains info:
- ConsistentCutId (to start ConsistentCut on remote node, if not yet).
- Some messages sets additional field txCutId. It set on the node that commits first:
- For 2PC it is an originated node.
- For 1PC it is a backup node.
- If txCutId set to null then transaction starts committing Before Consistent Cut started, otherwise After.
- For receiving ConsistentCutAwareMessage that make transaction committed (FinishRequest for 2PC, PrepareResponse for 1PC) Ignite set tx#txCutId = message#txCutId before it starts handling the received message.
- After ConsistentCut future finish, DistributeProcess automatically notifies a node-initiator about local procedure has finished.
- After all nodes finished ConsistentCut, on every node:
- Updates lastFinishedCutId with the current id.
- ConsistentCut future becomes null.
- Stops signing outgoing transaction messages.
- Node initiator checks that every node completes correctly.
- If any node complete exceptionally - complete Incremental Snapshot with exception.
...
{"serverDuration": 129, "requestCorrelationId": "61263a2b7e59c1b6"}