Page History

...

For Ignite implementation it's proposed to use only single node to coordinate algorithm. User starts a command for creating new ConsistentCut on single node and this node becomes responsible for coordinating Consistent Cut iteration:

...

Consistent and inconsistent Cuts

Consistent Cut, in terms of Ignite implementation, is such cut that correctly finished on all baseline nodes - ConsistentCutStartRecord and ConsistentCutFinishRecord are written.

"Inconsistent" Cut is such a cut when one or more baseline nodes hasn't wrote ConsistentCutFinishRecord. It's possible in cases:

1. any errors appeared during processing local Cut.
2. if a transaction is recovered with transaction recovery protocol (txCutVer is unknown).
3. if transaction finished in UNKNOWN state.
4. baseline topology change, Ignite nodes finishes local Cuts running in this moment, making them inconsistent.

ConsistentCutVersion

Every ignite nodes tracks current ConsistentCutVersion:

class ConsistentCutVersion {
	long version;
}

`version` is a simple counter. It's guaranteed it is raising monotonically, due to it is incremented by discovery communication.

ConsistentCutVersion initialization

ConsistentCutVersion can be initialized with:

For changed server topology: 

1. On start a server node checks local MetaStorage for pre-stored latest ConsistentCutVersion, default is 0.
2. This version is packed with JoiningNodeDescoveryData and sent to coordinator.
3. Every server node on receiving TcpDiscoveryNodeAddedMessage:
   1. finishes local ConsistentCut.
   2. checks received ConsistentCutVersion, and if it's greater than local it updates local version to the received version.

For client node on start-up just sets its version to 0. It automatically updates with cluster version by receiving next GridNearTxPrepareResponse message.

After finished restoring (PITR) the version is used for restoring is used and set after reading WAL archives.

Description of Algorithm 

...

1. It initialize a new version - increment of local ConsistentCutVersion 
2. It sends ConsistentCutStartRequest with new version to every node in a cluster (piggy backed on DistributedProcess).

...

1. upgrades version
2. collects active transactions and decides which side of ConsistentCut they belong to
3. writes start mark into WAL (see below).

...

1. Near (transaction originated node) for 2PC.
2. Backup (or primary if backups=0) for 1PC.

1. Every node uses local ConsistentCutVersion to sign transaction messages (see below "Signing messages").
2. New global ConsistentCut is started by user command.
   1. It initialize a new version - increment of local ConsistentCutVersion 
   2. It sends ConsistentCutStartRequest with new version to baseline nodes in a cluster (piggy backed on DistributedProcess).
3. Every node can receive new version by two ways: with the ConsistentCutStartRequest or by transaction message signed with new version.
4. On receiving new version node inits local ConsistentCut:
   1. upgrades local version
   2. prepares ConsistentCut future
   3. collects active transactions and decides which side of ConsistentCut they belong to
   4. writes start mark into WAL (see below).
5. Node that first commits transaction signes Finish message with local ConsistentCutVersion (txCutVer)
   1. Near (transaction originated node) for 2PC.
   2. Backup (or primary if backups=0) for 1PC.
6. For collected active transaction nodes check order of events relative to ConsistentCut by comparing the txCutVer with local ConsistentCutVersion:
   1. if local version is greater than txCutVer then transaction belongs to BEFORE side
   2. if local version is equal to txCutVer then transaction belongs to AFTER side
   3. Node, that local version never is less than txCutVer. Because node must process ConsistentCutVersion before applying message with this version.
7. After every transaction was finished - node writes FinishRecord with transaction info into WAL.
8. Notifies a node-initiator about finishing local procedure (with DistributedProcess protocol).

Consistent and inconsistent Cuts

Consistent Cut, in terms of Ignite implementation, is such cut that correctly finished on all baseline nodes - ConsistentCutStartRecord and ConsistentCutFinishRecord are written.

"Inconsistent" Cut is such a cut when one or more baseline nodes hasn't wrote ConsistentCutFinishRecord. It's possible in cases:

1. any errors appeared during processing local Cut.
2. if a transaction is recovered with transaction recovery protocol (txCutVer is unknown).
3. if transaction finished in UNKNOWN state.
4. baseline topology change, Ignite nodes finishes local Cuts running in this moment, making them inconsistent.

ConsistentCutVersion

Every ignite nodes tracks current ConsistentCutVersion:

class ConsistentCutVersion {
	long version;
}

`version` is a simple counter. It's guaranteed it is raising monotonically, due to it is incremented by discovery communication.

ConsistentCutVersion initialization

ConsistentCutVersion can be initialized with:

For changed server topology: 

1. On start a server node checks local MetaStorage for pre-stored latest ConsistentCutVersion, default is 0.
2. This version is packed with JoiningNodeDescoveryData and sent to coordinator.
3. Every server node on receiving TcpDiscoveryNodeAddedMessage:
   1. finishes local ConsistentCut.
   2. checks received ConsistentCutVersion, and if it's greater than local it updates local version to the received version.

For client node on start-up just sets its version to 0. It automatically updates with cluster version by receiving next GridNearTxPrepareResponse message.

After finished restoring (PITR) the version is used for restoring is used and set after reading WAL archives

...

1. if local version is greater than txCutVer then transaction belongs to BEFORE side
2. if local version is equal to txCutVer then transaction belongs to AFTER side
3. Node, that local version never is less than txCutVer. Because node must process ConsistentCutVersion before applying message with this version.

...

.

Order of transaction messages

...