...

On primary data node failure

If primary node fails during update we may apply some strategy and whether retry statement ignoring previous changes (using cid) or rollback tx.

if primary node fails during prepare we check whether partitions have not been lost and continue commit procedure in case all partitions are still available.

On loss of partition

We add a special meta record (let's say TxDataLostRecord) with tx, node id and list of lost partitions.

This means that we cannot cleanout tx info with XID upper the lowest XID from the list.

The transaction during which the partition was lost will be aborted or committed in case PREPARE stage has been already compleeted.

The transaction will be fully finished on partition return or manually (deleting corresponding TxDataLostRecord) in case the partition is lost eternally.

During further rebalances nodes in addition to rows sends TxDataLostRecords and tx state records to finish corresponding transactions properly on lost failed node rejoin.

On tx datanode rejoin

In case there is a TxDataLostRecord for rejoining node, partitions from the record will not be invalidated or rebalanced.

Corresponding transactions are finished on all nodes (using tx->partitions map from tx metadata), TxDataLostRecords are deleted.

Read (getAll and SQL)

Each read operation outside active transaction creates a special read only transaction and uses its tx snapshot for versions filtering.

...