Page History

...

Lets assume all locks are aquired and transactions are started to prepare almost simultaneously.

Consider an example of counter status:

lwm = 5, ooo = (6.1) (8.2), hwm = 11

Lwm - low water mark - number of the last applied sequential update.
That is, in this partition all updates from 1 to 5 are present.
Partial - applied out-of-order updates. In the example in the partition there are updates 7,9,10
But skipped updates 6, 8, 11
Hwm = 11 indicates the number of updates made to the partition, some of which are still “in flight”.
highestApplied = 9 the deduced property - an update with the maximum serial number

The diagram shows counter state updates for primary and backup during tx processing.

TODO pic

HWM is increment Hwm is wound only on the primary node to ensure the monotonicity of the counter lwm <= hwm.Due to the fact that it is possible to reorder transactions into one partition, updates can be applied in any order, which is why omissions are possible in the sequence of updates. To do this, a special structure has been introduced into the counter for tracking updates applied in random orderand invariant HWM >= LWM holding.

Update counters are used to determine the state when the backup one partition copy is behind the primaries. A lag can occur, for example, if the node went out under load and while there were no updates to one of the partitions for which it is the owner. If for each partition the number of updates is the same, then the partitions are identical.
If on one of the partitions the counter lags by k, then it means you need to use k lagging updates to synchronize the partitions. Only relevant for persistent mode.
For in-memory, the partition is always restored from scratchother(s).

It is only safe to compare LWMs because of guarantee of presence of sequential updates up to LWM.

Lets assume partition N on node1 have LWM=m and partition N on node2 have LWM=k, m > k. This means partition on node2 is lagging behind node1 and number of missing updates m-k.

Thus, it is extremely important to correctly track update counters, since it depends on them whether a rebalance will be launched thier values are used to calculate when  rebalancing is necessary to launch to restore the integrity of partitions [2].

...