...
As shown on fig. 4, when the primary node fails then one of the backups must become a new primary according to affinity assignment for next topology version. This is happens during counters partition map exchange on PME.
A During PME on counter exchange phase a new primary adjusts it's HWM to max known LWM value and serves as new primary generating next countersfor each cache partition. This guarantees monotonous grow of HWM in the grid and valid generation of counters by new primaries.
It is important to ensure the atomicity of switching the node’s primaries for partitioning in node entry or exit scenarios, primary node for partition on all grid nodes, otherwise the HWM >= LWM invariant will break, which ensures that if at least one stable copy is available, everyone else can restore its integrity from it.
This is achieved due to the fact that transactions on the previous topology are completed before switching (as a phase Atomic switchig is one of the PME protocol ) to a new one and properties, making the situation when two primaries of a node simultaneously exist in a grid for one partition is impossibledifferent grid nodes use different primary node for mapping impossible.
The same issue exist when node joins.