Motivation

...

On top of modified Discovery SPI could be built more effective mechanisms for detecting and mitigating split-brain situations.

High-level plan

Dicsovery SPI improvements

Modification of TcpDiscoverySpi is needed to fulfill the following requirements:

1. No more than 2 cross-DC connections between two datacenters are allowed in the ring.
2. In order to achive the first requirement a join protocol should be modified to support adding a node at an arbitrary point in the ring in contrast with current implementation (new server node is always added to the very end of the ring).
3. To enable the second requirement new node attribute is introduced labeling the node as belonging to a particular DC (e.g. NODE_DC with String values).
4. TcpDiscoveryNode properties order and internalOrder are reworked to protect higher-level code from changed behavior and to preserve as much internal discovery invariants as possible (to avoid throwing away battle-tested code).

At the same time any improvements to TcpDiscoverySpi should not affect single-DC installations (no additional complexity or overhead is added for an existing use-cases).

Risks and Assumptions

Current approach to this IEP introduces components' mostly internal logic modifications, no public API changes or breaking binary compatibility are needed. Protocols stay the same as well with some internal tweaks and possibly some refactoring.

...