...

1. The new node enters the physical topology.;
2. CMG leader discovers the new node and sends a message to it, containing the location of the CMG nodes.;
3. Upon receiving the message, the joining node should execute the validation procedure and be added to the baseline topology, as described by steps 7-9 of the Initial cluster setup section.

Initialized node joins a running cluster

1. The new node starts the CMG group server or client, depending on the existing local node configuration;
2. The new node enters the physical topology;
3. If the new node is elected CMG leader, it should use the local CMG state to validate itself, otherwise see point 4.
4. CMG leader discovers the new node and sends a message to it, containing the location of the CMG nodes;
5. Upon receiving the message, the joining node should execute the validation procedure and be added to the baseline topology, as described by steps 7-9 of the Initial cluster setup section.

Implementation details

...

Node start flow

The following changes are proposed to the node start scenario in regards to the changes to the join protocol:

Image Added

Each blue rectangle represents a start of a component, changes are marked in red and notable action points are marked in green.

According to the diagram, the following changes are proposed:

1. RESTManager component is started earlier.
2. CMGManager component, responsible for managing CMG interactions, introduced.
   
3. nodeRecoveryFinished action item introduced. It’s a step within components' start process that denotes that a given node has finished its recovery and is ready to be included in a baseline topology.

RestManager changes

RestManager should be started earlier, since it is required to register REST message handlers early to handle the init command.

Image Added

CMGManager

CMGManager is responsible for interacting with the CMG and should perform the following actions:

1. Launch the local CMG Raft server in case an existing local configuration exists;
2. Register a REST message handler for the init command;
3. Register a message handler that will be listening for messages from the CMG leader and sending a join request.

Changes in API

TODO

Risks and Assumptions

1. Proposed implementation does not discuss message encryption and security credentials;
2. Protocol for nodes leaving the topology is out of scope of this document;
3. Protocol for migrating CMG and Meta Storage to different nodes is out of scope of this document.

Discussion Links

TODO

Reference Links

...