Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Initial set of nodes is started, providing the following properties:
    1. A subset of nodes (minimum 1, more can be specified to increase startup reliability) in the initial cluster setup, provided by an IP Finder. Concrete IP Finder implementations are used to obtain the seed members from different environments and are specified either via the configuration or the CLI.
  2. The nodes assemble into a physical topology using a network discovery protocol (e.g. SWIM) and the provided seed members.
  3. An init command is sent by a user to a single node in the cluster, providing the following information:
    1. Addresses of the nodes that will host the Meta Storage;
    2. Addresses of the nodes that will comprise the Cluster Management Group. It is possible for both of these address sets to be the same.
  4. The node, that has received the command, propagates it to all members of the physical topology that were specified in the init command. These members should start the corresponding Raft groups and, after the group leaders are elected, the initial node should return a response to the user. In case of errors, Raft groups should be removed and an error response will be returned to the user. If no response has been received, the user should retry sending the command with the same parameters to a different node.  
  5. As soon as the CMG leader is elected, the leader initializes the CMG state by applying a Raft command (let's call it e.g. ClusterInitCommand), which includes:
    1. A generated Cluster Tag;
    2. Ignite product version.
  6. After the command has been applied, the leader sends a message to all nodes in the physical topology, containing the location of the CMG nodes.
  7. Upon receiving the message, each node sends a join request to the CMG leader, which consists of:
    1. Protocol version;
    2. Ignite product version;
  8. Information from the join requests gets validated on the leader and, if the properties are equal to the CMG state, a successful response is sent, containing:
    1. Addresses of the Meta Storage nodes;
    2. Cluster Tag.
      If the properties do not match, an error response is sent and the joining node is rejected.
  9. If a node is validated successfully, the CMG leader issues a Raft command (e.g. AddNodeCommand), which adds the node to the baseline topology.

...

Empty node joins a running cluster

  1. The new node enters the physical topology.
  2. CMG leader discovers the new node and sends a message to it, containing the location of the CMG nodes.
  3. Upon receiving the message, the joining node should execute the validation procedure and be added to the baseline topology, as described by steps 7-9 of the Initial cluster setup section.

Initialized node joins a running cluster

...

TODO

Risks and Assumptions

...

  1. Proposed implementation does not discuss message encryption and security credentials.

...