Page History

...

1. Cluster setup - the process of assembling a predefined set of nodes into a cluster and preparing it for the next lifecycle steps;
2. Cluster initialization - according to the Node Lifecycle description, a cluster can exist in a "zombie" state, until an "init" command is sent by the system administrator;
3. Node validation - in order to maintain the cluster configuration in a consistent state, the joining nodes should be compatible with all of the existing cluster nodes. Compatibility is ensured by validating nodes' properties, which may include some local properties (Ignite product version, Cluster Tag), as well as distributed properties from the Meta Storage. NB: it is currently unclear what properties should be retrieved from the Meta Storage, however, it was decided to keep this concept for possible future features implementation.

Terminology

Meta Storage

Meta Storage is a subset of cluster nodes hosting a Raft group responsible for storing  a master copy of cluster metadata.

Cluster Management Group

Cluster Management Group or CMG is a subset of cluster nodes hosting a Raft group. CMG leader is responsible for orchestrating the node join process.

Init command

...

The "init" command moves the a cluster from the "zombie" state into the "active" state. It is supposed to have the following characteristics (note that the "init" command has not been specified at the moment of writing and is out of scope of this document, so all statements are approximate and can change in the future):

...

idle state into the running state.

Initialized and empty nodes

This document uses a notation of "initialized" and "empty" nodes. An initialized node is a node that has received and successfully applied the "init" command sometime in its lifetime and therefore possesses the Cluster Tag and the location of the Meta Storage Topology version . An empty node is a node that has never received nor applied the "init" command and therefore does not possess the aforementioned properties.

Meta Storage Topology Version

Meta Storage Topology Version is a property that should be used to compute the most "recent" state of a given Meta Storage configuration. At the moment of writing, Meta Storage configuration consists of a list of cluster node names that host the Meta Storage Raft group. A possible implementation can be a monotonically increasing counter, which is increased each time this list is updated.

Join Coordinator

Idle and running cluster

An idle cluster is a cluster that has been assembled for the first time and has never received the init command, therefore the Meta Storage of this cluster does not exist.

Empty and initialized node

Every Ignite node starts in the empty state. After joining the cluster and passing the validation step, a node obtains the location of the Meta Storage and transitions to the initialized stateThe node join process is proposed to be made centralized: a single node is granted the role of the Join Coordinator which is responsible of orchestrating the cluster lifecycle and validating new nodes.

Cluster Tag

A cluster tag is a string that uniquely identifies a cluster. It is generated once per cluster and is distributed across the nodes during the node join process. The purpose of a cluster tag is to understand whether a joining node used to be a member of another cluster, in which case its Meta Storage Topology Version is not comparable and the joining node should be rejected. In other words, Cluster Tag, together with the Meta Storage Topology Version, creates a partial ordering that allows to compare different Meta Storage configurations.it should be rejected during

A cluster tag should consist consists of two parts:

1. Human-readable part (Cluster Name): a string property that is set by the system administrator. Its purpose is to make the debugging and error reporting easier.
2. Unique part (Cluster ID): a generated unique string (e.g. a UUID). Its purpose is to ensure that Cluster Tags are different between different clusters.

Physical topology

Physical topology consists of nodes that can communicate with each other on the network level. Nodes enter the topology though a network discovery mechanism, currently provided by the SWIM protocol. However, such nodes may not yet have passed the validation step, so not all nodes from the physical topology can participate in cluster-wide activities. In order to do that, a node must enter the baseline topology.

Baseline topology

Baseline topology consists of nodes that have passed the validation step and are therefore able to participate in cluster-wide activities. Baseline topology is maintained by the Cluster Management GroupDISCUSSION NEEDED: human-readable part can also be generated automatically, similar to Ignite 2.

Implementation details

Join Coordinator election

...