Class Design

Before starting discussing the design of the cluster module, let's review the design of IoTDB single node:

1. IoTDB.java is the entrance of the whole module, and it starts many services like RPC service and Storage Engine.

2. TSServiceImpl.java is the entrance that serves terminal users.

Now, let's turn around to the Cluster module. Our main idea is: add a cluster-layer service to wrap the IoTDB single node instance. So,

1. We still need IoTDB.java as a service, but it is not the entrance anymore.

2. We still keep the RPC interface compatible with the single node version, that is to say, we need to reimplement TSserviceImpl.java.

So,  the main class is:

The RPCService is quite similar with the `RPCService` in the single node module, which initializes the Thrift configurations and then bind a TSIService implementation.

The difference is, we need a new TSIService implementation that is different from TSServiceImpl. The implementation may need to control the Storage Engine and MManageer of IoTDB, so we add IoTDB instance as its attribute. So, it is:

Now, consider a terminal user sends a request to clusterTsServiceImpl, there are 3 routings:

1. It is just a local command, which has nothing to do with the cluster module, then we just need to forward the request to IoTDB instance.

2. It is a command that a quorum nodes in the cluster (which means that the command is a cluster-related metadata) need to know, then we need to forward the request using Raft protocol.

3. It is a command that a data partition group  (which means that the command is a cluster-related data, including the schema and data) need to know, then we need to forward the request using Raft protocol, too. 

To serve 2, we create a class called MetaGroupMember and DataGroupMember. As both of them may need to operate the local IoTDB instance, they all have a pointer to the IoTDB instance.

The MetaGroupMember runs raft protocol for commit operations to its automata (i.e., IoTDB instance), and communicate with other Nodes.

That is to say, each node has another RPC service for serving internal metadata requests among nodes.

So, we need to create a new RPCService called MetaRPCService. When the metaRPCService receives requests, it may operate metadata attributes which are in MetaGroupMember, so we need to add a pointer between them.

Similarly, we need also to create a new RPCService called DataRPCService.

So, the UML class graph is:

Startup Process

A: Start IoTDB instance.

B: Start MetaGroup Member and RPC:

Step 1: MetaGroupMember needs to be created.

Step 2: MetaServiceImpl should be started up for serving others' requests.

Step 3: MetaGroupMember needs to get the correct partitionTable. (That means, the heartbeat has been started and metagroup has begun the election).

Step 4: if step 3 success, done. Others continue to retry.

C: Start DataGroup Member and RPC:

Step 1: DataGroupMember needs to be created and the heartbeat starts.

D: Start RPC Service for terminal users.

Notes:

As there may be inconsistent between two nodes, e.g., node A thinks the timestamp precision as "ms" while node B thinks the timestamp precision as "ns",

each two nodes pair needs to check each other when they first create a communication. 

In detail:

- if Node A does not know all info of Node B, e.g., A only know B's basic info (internal IP and port), then it needs to check B's setting.

- if Node A knows all info of Node B, then even it creates a new connection with Node B, it does not check B's setting.