THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Page History
| Table of Contents |
|---|
Basic
...
Definitions
Node - individual instance of Ignite, either a server or a client.
...
The main goal of the discovery mechanism is to create a topology of Ignite nodes and build and maintain a consistent in-memory view of it on every node. For example, this view can may contain the number of nodes in the cluster and their orders.
Discovery mechanism is represented by the DiscoverySpi interface, while TcpDiscoverySpi being its default is its default implementation. Other Other implementations, like ZookeeperDiscoverySpi, also exist but are out of the scope of this article.
Exact topology structure is defined by the particular DiscoverySpi implementations, for example, TcpDiscoverySpi defines a ring-shaped topology.
| Info |
|---|
When describing cluster topology we , we are talking about a logical arrangement that only exists on the "discovery" level. For example, when querying data residing in caches, a cluster might use a different topology than described here. |
Ring-
...
Shaped Topology
TcpDiscoverySpi organizes all server nodes of a cluster into a ring-shaped structure where each node can only send discovery messages to a single node (called a "neighbor"). Client nodes stay outside of the ring and are always connected to a single server. This logic is contained inside ServerImpl and ClientImpl classes for server and client nodes respectively.
| Gliffy Diagram | ||||||
|---|---|---|---|---|---|---|
|
Node
...
Join Process
Overview
When a new node starts, it tries to find an existing cluster by probing all addresses provided by the TcpDiscoveryIpFinder. If all addresses are unavailable, the node considers itself as the only node, forms a cluster from itself and becomes the coordinator. Otherwise, it starts the join process.
Node join process consists of several phases:
- Joining node sends a
TcpDiscoveryJoinRequestMessageto a random node in the cluster, which gets forwarded to the coordinator. - Coordinator places the new node in the ring between the last node and itself and propagates the topology changes by sending the
TcpDiscoveryNodeAddedMessageacross the ring. - After
TcpDiscoveryNodeAddedMessagehas been received by all members of the cluster,TcpDiscoveryNodeAddFinishedMessageis sent to finalize the changes.
...
When the coordinator receives the request it validates the message and generates the TcpDiscoveryNodeAddedMessage, if validation a validation was successful (see ServerImpl.RingMessageWorker#processJoinRequestMessage). This message is then sent across the ring.
...
When handling the TcpDiscoveryNodeAddedMessage, every node in the cluster applies joining node discovery data to components, collects its local discovery data, and adds it to the message (see ServerImpl.RingMessageWorker#processNodeAddedMessage for details). The message is then propagated further across the ring by calling ServerImpl.RingMessageWorker#sendMessageAcrossRing.
...
TcpDiscoveryNodeAddFinishedMessage finishes the process of node join. When receiving this message, each node fires the NODE_JOINED event to notify the discovery manager about the new joined node.
| Info | ||
|---|---|---|
| ||
The joining node will send an additional join request if it doesn't receive |
Detecting and
...
Removing Failed Nodes from Topology
Detecting failed nodes is the responsibility of each server node in a cluster. However, detecting failed servers and clients works slightly differently.
Server
...
Nodes Failures
As servers are organized into a ring each node can easily detect failure of its next neighbor node if sending of a discovery message fails.
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
However, the process of removing a failed node from the ring is not that simple and has to be managed by the coordinator. It contains two steps:
...
During the first step, the server that detected that its neighbor has failed adds the broken node's address to a special in-memory structure called failedNodes (see TcpDiscoveryAbstractMessage#failedNodes). This structure is then used to filter out the failed nodes from the list of all nodes in the ring until the second step is complete.
After that, the server tries to establish a connection to the next node in the ring. If the connection is established, this node becomes the new neighbor, otherwise the process gets repeated until a working node is found.
...
The second step starts on the coordinator. When the coordinator receives a TcpDiscoveryNodeFailedMessage it verifies it and sends it across the ring. All servers (including the coordinator) update their local topology versions, remove remove the failed node(s) from the ring structure and failedNodes internal structures (see ServerImpl$RingMessageWorker#processNodeFailedMessage).
Client
...
Nodes Failures
Section patiently waits for someone to contribute contentthe content
Resilience in
...
Presence of Short Network Glitches
Algorithm of restoring ring without failing nodes (that one involving CrossRingMessageSendState and other tricky things and special timeouts) should be covered here in great details (including special timeouts)
Corner
...
Cases with
...
Filtering of
...
Unverified NodeFailed
...
Messages
Section patiently waits for someone to contribute contentthe content
Custom
...
Discovery Messages and
...
Events
Section patiently waits for someone to contribute the content
Message
...
Delivery Guarantees
Section patiently waits for someone to contribute the content
Overview
Content Tools
Apps