Motivation

The current implementation of Discovery SPI implies association of the cluster nodes in the topology of type "ring".

...

• a connection of a new node in the topology

• node Disconnecting from the topology

• of the order of grease retention, in which nodes are connected to the cluster and disconnect from it

• perhaps through a subsystem send custom messages

• Ability to save node attributes

• Ability to establish authenticator to be validated connected nodes

Description

Glossary

Znode - Any node or ephemeral node.

...

/customEvents - znode to store user messages

Description implementation

In order to eliminate these drawbacks is proposed implementation using ZooKeeper-a service. As a point of connection and synchronization for all server nodes of Apache Ignite cluster is a ZooKeeper .

...

Thus ZooKeeper cluster is to guarantee the storage of the current topology, attribute nodes, the next connection nodes, queues for storing user and service events.

Functional ZooKeeper

ZooKeeper main functionality is to provide for the allocation process through shared hierarchy, service coordination and synchronization.

...

Any client ZooKeeper may be asked to send a one-time or recurring notification when updates to certain znode.For example, when the changed data associated with this znode or znode ceased to exist, or at a given znode created new ( "daughter") znode.  

Features of realization

Zookeeper guarantees the consistency in message handling, but for Apache Ignite is necessary to guarantee the preservation of the sequence processing of events, and at their own level. To do this in Apache Ignite cluster remains the coordinator, whose role performs the very first node (from active now) are connected to the service ZooKeeper.

Connection node to the cluster Apache Ignite

1. The configuration of each server node Apache Ignite specified range of addresses and ports for connection to ZooKeeper cluster.

2. Each server Apache Ignite node is connected to an arbitrary server ZooKeeper.

1. Creates a new entry in the / joinData data connection.

2. Recorded in the ephemeral znode / alive - the list of all active nodes in the topology.

3. It checks whether it is a focal point.

4. Subscribes to notifications on changes in / discoveryEvents

5. subscribes to receive notification of a registered output of the previous node from /alive

3. coordinator, having received notification that a / alive, a new node

1. processes data from / joinData, checking that the node may be included in the topology.

2. Creates event nodeJoin and writes it to / discoveryEvents

4. nodes topology, received notice that / discoveryEvents, a new message is processed topology change.

Disabling a node from the cluster Apache Ignite

1. Coordinator, received notice that from / alive missing node creates event NodeFailEvent and writes it to / discoveryEvents

2. nodes topology, received notice that / discoveryEvents, a new message is processed topology change.

3. Node, connected for the disabled, subscribe to receive notifications of withdrawal of the previous registered site / alive.

Selection Apache Ignite coordinator

1. When disconnecting the coordinator, the node connected to the second cluster, is notified of the registered output of the previous node from / alive

2. becomes the coordinator, changing topology version

3. Processes unprocessed messages / discoveryEvents, to synchronize the global and local topology.

4. Processes the raw messages / customEvents

5. sign up to receive notifications of changes to the / alive

6. sign up to receive notifications of changes in / customEvents

Custom Posts

Discovery the SPI interface allows the user to transmit messages through Discovery subsystem. In the current implementation for the transmission of user messages used znode / customEvent. The basic algorithm implementation is as follows.

...

It should be noted that the mechanism customEvent implemented informing nodes of the early formation of a snapshot. This implies a modification of the message sent by topology. In the current implementation of the communication are immutable, which will require improvements in the mechanism of formation of snapshots.

Risks and Assumptions

TBD

Discussion Links

TBD

Reference Links

TBD

Tickets

...