Page History

...

All deployed services are identified internally by “serviceId” (IgniteUuid). This allows us to build a base for such features as hot redeployment and service versioning. It’s important to have an ability to identify and manage services with the same name, but a different version.

Deployment process

Services deployment process - is a process of change deployed services state.

Unified steps of service deployments process:

...

Service deployment is managed via DiscoverySpi and CommunicationSpi messages.

Users requests (deploy/undeploy) are represented as discovery custom message ServiceChangeBatchRequest which contains collection of actions intended changing services states. The action has to extend ServiceChangeAbstractRequest.

The requests are sent via DiscoverySpi, it allows us to get following guarantees:

• The request will be received by all the nodes in the topology. This allows continuing processing request if the coordinator failed and to obtain all needed metadata by one message to handle the request (service configuration, id, class etc.);

• A strict order of requests, to be able to validate request in case of duplication or conflicts in services configuration;

• The relation (order) between user requests and exchange of node joining process (which is also managed via DiscoverySpi). This allows sending a complete set of existing services metadata (also waiting for deployment) to joining node and not to lose any metadata (IgniteServiceProcessor#registeredServices);

Once the request is received, it is stored in a deployment queue as ServiceDeploymentTask to be processed in a separate thread because of requiring significant time. That means each request will be processed in order of queue.

The deployment queue is managed by ServiceDeploymentManager. Deployment worker (special thread) takes from the queue a deployment task and calls ServiceDeploymentTask#init to start the deployment process:

• The task performs actions of changing services states. In case of deploy request, each node calculates assignments independently using a deterministic function (IgniteServiceProcessor#reassign);

• Deployment results are represented as communication message ServiceSingleNodeDeploymentResultBatch, which is sent to coordinator via CommunicationSpi (p2p) once actions were performed. The message contains deployments errors and count of locally deployed instances of services related to the current deployment process;

...

• The coordinator aggregates the deployment results from the cluster. The result of the whole deployment process is represented as discovery custom message ServiceClusterDeploymentResultBatch which is built and sent to all nodes via DiscoverySpi once all single nodes results were received.

• Each node handles ServiceClusterDeploymentResultBatch, updates deployments

...

• information and complete initiators futures if needed, then finishes the deployment process;

TriggersThe following events cause deployment process:

• Users deploy/undeploy requests;

• Affinity topologies change if affinity services exist;

Topology change events (EVT_NODE_JOINED/LEFT/FAILED);

Example of user’s service deployment request according to unified flow:

1. Initiator sends ServiceChangeBatchRequest request using discovery spi to change service state [deploy, undeploy]. The request will be stored by all nodes in own queue to be processed, event if the coordinator failed;
2. All nodes process tasks from queue, when deployment request received each node calculates a new service’s assignments independently using deterministic assignments function and apply actions (deploy, undeploy) if needed, then node builds single deployments message ServiceSingleNodeDeploymentResult that contains services id and amount of instances were deployed on this single node and sends the message to coordinator (p2p);
3. Once the coordinator receives all single deployments messages then it builds ServiceClusterDeploymentResult that contains services deployments across the cluster and sends the message through disco-spi to be processed by all nodes;

Coordinator change

All nodes handle requests of service’s state changes and put them into deployment queue, but only the coordinator aggregates single deployments messages received from nodes and builds a full deployments to finish exchange. If coordinator left or fail during deployment exchange another server node will become the coordinator and continue the process.

...

Topology/coordinator change

Each topology change event (NODE_JOIN/LEFT/FAILED) causes  deployment causes  deployment task. Assignments will be recalculated and applied for each deployed service if needed.

Services reassignment process takes into account previous assignments to avoid redundant redeploymentsredeployment.

If left/failed node had not deployed any services, then deployment task a deployment task will be finished without sending messageswithout sending messages.

If a coordinator was changed during the service deployment process then all nodes send of ServiceSingleNodeDeploymentResultBatch to a new coordinator which continues the process as usual.

Cluster activation/deactivation

• On deactivation:

• • local services are being undeployed;
  • requests are not handling (including deployment / undeployment);

...