Motivation

Currently, compute grid functionality is not supported by thin client protocol. 

Description

We can start implementing compute grid functionality with an ability to run already deployed to grid compute tasks by task name (as a first step). In this case, to run some custom task users should deploy class with this task to server manually. The same functionality already implemented for binary-rest clients (GridClientCompute). This IEP describes steps to achieve such a goal and doesn't cover any class deployment functionality or other compute grid functionality (ability to run Callable, Runnable or other jobs).  

Currently, there are some limitations in the thin client protocol exists that prevents to implement the execution of long-running async tasks. The protocol allows only one-way requests (from client to server), but to inform the client about task completion effectively we need some kind of server-initiated messages to the client.

Protocol changes

Notifications

The new ability should be added to notify clients from the server-side. Notification - it's a one-way message from server to client. Notifications format should be compatible with existing response messages format (from server to client), to make it possible to distinguish notifications and responses on the client-side. 

Proposed notification format:

Operation codes

New request type should be added to start a new task:

To notify about task completion following notification code should be used:

To cancel the task existing request type should be used:

OP_COMPUTE_TASK_EXECUTE message format

OP_COMPUTE_TASK_FINISHED message format

Overall workflow

Proposed task execution workflow:

1. Client sends OP_COMPUTE_TASK_EXECUTE request and gets task ID as a response if task was successfully registered.
2. Client has the ability to cancel the task using OP_RESOURCE_CLOSE request and passing task ID as a resource ID to the server.
3. Server notifies the client with OP_COMPUTE_TASK_FINISHED message when the task is completed (successfully, unsuccessfully or if it was canceled by client). Notification should be sent eventually for each task which was successfully registered by OP_COMPUTE_TASK_EXECUTE request.

Feature masks

Currently, protocol versions are used to support backward-compatibility. Sometimes it's not convenient since to support some feature client should increase protocol version and support all other features introduced in all the previous versions of the protocol. This problem can be solved by using feature masks in the new protocol version. Clients and servers should inform each other on handshake about features that they supported.

Proposed new protocol version: 2.0.0

Proposed changes to handshake request and response: 

Features mask - it's an array, where some bit is set if the feature with the corresponding id is supported. 

For compute tasks execution following future should be introduced:

So, bit 0 should be set on features mask if the client supports this feature.

Client-side API (java thin client)

All task-related operations should be started using a new ClientCompute interface. To get an instance of this interface new methods should be added to IgniteClient interface:

    public ClientCompute compute(ClientClusterGroup grp);
    public ClientCluster cluster();

Proposed ClientCompute interface:

public interface ClientCompute {
    public ClientClusterGroup clusterGroup();
    // Sync and async task execution methods.
    public <T, R> R execute(String taskName, @Nullable T arg) throws ClientException, InterruptedException;
    public <T, R> Future<R> executeAsync(String taskName, @Nullable T arg) throws ClientException;
    // ClientCompute modificators.
    public ClientCompute withTimeout(long timeout);
    public ClientCompute withNoFailover();
    public ClientCompute withNoResultCache();
}

Async execution methods should return Future. Using this Future users can get task results or cancel the task.

Risks and Assumptions

If blocking IO is used (as in java thin client, for example) it's impossible to implement async operations without dedicated for each channel thread to process incoming messages. So, the count of threads on the client-side will be increased and can be raised dramatically if partition awareness functionality with a lot of server connections is used.

Discussion Links

http://apache-ignite-developers.2346864.n4.nabble.com/Thin-client-compute-support-td44405.html

Reference Links

// Links to various reference documents, if applicable.

Tickets