Motivation

Implement thin Ignite client in any programming language / platform using a well-defined binary connectiona protocol.

"Thin" here means that we do not start an Ignite node in order to communicate with the cluster, we do not implement discovery/communication spi logic, as opposed to currently available Ignite Client Mode.

Description

TCP Socket

Every Ignite node listens on a TCP port. Thin client implementations connect to any node in the cluster through TCP socket and perform Ignite operations using a well-defined binary protocol.

Server-side connection parameters are defined in ClientConnectorConfiguration class. Default port is 10800. Connector is enabled by default, no configuration changes needed.

Data Format

Byte order is little-endian. Strings are UTF-8 with int prefix (no null terminator). All data types in this proposal are Java-like (byte = 8 bits, short = 2 bytes, int = 4 bytes, long = 8 bytes).

Binary Objects

User data, such as cache keys and values, is represented in Ignite Binary Object format. Binary Object can be a primitive (predefined type) or a complex object.

Primitives (predefined types)

Primitives are represented as a type code + value:

Available primitives are listed below:

Complex Objects

Complex objects consist of a 24-byte header and a set of fields (key-value pairs).

Message format

All messages, request and response, including handshake, start with int message length (excluding these first 4 bytes). E.g. empty message would be represented by 4 zero bytes.

Length is omitted in all message descriptions below for brevity.

Handshake

The first message upon connection establishment must be a handshake. Handshake ensures that client and server versions are compatible.

Client Operations

 Upon successful handshake client can start performing operations by sending a request with specific op code. Each operation has it's own request and response format, with a common header.

Operation codes and request ids are omitted everywhere below for brevity.

OP_CACHE_GET = 1

Retrieves a value from cache by key.

int

OP_CACHE_GET_ALL = 12

Retrieves multiple values from cache given multiple keys. 

int

OP_CACHE_PUT = 4 

Puts a value with a given key to cache (overwriting existing value if any).

int

Empy response.

OP_CACHE_PUT_ALL = 20

Puts multiple key-value pairs to cache (overwriting existing associations if any).

int

Empy response.

OP_CACHE_CONTAINS_KEY = 10

Returns a value indicating whether given key is present in cache.

int

OP_CACHE_CONTAINS_KEYS = 11

Returns a value indicating whether all given keys are present in cache.

int

OP_CACHE_GET_AND_PUT = 13

Puts a value with a given key to cache, returning previous value for that key.

int

OP_CACHE_GET_AND_REPLACE = 14 

Puts a value with a given key to cache, returning previous value for that key, if and only if there is a value currently mapped for that key.

int

OP_CACHE_GET_AND_REMOVE = 15

Removes the cache entry with specified key, returning the value. 

int

OP_CACHE_PUT_IF_ABSENT = 16 

Puts a value with a given key to cache only if the key does not already exist. 

int

OP_CACHE_GET_AND_PUT_IF_ABSENT = 17

Puts a value with a given key to cache only if the key does not already exist.  

int

OP_CACHE_REPLACE = 18 

Puts a value with a given key to cache only if the key already exists. 

int

OP_CACHE_REPLACE_IF_EQUALS = 19 

Puts a value with a given key to cache only if the key already exists and value equals provided value. 

int

OP_CACHE_CLEAR = 21

Clears the cache without notifying listeners or cache writers.

int

Empy response.

OP_CACHE_CLEAR_KEY = 22

Clears the cache key without notifying listeners or cache writers.

int

Empy response.

OP_CACHE_CLEAR_KEYS = 23 

Clears the cache keys without notifying listeners or cache writers. 

int

Empy response.

OP_CACHE_REMOVE_KEY = 24

Removes an entry with a given key, notifying listeners and cache writers.

int

OP_CACHE_REMOVE_IF_EQUALS = 25 

Removes an entry with a given key if provided value is equal to actual value, notifying listeners and cache writers. 

int

OP_CACHE_GET_SIZE = 26 

Gets the cache size.  

int

OP_CACHE_REMOVE_KEYS = 27

Removes entries with given keys, notifying listeners and cache writers. 

int

Empty response.

OP_CACHE_REMOVE_ALL = 28 

Removes all entries from cache, notifying listeners and cache writers. 

int

Empty response.

OP_CACHE_CREATE_WITH_NAME = 29

Creates a cache with a given name. Cache template can be applied if there is '*' in the cache name. Throws exception if a cache with specified name already exists.

string

Empty response.

OP_CACHE_GET_OR_CREATE_WITH_NAME = 30

Creates a cache with a given name. Cache template can be applied if there is '*' in the cache name. Does nothing if the cache exists.

string

Empty response.

OP_CACHE_DESTROY = 31

Destroys cache with a given name.

string

Empty response.

OP_CACHE_GET_NAMES = 32

Gets existing cache names.

Empty request.

string * count

TODO: Other operations

Message Exchange Example

As an example let's see how to connect to Ignite instance on a local machine and retrieve an int value for a given int key, in pseudocode.

var socket = openSocket("127.0.0.1:10800");
 
// Message length
socket.writeInt(8);
 
// Handshake operation
socket.writeByte(1);
 
// Protocol version 1.0.0
socket.writeShort(1);
socket.writeShort(0);
socket.writeShort(0);
 
// Client type: thin client
socket.writeByte(2);
 
// Receive result
var resLen = socket.readInt();
assert resLen == 1;  // Success message length is 1
var res = socket.readByte();
assert res == 1;  // Success code
// END HANDSHAKE
 
 
// Perform OP_CACHE_GET
// Message length
socket.writeInt(20)
 
// Op code = OP_CACHE_GET
socket.writeShort(1);
 
// Request id (can be anything; does not matter for simple blocking socket IO)
var reqId = 1;
socket.writeLong(reqId);
 
// Cache id
var cacheName = "myCache";
var cacheId = javaHashCode(cacheName);
socket.writeInt(cacheId);
 
// Flags = none
socket.writeByte(0);
 
// Cache key 
socket.writeByte(3);  // Integer type code
socket.writeInt(1);  // Our cache value
 
// Read result
var len = socket.readInt();
assert len == 17;   // requestId (8) + status code (4) + int object (1 + 4)
 
var resReqId = socket.readLong();
assert reqId == resReqId;
 
var statusCode = socket.readInt();
assert statusCode == 0;  // Success
 
var resTypeCode = socket.readByte();
assert restypeCode == 3;  // Integer
 
var res = socket.readInt();
print(res);  // Resulting cache value

Discussion Links

http://apache-ignite-developers.2346864.n4.nabble.com/Thin-client-protocol-message-format-td20300.html

Reference Links

Similar protocols from other vendors:

• https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/
• http://docs.hazelcast.org/docs/protocol/1.0-developer-preview/client-protocol.html

Tickets

See "thin client" component in JIRA