Motivation

Implement thin Ignite client in any programming language / platform using a well-defined binary connectiona protocol.

"Thin" here means that we do not start an Ignite node in order to communicate with the cluster, we do not implement discovery/communication spi logic, as opposed to currently available Ignite Client Mode.

Description

TCP Socket

Every Ignite node listens on a TCP port. Thin client implementations connect to any node in the cluster through TCP socket and perform Ignite operations using a well-defined binary protocol.

Server-side connection parameters are defined in ClientConnectorConfiguration class. Default port is 10800. Connector is enabled by default, no configuration changes needed.

Data Format

Byte order is little-endian. Strings are UTF-8 with int prefix (no null terminator). All data types in this proposal are Java-like (byte = 8 bits, short = 2 bytes, int = 4 bytes, long = 8 bytes).

Binary Objects

User data, such as cache keys and values, is represented in Ignite Binary Object format. Binary Object can be a primitive (predefined type) or a complex object.

Primitives (predefined types)

Primitives are represented as a type code + value:

Available primitives are listed below:

Complex Objects

Complex objects consist of a 24-byte header and a set of fields (key-value pairs).

Message format

All messages, request and response, including handshake, start with int message length (excluding these first 4 bytes). E.g. empty message would be represented by 4 zero bytes.

Length is omitted in all message descriptions below for brevity.

Handshake

The first message upon connection establishment must be a handshake. Handshake ensures that client and server versions are compatible.

Client Operations

 Upon successful handshake client can start performing operations by sending a request with specific op code. Each operation has it's own request and response format, with a common header.

Operation codes and request ids are omitted everywhere below for brevity.

OP_CACHE_GET = 1

Retrieves a value from cache by key.

int

OP_CACHE_GET_ALL = 12

Retrieves multiple values from cache given multiple keys. 

int

OP_CACHE_PUT = 4 

Puts a value with a given key to cache (overwriting existing value if any).

int

Empy response.

OP_CACHE_PUT_ALL = 20

Puts multiple key-value pairs to cache (overwriting existing associations if any).

int

Empy response.

OP_CACHE_CONTAINS_KEY = 10

Returns a value indicating whether given key is present in cache.

int

OP_CACHE_CONTAINS_KEYS = 11

Returns a value indicating whether all given keys are present in cache.

int

OP_CACHE_GET_AND_PUT = 13

Puts a value with a given key to cache, returning previous value for that key.

int

OP_CACHE_GET_AND_REPLACE = 14 

Puts a value with a given key to cache, returning previous value for that key, if and only if there is a value currently mapped for that key.

int

OP_CACHE_GET_AND_REMOVE = 15

Removes the cache entry with specified key, returning the value. 

int

OP_CACHE_PUT_IF_ABSENT = 16 

Puts a value with a given key to cache only if the key does not already exist. 

int

OP_CACHE_GET_AND_PUT_IF_ABSENT = 17

Puts a value with a given key to cache only if the key does not already exist.  

int

OP_CACHE_REPLACE = 18 

Puts a value with a given key to cache only if the key already exists. 

int

OP_CACHE_REPLACE_IF_EQUALS = 19 

Puts a value with a given key to cache only if the key already exists and value equals provided value. 

int

OP_CACHE_CLEAR = 21

Clears the cache without notifying listeners or cache writers.

int

Empy response.

OP_CACHE_CLEAR_KEY = 22

Clears the cache key without notifying listeners or cache writers.

int

Empy response.

OP_CACHE_CLEAR_KEYS = 23 

Clears the cache keys without notifying listeners or cache writers. 

int

Empy response.

OP_CACHE_REMOVE_KEY = 24

Removes an entry with a given key, notifying listeners and cache writers.

int

OP_CACHE_REMOVE_IF_EQUALS = 25 

Removes an entry with a given key if provided value is equal to actual value, notifying listeners and cache writers. 

int

OP_CACHE_GET_SIZE = 26 

Gets the cache size.  

int

OP_CACHE_REMOVE_KEYS = 27

Removes entries with given keys, notifying listeners and cache writers. 

int

Empty response.

OP_CACHE_REMOVE_ALL = 28 

Removes all entries from cache, notifying listeners and cache writers. 

int

Empty response.

OP_CACHE_CREATE_WITH_NAME = 29

Creates a cache with a given name. Cache template can be applied if there is '*' in the cache name. Throws exception if a cache with specified name already exists.

string

Empty response.

OP_CACHE_GET_OR_CREATE_WITH_NAME = 30

Creates a cache with a given name. Cache template can be applied if there is '*' in the cache name. Does nothing if the cache exists.

string

Empty response.

OP_CACHE_DESTROY = 31

Destroys cache with a given name.

string

Empty response.

OP_CACHE_GET_NAMES = 32

Gets existing cache names.

Empty request.

string * count

OP_CACHE_GET_CONFIGURATION = 33

Gets cache configuration

Cache configuration is sent and received by server in the following format:

OP_CACHE_CREATE_WITH_CONFIGURATION = 34 

Creates cache with provided configuration. Throws an exception if the name is already in use. 

Empty response.

OP_CACHE_GET_OR_CREATE_WITH_CONFIGURATION = 35 

Creates cache with provided configuration. Does nothing if the name is already in use.  

 Empty response.

OP_QUERY_SQL = 36 

Performs SQL query. 

Response includes first page of results.

OP_QUERY_SQL_CURSOR_GET_PAGE = 37 

Retrieves next SQL query cursor page by cursor id from OP_QUERY_SQL. 

OP_RESOURCE_CLOSE = 9

Closes a resource, such as query cursor.

Empty response.

OP_QUERY_SQL_FIELDS = 38

Performs SQL fields query

OP_QUERY_SQL_FIELDS_CURSOR_GET_PAGE = 39

Retrieves next query page.

OP_GET_BINARY_TYPE_NAME = 2

Gets the platform-specific full binary type name by id. For example, .NET and Java can map to the same type Foo, but classes will be Apache.Ignite.Foo in .NET and org.apache.ignite.Foo in Java.

Names are registered with OP_REGISTER_BINARY_TYPE_NAME.

OP_GET_BINARY_TYPE = 3

Gets the binary type information by id.

OP_REGISTER_BINARY_TYPE_NAME = 5

Registers the platform-specific full binary type name by id. For example, .NET and Java can map to the same type Foo, but classes will be Apache.Ignite.Foo in .NET and org.apache.ignite.Foo in Java.

Empty response.

OP_PUT_BINARY_TYPE = 6

Registers binary type information in cluster.

 Empty response.

OP_QUERY_SCAN = 7

Performs scan query.

OP_QUERY_SCAN_CURSOR_GET_PAGE = 8

Retrieves next SQL query cursor page by cursor id from OP_QUERY_SCAN. 

Message Exchange Example

As an example let's see how to connect to Ignite instance on a local machine and retrieve an int value for a given int key, in pseudocode.

var socket = openSocket("127.0.0.1:10800");
 
// Message length
socket.writeInt(8);
 
// Handshake operation
socket.writeByte(1);
 
// Protocol version 1.0.0
socket.writeShort(1);
socket.writeShort(0);
socket.writeShort(0);
 
// Client type: thin client
socket.writeByte(2);
 
// Receive result
var resLen = socket.readInt();
assert resLen == 1;  // Success message length is 1
var res = socket.readByte();
assert res == 1;  // Success code
// END HANDSHAKE
 
 
// Perform OP_CACHE_GET
// Message length
socket.writeInt(20)
 
// Op code = OP_CACHE_GET
socket.writeShort(1);
 
// Request id (can be anything; does not matter for simple blocking socket IO)
var reqId = 1;
socket.writeLong(reqId);
 
// Cache id
var cacheName = "myCache";
var cacheId = javaHashCode(cacheName);
socket.writeInt(cacheId);
 
// Flags = none
socket.writeByte(0);
 
// Cache key 
socket.writeByte(3);  // Integer type code
socket.writeInt(1);  // Our cache value
 
// Read result
var len = socket.readInt();
assert len == 17;   // requestId (8) + status code (4) + int object (1 + 4)
 
var resReqId = socket.readLong();
assert reqId == resReqId;
 
var statusCode = socket.readInt();
assert statusCode == 0;  // Success
 
var resTypeCode = socket.readByte();
assert restypeCode == 3;  // Integer
 
var res = socket.readInt();
print(res);  // Resulting cache value

Discussion Links

http://apache-ignite-developers.2346864.n4.nabble.com/Thin-client-protocol-message-format-td20300.html

Reference Links

Similar protocols from other vendors:

• https://docs.mongodb.com/manual/reference/mongodb-wire-protocol/
• http://docs.hazelcast.org/docs/protocol/1.0-developer-preview/client-protocol.html

Tickets

See "thin client" component in JIRA